Multi-view spatial positioning danger early warning device and method for crane lifting and supporting operation

Abstract

The invention provides a multi-view-angle spatial positioning danger early warning device and method for crane lifting and supporting operation, and relates to the technical field of computer vision. The device comprises a horizontal camera, a vertical camera, a first wireless bridge transmitting end, a second wireless bridge transmitting end, a wireless bridge receiving end, a video server, a video switch, an inference server and an alarm module. A front image and a top image are acquired through a horizontal camera and a vertical camera, and are sent to a reasoning server through a first wireless network bridge sending end, a second wireless network bridge sending end, a wireless network bridge receiving end, a video server and a video switch; and the reasoning server processes the image by adopting a visual method, carries out danger analysis and identification, and controls the alarm module to start and stop according to an analysis result. According to the method and the device, the technical problem of insufficient danger prediction precision caused by a single view angle space error in the prior art is solved, and the technical effect of improving the accuracy and the reliability of danger identification and prediction in the operation process is achieved.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a device and method for early warning of dangers in crane hoisting and supporting operations with multi-view spatial positioning. Background Art

[0002] In the construction and industrial fields, crane lifting and positioning operations are a key link in improving production efficiency, but they are also a high-incidence link for safety accidents. Due to the movement of the crane and the swinging of the cargo, workers are at risk of being hit, and the operator's limited field of vision may lead to misoperation, increasing the possibility of accidents. Traditional methods of early warning of dangers in crane lifting and positioning operations mainly rely on manual monitoring, single sensor systems, and simple logic control systems. However, manual monitoring is limited by people's observation ability and reaction speed, and is prone to omissions and delays; single sensor systems such as lidar, ultrasonic sensors, etc., although they can provide spatial distance and obstacle information to a certain extent, cannot fully cover complex and changing working environments. In addition, simple logic control systems lack flexibility and intelligent judgment capabilities and cannot effectively handle complex situations.

[0003] The development of computer vision technology has provided a new way to solve this problem. With the help of advanced technologies such as deep learning, computer vision can analyze and understand the image information of the work site in real time, thereby effectively identifying potential dangerous conditions. However, due to problems such as feature confusion and target omission caused by the interaction of objects, as well as the spatial errors that may exist in a single perspective, single-perspective computer vision systems are prone to false detection and missed detection in practical applications, affecting the accuracy of early warnings. Summary of the invention

[0004] The present application provides a multi-perspective spatial positioning hazard warning device and method for crane lifting and supporting operations, which solves the technical problem that the single-perspective computer vision system cannot cope with the complex working environment and has low hazard prediction accuracy and reliability, and achieves the technical effect of accurately judging the spatial status and potential hazards and improving the accuracy and reliability of the warning.

[0005] In view of the above problems, on the one hand, the present application provides a multi-perspective spatial positioning driving lifting and supporting operation hazard warning device, the device comprising: a horizontal camera, used to capture the front image; a vertical camera, used to capture the top image; a first wireless bridge transmitting end, connected to the horizontal camera, receiving the front image and sending it; a second wireless bridge transmitting end, connected to the vertical camera, receiving the top image and sending it; a wireless bridge receiving end, connected to the first wireless bridge transmitting end and the second wireless bridge transmitting end, receiving the front image and the top image and sending them; a video server, connected to the wireless bridge receiving end, receiving the front image and the top image and sending them after encoding and decoding; a video switch, connected to the video server, receiving the encoded and decoded front image and the top image and sending them; an inference server, connected to the video switch, receiving the encoded and decoded front image and the top image and performing image processing, obtaining the supporting operation hazard analysis results, and visually displaying them; an alarm module, connected to the inference server, receiving the image processing results to distinguish the alarm.

[0006] On the other hand, the present application also provides a multi-perspective spatial positioning method for hazard warning of crane lifting and supporting operations, the method is applied to a multi-perspective spatial positioning device for hazard warning of crane lifting and supporting operations, the device includes a horizontal camera, a vertical camera, a first wireless bridge transmitting end, a second wireless bridge transmitting end, a wireless bridge receiving end, a video server, a video switch, an inference server, and an alarm module, the inference server includes a multi-perspective spatial positioning algorithm module and an alarm judgment logic algorithm module; the multi-perspective spatial positioning algorithm module includes a parallel data reading module, a detection module and a memory module; the alarm judgment logic algorithm module includes a violation calculation module, a violation storage module, and an output module, the method includes: through the horizontal camera and the vertical camera, a front image and a top image are collected and obtained, through the first The wireless bridge transmitting end, the second wireless bridge transmitting end, the wireless bridge receiving end, the video server and the video switch are sent to the inference server; the front image and the top image are preprocessed by the parallel data reading module to obtain the preprocessed front image and the preprocessed top image, and the preprocessed front image and the preprocessed top image are detected by the detection module and the memory module to obtain the horizontal space information and the vertical space information; the hazard identification of the support operation is performed according to the horizontal space information and the vertical space information by the violation calculation module, and the safety signal or the danger signal is obtained and stored in the violation storage module, and the hazard analysis result of the support operation is obtained and sent to the output module; the output module transmits the hazard analysis result of the support operation to the alarm module, and controls the alarm module to start or close the alarm.

[0007] One or more technical solutions provided in this application have at least the following technical effects or advantages: Through the horizontal camera and the vertical camera, the front image and the top image of the crane lifting and supporting position operation site are collected and obtained, providing comprehensive spatial information, reducing the monitoring blind area, and improving the accuracy of hazard identification. The first wireless bridge transmitting end is connected to the horizontal camera, receiving the front image and sending it; the second wireless bridge transmitting end is connected to the vertical camera, receiving the top image and sending it, ensuring the real-time and stability of data transmission, reducing wiring work, and improving installation flexibility and safety of the operation site. The wireless bridge receiving end is connected to the first wireless bridge transmitting end and the second wireless bridge transmitting end, receiving the front image and the top image and sending it, ensuring data integrity and transmission efficiency. The video server is connected to the wireless bridge receiving end, receiving the front image and the top image and sending it after encoding and decoding processing; the video switch is connected to the video server, receiving the encoded and decoded front image and the top image and sending it, providing a stable data stream, and ensuring the continuity and real-time nature of image analysis. The inference server is connected to the video switch, receives the encoded front image and top image, performs image processing, obtains the hazard analysis results of the support operation, and displays them visually. By executing image processing and hazard warning algorithms, it realizes rapid processing and analysis of image data, and combines horizontal and vertical viewing angles to achieve accurate spatial positioning. The alarm module is connected to the inference server, receives the image processing results, distinguishes alarms, identifies potential dangerous situations and sends warnings based on spatial positioning information, and promptly reminds on-site personnel to take emergency measures to reduce the risk of accidents.

[0008] To summarize, the present application uses multi-view cameras to collect comprehensive spatial information, and utilizes real-time image processing and spatial positioning algorithms to accurately locate the three-dimensional spatial position of the working area. It can quickly, accurately and effectively identify potential dangerous situations in operations in various working environments and immediately issue alarms, thereby shortening emergency response time, improving the accuracy and reliability of hazard identification and prediction, enhancing the safety of crane lifting and positioning operations, and improving work efficiency.

[0009] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 A schematic diagram of the structure of a multi-view spatial positioning driving crane lifting and supporting operation hazard warning device provided in an embodiment of the present application; Figure 2A schematic flow chart of a method for early warning of dangers in crane hoisting and supporting operations with multi-perspective spatial positioning provided in an embodiment of the present application; Figure 3 A schematic diagram of a flow chart of a hazard identification of a lifting operation by a violation calculation module in a multi-view spatial positioning driving lifting operation hazard warning method provided in an embodiment of the present application; Explanation of the accompanying drawings: horizontal camera 10-A, vertical camera 10-B, first wireless bridge transmitting end 20-A, second wireless bridge transmitting end 20-B, wireless bridge receiving end 30, video server 40, video switch 50, inference server 60, alarm module 70. DETAILED DESCRIPTION

[0011] The embodiments of the present application provide a multi-perspective spatial positioning crane lifting operation hazard warning device and method, and accurately judge the spatial status and potential hazards by integrating multi-dimensional visual information, real-time image processing, intelligent algorithms and data analysis, thereby solving the technical problem that the single-perspective computer vision system cannot cope with the complex working environment and has low hazard prediction accuracy and reliability, thereby achieving the technical effect of improving the accuracy and reliability of the warning.

[0012] Example 1: Figure 1 As shown, the embodiment of the present application provides a multi-view spatial positioning driving lifting and supporting operation hazard warning device, the device comprising: The horizontal camera 10-A is used to capture the front image.

[0013] Specifically, the horizontal camera 10-A is a high-resolution network camera with a wide field of view and high-definition image output capability. It is installed on the front of the crane hoisting equipment, usually located in the crane control room or on the column, and collects the front image information of the workers and goods in the hoisting area in the horizontal direction, and transmits the front image information to the wireless bridge transmitter 20-A.

[0014] The vertical camera 10-B is used to capture top surface images.

[0015] Specifically, the vertical camera 10-B is a high-resolution network camera with a narrow field of view and high-definition image output capability. It is installed on the top of the crane lifting equipment to obtain a vertical viewing angle, collect the top surface image information of the workers and goods in the lifting area in a vertical downward direction, and transmit the top surface image information to the wireless bridge transmitter 20-B.

[0016] The first wireless bridge transmitting end 20-A is connected to the horizontal camera 10-A, receives the front image and sends it.

[0017] The second wireless bridge transmitting end 20-B is connected to the vertical camera 10-B, receives the top surface image and sends it.

[0018] Specifically, the first wireless bridge transmitting end 20-A and the second wireless bridge transmitting end 20-B are respectively arranged near the horizontal camera 10-A and the vertical camera 10-B, and are electrically connected to the cameras using PoE to ensure the strength and stability of the wireless signal. The collected front image information and top image information are respectively sent to the wireless bridge receiving end 30 by wireless transmission.

[0019] The wireless bridge receiving end 30 is connected to the first wireless bridge transmitting end 20 -A and the second wireless bridge transmitting end 20 -B, receives the front image and the top image and transmits them.

[0020] Specifically, the wireless bridge receiving end 30 is arranged in the control center or server room of the work site, receives wireless signals from the two wireless bridge transmitting ends, and transmits the collected front image information and top image information to the video server 40 .

[0021] The video server 40 is connected to the wireless bridge receiving end 30, receives the front image and the top image and sends them after encoding and decoding.

[0022] Specifically, the video server 40 is a high-performance video processing server with video encoding and decoding and streaming media transmission capabilities. It is usually located in a server room or control center for centralized management and maintenance, and is connected to the wireless bridge using a network port. It receives the collected front image information and top image information, performs encoding and decoding processing, and then transmits it to the inference server 60 through the video switch 50.

[0023] The video switch 50 is connected to the video server 40, receives and transmits the encoded front image and top image.

[0024] Specifically, the video switch 50 is a network switch with high-bandwidth and low-latency data exchange capabilities. It is connected to the video server 40 and the inference server 60 via a network port, is adjacent to the video server 40, receives processed image data, and forwards it to the inference server 60 to ensure efficient data transmission.

[0025] The inference server 60 is connected to the video switch 50, receives the encoded front image and top image, performs image processing, obtains the hazard analysis result of the positioning operation, and performs visual display.

[0026] Specifically, the inference server 60 is a high-performance computing server equipped with a professional graphics processing unit and sufficient storage space, located in a server room or a control center to centrally process and analyze image data. The inference server 60 receives the encoded and decoded image data, runs the image processing algorithm, analyzes and processes the image data, and visualizes the analysis results.

[0027] The inference server 60 includes a multi-view spatial positioning algorithm module and an alarm judgment logic algorithm module, wherein the multi-view spatial positioning algorithm module includes a parallel data reading module, a detection module and a memory module; the alarm judgment logic algorithm module includes a violation calculation module, a violation storage module and an output module. The parallel data reading module is used to preprocess the front image and the top image to obtain a preprocessed front image and a preprocessed top image. The detection module and the memory module are used to detect the preprocessed front image and the preprocessed top image to obtain horizontal space information and vertical space information. The violation calculation module is used to identify the danger of the support operation according to the horizontal space information and the vertical space information, obtain a safety signal or a danger signal and store it in the violation storage module, and determine the result of the support operation danger analysis and send it to the output module. The violation storage module is used to store a safety signal or a danger signal. The output module is used to transmit the result of the support operation danger analysis to the alarm module 70, and control the alarm module 70 to start or close the alarm.

[0028] For the specific working methods of each module in the above-mentioned inference server 60, please refer to the relevant content in the multi-perspective spatial positioning driving lifting and supporting operation hazard warning method in Example 2 of the present application, which will not be introduced in detail here.

[0029] The alarm module 70 is connected to the inference server 60 and receives the image processing result to determine the alarm.

[0030] Specifically, the alarm module 70 includes an audio splitter and several alarm playing devices; the audio splitter is set in the computer room, and the audio of the inference server 60 is pushed to several alarm playing devices respectively, and its input end is connected to the inference server 60 using a USB adapter 3.5 audio cable; several alarm playing devices are evenly arranged at various key positions on the work site, and are connected to the output end of the audio splitter using a 3.5 audio extension cable, and play in real time according to the pushed audio to ensure that the alarm can be heard or seen by all people on site.

[0031] The specific working process of the multi-view spatial positioning driving and lifting support operation danger warning device described in the embodiment of the present application is as follows: when performing driving and lifting support operations, start this device. First, the horizontal camera 10-A installed on the front of the driving and lifting equipment and the vertical camera 10-B installed on the top of the lifting equipment respectively collect high-definition images of workers and goods in the lifting area in real time from the horizontal direction and the vertical direction, and wirelessly transmit the collected front image and top image through the first wireless bridge transmitting end 20-A and the second wireless bridge transmitting end 20-B connected to each other, ensuring the efficient transmission and stability of the image data. Then, the wireless bridge receiving end 30 located in the control center or the server room receives the collected front image and top image, and sends the data to the video server 40 through the Ethernet cable for encoding and decoding processing. The video server 40 forwards the encoded and decoded image to the inference server 60 through the video switch 50. The inference server 60 receives the encoded and decoded image data, runs the image processing algorithm through the multi-view spatial positioning algorithm module, processes and analyzes the image data, and transmits the image and analysis results to the visualization module for visualization, and then controls the alarm module 70 to start and stop working according to the analysis results through the alarm judgment logic algorithm module. After the alarm module 70 starts working, the audio of the inference server 60 is pushed to several alarm playback devices through the audio splitter for alarm, ensuring that on-site personnel can obtain danger warnings in time and take corresponding safety measures.

[0032] In summary, the multi-perspective spatial positioning driving crane lifting and supporting operation hazard warning device provided in the embodiment of the present application has the following technical effects: The working process of the whole device forms a closed-loop system, which provides a comprehensive, efficient and real-time safety monitoring system for the crane lifting and positioning operation through multi-view image acquisition, wireless image transmission, image data processing and danger warning. From image acquisition to danger warning, each link is closely connected, ensuring the safety and efficiency of the crane lifting and positioning operation, significantly improving the safety management level of industrial operations, and providing more solid safety guarantees for on-site workers.

[0033] Example 2: Figure 2 As shown, the embodiment of the present application provides a multi-perspective spatial positioning driving crane lifting and supporting operation hazard warning method, the method is applied to the multi-perspective spatial positioning driving crane lifting and supporting operation hazard warning device described in Example 1, the method includes: S1: The front image and the top image are acquired through the horizontal camera 10-A and the vertical camera 10-B, and are sent to the inference server 60 through the first wireless bridge transmitting end 20-A, the second wireless bridge transmitting end 20-B, the wireless bridge receiving end 30, the video server 40 and the video switch 50.

[0034] Specifically, first, the horizontal camera 10-A and the vertical camera 10-B respectively collect the front and top images of the crane hoisting operation site. These image data are wirelessly transmitted through the first wireless bridge transmitter 20-A and the second wireless bridge transmitter 20-B. During the transmission process, the data is received and forwarded through the wireless bridge receiver 30. The received image data is transmitted to the video server 40. After encoding and decoding, the video server 40 sends the image data to the video switch 50. The video switch 50 receives the processed image data and forwards it to the inference server 60.

[0035] This step can achieve multi-angle monitoring of the crane lifting operation site, provide comprehensive spatial information, ensure real-time acquisition of the on-site situation, and provide a data basis for subsequent analysis.

[0036] S2: Preprocess the front image and the top image through the parallel data reading module to obtain a preprocessed front image and a preprocessed top image, and detect the preprocessed front image and the preprocessed top image through the detection module and the memory module to obtain horizontal space information and vertical space information.

[0037] Further, in the embodiment S2 of the present application, the front image and the top image are preprocessed by the parallel data reading module to obtain a preprocessed front image and a preprocessed top image, and after the preprocessed front image and the preprocessed top image are detected by the detection module and the memory module, the method further includes: S21: The parallel data reading module deletes the pre-processed front surface image and the pre-processed top surface image read by the detection module processing process.

[0038] S22: The parallel data reading module transmits the latest pre-processed front surface image and the pre-processed top surface image to the detection module for detection.

[0039] Furthermore, Embodiment S2 of the present application also includes: S23: Start the horizontal detection model and the vertical detection model in the detection module to detect and distinguish the preprocessed front image and the preprocessed top image respectively, and judge whether there is a target in the preprocessed front image and the preprocessed top image respectively.

[0040] S24: If yes, then detect and identify the pre-processed front image and the pre-processed top image to obtain horizontal space information and vertical space information, and update the memory module; if no, then output the obtained horizontal space information and vertical space information according to the memory module, wherein the horizontal space information includes horizontal height , horizontal width , horizontal center point position , the vertical space information includes the vertical area , vertical center point position , the subscripts l and p characterize goods and workers.

[0041] Furthermore, the embodiment S24 of the present application also includes: S24-1: If so, the pre-processed front image and the pre-processed top image are detected and identified to obtain horizontal space information and vertical space information, and the horizontal space information and vertical space information are updated and recorded in the horizontal memory list and the vertical memory list in the memory module, wherein the horizontal memory list and the vertical memory list include horizontal space information and vertical space information of multiple frames in which the target is recognized or not recognized.

[0042] S24-2: If not, when the number of frames in which the target is recognized in the horizontal memory list and the vertical memory list is greater than the frame number threshold, output the latest record results of the horizontal memory list and the vertical memory list as the current horizontal space information and vertical space information.

[0043] Specifically, the parallel data reading module is used to read and process multiple data streams at the same time. It can process the image data of the horizontal camera 10-A and the vertical camera 10-B at the same time to improve the data processing efficiency. The detection module is used to analyze and identify the image. It can identify specific objects and dangerous situations in the image. The memory module is used to store and memorize historical data in the historical image processing process for subsequent analysis and comparison.

[0044] First, the parallel data reading module simultaneously reads the front image and the top image from the horizontal camera 10-A and the vertical camera 10-B, and performs preprocessing operations on the read data images, including adjusting the image size, grayscale conversion, image enhancement, image filtering, etc., to improve the image quality. Then the preprocessed image is transmitted to the detection module, which analyzes and recognizes the preprocessed image to identify specific objects and dangerous situations in the image.

[0045] The detection module includes a horizontal detection model and a vertical detection model. The horizontal detection model is used to detect horizontal targets in the preprocessed front image and identify the size and position of the target in the horizontal direction. The vertical detection model is used to detect vertical targets in the preprocessed top image and identify the size and position of the target in the vertical direction. These targets refer to goods and workers in the work area.

[0046] The horizontal detection model is based on the target detection technology in computer vision, and is obtained by collecting a large number of actual front images and annotating goods and workers for training; the vertical detection model is based on the target detection technology in computer vision, and is obtained by collecting a large number of actual top images and annotating goods and workers for training.

[0047] The horizontal detection model and vertical detection model built into the detection module are started. These two models are trained to identify goods or workers in the image. The two models analyze the pre-processed front image and the pre-processed top image respectively to determine whether there is a target. If a target is detected, the model will further identify the specific spatial information of the target, including: the horizontal detection model generates horizontal spatial information of goods and workers based on the front image information. The horizontal spatial information includes the horizontal height. , horizontal width , horizontal center point position , subscript and Represent goods and workers respectively; the vertical detection model generates vertical space information of goods and workers based on the top surface image information. The vertical space information includes the vertical area , vertical center point position , subscript and Represent goods and workers respectively; according to the vertical area of ​​the goods ,pass Generate support area The information will be updated to the horizontal memory list and vertical memory list in the memory module.

[0048] The memory module contains horizontal memory lists and vertical memory lists. The horizontal memory lists and vertical memory lists are used to store and memorize historical data, including horizontal and vertical spatial information of multiple frames in which targets are recognized or not recognized. These lists can help the system build long-term memory of the operating environment for easy backtracking and reference when needed. Among them, the horizontal memory list records the horizontal spatial information generated by the horizontal detection model within the N-frame time period. The horizontal memory list contains , , where 1 represents that the target is detected in this frame, and 0 represents that the target is not detected in this frame; the vertical memory list records the vertical spatial information generated by the vertical detection model within the N-frame time period, and the vertical memory list contains , , where 1 represents that the target is detected in this frame, and 0 represents that the target is not detected in this frame.

[0049] If no target is detected in the preprocessed front image and the preprocessed top image, the horizontal space information and the vertical space information are output according to the historical data in the memory module. The specific process is as follows: If there is no target in the pre-processed front image and the pre-processed top image, the system will search for the most recent record of the target in the horizontal memory list and the vertical memory list, and count the number of frames in which the target is recognized in the horizontal memory list and the vertical memory list. If the number of frames in which the target is recognized in these lists is greater than the frame number threshold, it means that the target has appeared frequently in the past period of time. The system will output the latest record results in the horizontal memory list and the vertical memory list as the current horizontal spatial information and vertical spatial information. In this way, the system can provide effective spatial information even when no new targets appear. For example, if cargo is detected in five consecutive frames of images in the horizontal memory list, the system will output the latest record results, even if the target is not detected in the current image. Among them, the frame number threshold is a preset value used to determine when to extract information from the memory module as the current analysis result.

[0050] After the detection module has processed the image, the parallel data reading module deletes the image information that has been read by the detection module, and transmits the latest front image information and top image information to the detection module, repeating the above image analysis and recognition process.

[0051] Through this parallel processing and memory method, the S2 step can realize the preprocessing and continuous monitoring of the front and top images, thereby obtaining horizontal and vertical spatial information. This information provides the basis for subsequent hazard identification and early warning, and helps to more accurately predict potential dangerous situations.

[0052] S3: Through the violation calculation module, the hazard identification of the support operation is performed according to the horizontal space information and the vertical space information, a safety signal or a danger signal is obtained and stored in the violation storage module, and the hazard analysis result of the support operation is obtained and sent to the output module.

[0053] Further, such as Figure 3 As shown, embodiment S3 of the present application also includes: S31: According to the vertical area ,calculate Generate support area ; S32: According to the vertical area , vertical center point position and support area , determine whether there is a worker in the support area, specifically, when the vertical area , the vertical center point position The area of ​​the support area If the safety signal is not generated, it is determined that there is a worker in the support area, otherwise it is determined that there is a worker in the support area; if not, a safety signal is generated and stored in the violation storage module, wherein the violation storage module includes a preset signal storage length, and when storing the latest safety signal or danger signal, the last signal is deleted; S33: If yes, then according to the horizontal height of the cargo in the horizontal space information Whether it is greater than 0, determine whether there is cargo, if so, extract the highest cargo horizontal space information of the highest cargo, if not, generate a safety signal and store it in the violation storage module; S34: judging whether the worker exists according to the horizontal space information, if yes, traversing the worker's horizontal space information to obtain the worker's horizontal space information, if no, generating a safety signal and storing it in the violation storage module; S35: judging whether there is any illegal supporting behavior according to the area of ​​the supporting area, the highest cargo horizontal space information and the worker horizontal space information, and if so, generating a danger signal and storing it in the violation storage module; if not, generating a safety signal and storing it in the violation storage module; S36: According to the signal storage result in the violation storage module, the hazard analysis result of the support operation is determined and sent to the output module.

[0054] Preferably, Figure 3 As shown, the embodiment S36 of the present application also includes: Determine whether the number of danger signals in the illegal storage module is greater than or equal to the danger signal number threshold. If so, generate a danger analysis result of the support operation with an alarm activated; if not, generate a danger analysis result of the support operation without an alarm activated.

[0055] Specifically, the violation calculation module is a software module for analyzing and calculating potential violations, and it identifies the hazards of the support operation based on horizontal and vertical spatial information. The violation storage module is a data storage unit for storing the evaluation results generated by the violation calculation module, which is convenient for subsequent query and analysis. The output module is an interface for receiving analysis results and passing them to other system components or operators.

[0056] The violation calculation module receives horizontal and vertical spatial information from the detection module. This information describes in detail the location and size of objects in the work area. The violation calculation module uses preset safety rules and algorithms to analyze this information in real time to identify any possible violations of safe operating procedures. If the analysis results show that the operation complies with safety regulations, the violation calculation module will generate a safety signal and store it in the violation storage module. This means that the current operation is safe and no additional precautions are required. On the contrary, if the analysis reveals potential dangers, the system will generate a danger signal and also record it in the violation storage module. The violation storage module has a fixed preset signal storage length. Each update stores the latest signal content at the front and removes the last signal content. Among them, the safety signal or danger signal represents the evaluation result of the system. The safety signal indicates that the operation is in a safe state, while the danger signal indicates that there is a potential risk. The final output of the violation calculation module is a detailed hazard analysis result of the support operation, which describes the safety status of the current working environment. This result is sent to the output module. If the hazard analysis result of the support operation is a danger signal, the output module triggers the alarm system to remind the operator to pay attention to the potential risk.

[0057] The specific process is as follows: First, according to the vertical area in the vertical space information Calculate the area of ​​the support area ,in Then, according to the vertical area , vertical center point position and support area , determine whether there is a worker in the support area, specifically, when the vertical area , the vertical center point position The area of ​​the support area If yes, it is determined that there is a worker in the support area, otherwise no; if no, a safety signal is generated and stored in the violation storage module.

[0058] If a worker is detected in the support area, the horizontal space information is further checked to determine whether there is cargo. If there is cargo, the horizontal space information of the highest cargo is extracted; if there is no cargo, a safety signal is generated again and stored.

[0059] Subsequently, it is checked again whether there is a worker according to the horizontal space information. If there is no worker, a safety signal is generated and stored. If a worker is detected, its horizontal space information is traversed to obtain the worker's horizontal space information.

[0060] Finally, the comprehensive support area , the highest cargo horizontal space information and the worker horizontal space information, and judge whether there is any illegal positioning behavior. If there is any illegal positioning behavior, a danger signal is generated and stored; if there is no illegal positioning behavior, a safety signal is generated and stored.

[0061] The signal storage results in the violation storage module are used to determine the final support operation hazard analysis results and sent to the output module. A threshold value for the number of danger signals is pre-defined according to the actual operation safety requirements. This threshold value is used to determine the support operation hazard analysis results. The number of danger signals in the violation storage module is counted and compared with the threshold value for the number of danger signals. If the number of danger signals stored in the violation storage module is greater than or equal to the threshold value for the number of danger signals, a support operation hazard analysis result that activates the alarm is generated and sent to the output module. If the number of danger signals stored in the violation storage module is less than the threshold value for the number of danger signals, a support operation hazard analysis result that does not activate the alarm is generated and sent to the output module.

[0062] Preferably, embodiment S35 of the present application further includes: S35-1: When the area of ​​the supporting region is larger than the area threshold of the supporting region, determine whether the worker uses supporting tools for supporting based on the general target detection model. If not, there is illegal supporting behavior.

[0063] S35-2: When the area of ​​the supporting area is smaller than the area threshold of the supporting area, judging whether the first condition is met according to the highest cargo horizontal space information and the worker horizontal space information.

[0064] S35-3: If the first condition is met, determine whether the second condition is met. If the second condition is met, there is no illegal support behavior; otherwise, there is illegal support behavior.

[0065] Specifically, the area threshold of the support area is a value used to determine whether the area of ​​the support area exceeds the safety standard, which is preset according to the safety requirements of the actual operation process. Support tools refer to tools used for support, such as handrails, support frames, etc. The first condition refers to , used to determine whether the worker is in the supporting position state, the second condition refers to , used to determine whether the worker is supported below the waist.

[0066] The specific process of judging whether there is illegal support behavior is as follows: First, compare the area of ​​the support area and the area threshold of the support area. When the area of ​​the support area When it is greater than the area threshold of the supporting area, the general target detection model is used to determine whether the worker uses supporting tools for support. Failure to use supporting tools for support is considered an illegal supporting behavior, and a danger signal is generated and stored in the violation storage module. When it is detected that the worker uses supporting tools for support, a safety signal is generated and stored.

[0067] When the support area Less than the threshold, first determine the formula Is it true? If it is true, the worker is considered to be in the supporting position state, and then the judgment formula Is it established? If established, it is considered that the worker is supported below the waist, and there is no illegal support behavior. A safety signal is generated and stored. When the first condition is not established, that is, the worker is not in the support state, there is no need to judge the second condition, and there is no illegal support behavior. A safety signal is generated and stored. When the first condition is established and the second condition is not established, that is, the worker is in the support state but does not support below the waist, it is considered that there is illegal support behavior, and a danger signal is generated and stored.

[0068] Step S3 performs hazard identification based on horizontal and vertical spatial information, which can realize real-time monitoring and early warning of dangerous situations in the support operation, thereby ensuring the safety and efficiency of the crane lifting support operation.

[0069] S4: The output module transmits the hazard analysis result of the positioning operation to the alarm module 70, and controls the alarm module 70 to start or shut down the alarm.

[0070] Specifically, the alarm module 70 is a component for receiving instructions from the output module and performing actions accordingly. It can start or shut down the alarm system to notify operators or on-site managers of changes in safety conditions. The output module receives the results of the hazard analysis of the support operation from the violation calculation module. This result may be a safe or dangerous state indication. The output module determines whether to activate the alarm module 70 based on this result.

[0071] If the results of the hazard analysis of the support operation show that there is a dangerous situation, that is, the number of danger signals is greater than the threshold value of the number of danger signals, the output module will send an instruction to the alarm module 70 to activate the alarm. The alarm module 70 plays alarm audio, such as a loud siren, through several alarm playback devices evenly distributed in the working area to remind the operators in the area to pay attention to the danger. On the contrary, if the results of the hazard analysis of the support operation show that the working environment is safe, that is, the number of danger signals is less than the threshold value of the number of danger signals, the output module will send an instruction to the alarm module 70 to turn off the alarm. This means that no additional safety measures are required and the operation can continue.

[0072] In the crane lifting and positioning operation, step S1 continuously obtains real-time image information of the operation area, step S2 continuously updates horizontal space information and vertical space information according to the real-time image information, and then step S3 performs positioning operation hazard identification, updates the safety signal and danger signal in the violation storage module, continuously generates the latest positioning operation hazard analysis results and makes judgments, and the output module dynamically adjusts the status of the alarm module 70 to promptly respond to changes in the operation site situation and provide real-time and accurate danger warnings.

[0073] In addition, the output module can also transmit the front image information, top image information and corresponding analysis results to the visualization module, which visualizes the driving support screen and prediction results in real time based on the input. The visualization module is a component for receiving and processing data from the output module and converting these data into a graphical interface display. Its main function is to present complex analysis results and raw image data to users in an intuitive and easy-to-understand way, usually through a computer screen or other display device.

[0074] In summary, the multi-perspective spatial positioning method for early warning of dangers in crane hoisting and supporting operations provided by the embodiments of the present application has the following technical effects: Through the horizontal camera 10-A and the vertical camera 10-B, the front image and the top image are collected and obtained, and sent to the reasoning server 60 through the first wireless bridge transmitting end 20-A, the second wireless bridge transmitting end 20-B, the wireless bridge receiving end 30, the video server 40 and the video switch 50, so that the on-site situation can be obtained in real time, providing a data basis for subsequent analysis. The front image and the top image are preprocessed by the parallel data reading module to obtain the preprocessed front image and the preprocessed top image, providing clearer data for spatial positioning and hazard identification. Through the detection module and the memory module, the preprocessed front image and the preprocessed top image are detected to obtain horizontal space information and vertical space information, and accurate spatial position information of the operation area can be obtained, improving the positioning accuracy and the accuracy of the hazard identification results. Through the violation calculation module, the hazard identification of the support operation is carried out according to the horizontal space information and the vertical space information, and the safety signal or the danger signal is obtained and stored in the violation storage module. The hazard analysis result of the support operation is determined and sent to the output module, and the potential dangerous situation can be identified in time and an early warning is sent to improve the safety of the operation process. The output module transmits the hazard analysis result of the positioning operation to the alarm module 70, controls the alarm module 70 to start or shut down the alarm, and can issue accurate hazard warnings to the operator according to the real-time situation, and quickly respond to potential dangers.

[0075] As described above, the embodiments of the present application can integrate multi-dimensional visual information, realize all-round real-time monitoring of the crane lifting and supporting operations, reduce monitoring blind spots, and combine real-time image processing, computer vision algorithms and data analysis to accurately judge the spatial status and potential dangers. According to the actual situation of the work site, the early warning information is dynamically adjusted, and the safety status of the work area is accurately fed back, which improves the accuracy and reliability of the danger warning of the crane lifting and supporting operations, and ensures the safety and efficiency of the operation process.

[0076] The multi-perspective spatial positioning driving crane lifting support operation hazard warning device disclosed in Example 1 of this specification and the multi-perspective spatial positioning driving crane lifting support operation hazard warning method disclosed in Example 2 correspond to each other, and the two are referenced to each other. Those skilled in the art can refer to the above two embodiments to clearly understand the present application. In order to avoid redundancy in the content of the specification, the relevant parts will not be repeated in detail.

[0077] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multi-view spatial positioning driving crane lifting and supporting operation hazard warning device, characterized in that: The device comprises: A horizontal camera for collecting frontal images; A vertical camera for collecting top surface images; A first wireless bridge transmitting end is connected to the horizontal camera, receives the front image and transmits it; A second wireless bridge transmitting end is connected to the vertical camera, receives the top surface image and transmits it; A wireless bridge receiving end is connected to the first wireless bridge transmitting end and the second wireless bridge transmitting end, receives the front image and the top image and transmits them; A video server is connected to the wireless bridge receiving end, receives the front image and the top image and sends them after encoding and decoding; A video switch connected to the video server receives and sends the encoded front image and top image; An inference server is connected to the video switch, receives the encoded and decoded front image and top image, performs image processing, obtains the hazard analysis result of the positioning operation, and performs visual display; The alarm module is connected to the inference server and receives the image processing result to judge the alarm.

2. A multi-view spatial positioning method for early warning of dangers in crane hoisting and supporting operations, characterized in that: The method is applied to the multi-perspective spatial positioning driving crane lifting and supporting operation hazard warning device of claim 1, wherein the inference server of the device includes a multi-perspective spatial positioning algorithm module and an alarm judgment logic algorithm module; the multi-perspective spatial positioning algorithm module includes a parallel data reading module, a detection module and a memory module; The alarm judgment logic algorithm module includes a violation calculation module, a violation storage module, and an output module. The method includes: The front image and the top image are acquired by the horizontal camera and the vertical camera, and are sent to the inference server through the first wireless bridge transmitting end, the second wireless bridge transmitting end, the wireless bridge receiving end, the video server and the video switch; Preprocessing the front image and the top image by the parallel data reading module to obtain a preprocessed front image and a preprocessed top image, and detecting the preprocessed front image and the preprocessed top image by the detection module and the memory module to obtain horizontal space information and vertical space information; Through the violation calculation module, the hazard identification of the support operation is performed according to the horizontal space information and the vertical space information, a safety signal or a danger signal is obtained and stored in the violation storage module, and the hazard analysis result of the support operation is obtained by discrimination and sent to the output module; The output module transmits the hazard analysis result of the positioning operation to the alarm module, and controls the alarm module to start or shut down the alarm.

3. The multi-view spatial positioning method for early warning of dangers in crane lifting and supporting operations according to claim 2 is characterized in that: The front image and the top image are preprocessed by the parallel data reading module to obtain a preprocessed front image and a preprocessed top image, and the preprocessed front image and the preprocessed top image are detected by the detection module and the memory module, and then the method further includes: The parallel data reading module deletes the pre-processed front surface image and the pre-processed top surface image read by the detection module processing process; The parallel data reading module transmits the latest pre-processed front surface image and pre-processed top surface image to the detection module for detection.

4. The multi-view spatial positioning method for early warning of dangers in crane hoisting and supporting operations according to claim 2 is characterized in that: The pre-processed front image and the pre-processed top image are detected by the detection module and the memory module to obtain horizontal space information and vertical space information, including: Starting a horizontal detection model and a vertical detection model in the detection module to detect and distinguish the preprocessed front image and the preprocessed top image, respectively, and judging whether there is a target in the preprocessed front image and the preprocessed top image; If yes, the pre-processed front image and the pre-processed top image are detected and identified to obtain horizontal space information and vertical space information, and the memory module is updated; if no, the horizontal space information and vertical space information are output according to the memory module, wherein the horizontal space information includes horizontal height , horizontal width , horizontal center point position , the vertical space information includes the vertical area , vertical center point position , the subscripts l and p characterize goods and workers.

5. The multi-view spatial positioning method for early warning of dangers in crane hoisting and supporting operations according to claim 4 is characterized in that: If yes, the pre-processed front image and the pre-processed top image are detected and identified to obtain horizontal space information and vertical space information, and the memory module is updated; if no, the horizontal space information and vertical space information are output according to the memory module, including: If yes, detect and identify the pre-processed front image and the pre-processed top image to obtain horizontal space information and vertical space information, and update and record the horizontal space information and vertical space information into the horizontal memory list and the vertical memory list in the memory module, wherein the horizontal memory list and the vertical memory list include horizontal space information and vertical space information of multiple frames in which the target is recognized or not recognized; If not, when the number of frames in which the target is recognized in the horizontal memory list and the vertical memory list is greater than the frame number threshold, the latest recording results of the horizontal memory list and the vertical memory list are output as the current horizontal space information and vertical space information.

6. The multi-view spatial positioning method for early warning of dangers in crane hoisting and supporting operations according to claim 5 is characterized in that: Through the violation calculation module, the hazard identification of the positioning operation is performed according to the horizontal space information and the vertical space information, and a safety signal or a danger signal is obtained and stored in the violation storage module, including: According to the vertical area ,calculate Generate support area ; According to the vertical area , vertical center point position and support area , determine whether there is a worker in the support area, specifically, when the vertical area , the vertical center point position The area of ​​the support area If the safety signal is not generated, it is determined that there is a worker in the support area, otherwise it is determined that there is a worker in the support area; if not, a safety signal is generated and stored in the violation storage module, wherein the violation storage module includes a preset signal storage length, and when storing the latest safety signal or danger signal, the last signal is deleted; If yes, then according to the horizontal height of the cargo in the horizontal space information Whether it is greater than 0, determine whether there is cargo, if so, extract the highest cargo horizontal space information of the highest cargo, if not, generate a safety signal and store it in the violation storage module; According to the horizontal space information, determine whether the worker exists, if yes, traverse the worker's horizontal space information to obtain the worker's horizontal space information, if no, generate a safety signal and store it in the violation storage module; According to the area of ​​the supporting area, the highest cargo horizontal space information and the worker horizontal space information, it is determined whether there is any illegal supporting behavior, if so, a danger signal is generated and stored in the violation storage module, if not, a safety signal is generated and stored in the violation storage module; According to the signal storage result in the violation storage module, the hazard analysis result of the support operation is obtained and sent to the output module.

7. The multi-view spatial positioning method for early warning of dangers in crane hoisting and supporting operations according to claim 6 is characterized in that: According to the signal storage result in the violation storage module, the hazard analysis result of the support operation is obtained by judging, including: Determine whether the number of danger signals in the illegal storage module is greater than or equal to the danger signal number threshold. If so, generate a danger analysis result of the support operation with an alarm activated; if not, generate a danger analysis result of the support operation without an alarm activated.

8. The multi-view spatial positioning method for early warning of dangers in crane hoisting and supporting operations according to claim 6 is characterized in that: According to the area of ​​the support area, the highest cargo horizontal space information and the worker horizontal space information, it is determined whether there is any illegal support behavior, including: When the area of ​​the supporting area is greater than the area threshold of the supporting area, it is determined whether the worker uses a supporting tool to support the position, and if not, there is an illegal supporting behavior; When the area of ​​the support area is smaller than the area threshold of the support area, judging whether the first condition is met according to the highest cargo horizontal space information and the worker horizontal space information; If the first condition is met, determine whether the second condition is met. If the second condition is met, there is no illegal support behavior; otherwise, there is illegal support behavior.

Images