Alarm method, system and equipment for personnel intrusion in hoisting working area of automobile crane and medium

Abstract

The invention provides a method, a system, equipment and a medium for alarming personnel invasion in a hoisting working area of an automobile crane, and relates to the technical field of safety guarantee in the construction process of the automobile crane, which comprises the following steps: setting the radius of the hoisting working area of the automobile crane, and acquiring suspension arm state information of the automobile crane in real time; calculating the position of a lifting hook in real time according to the state information of the lifting arm; calculating the position of a lifting working area in real time according to the position of the lifting hook and the radius of the lifting working area; acquiring a monitoring image in the lifting working area in real time; identifying the monitoring image in the lifting working area; and an alarm is given out when a person intrudes into the hoisting working area. The method has the advantages of strong pertinence of monitoring image acquisition, small limitation of use scenes, high working efficiency, strong environmental adaptability and high identification accuracy, and can meet the real-time safety guarantee requirement of the operation of the automobile crane.

Description

Technical Field

[0001] The present invention relates to the technical field of safety assurance during the construction process of a truck crane, and in particular to an alarm method, system, equipment and medium for human intrusion into a hoisting work area of ​​a truck crane. Background Art

[0002] Truck cranes play a vital role in modern engineering construction, widely used in a variety of scenarios, including building construction, port logistics, and energy extraction. Truck crane operations involve complex personnel and equipment situations, placing extremely high safety requirements on lifting operations. Therefore, monitoring the crane's lifting area and providing timely warnings to personnel who intrude are key to ensuring safe lifting operations.

[0003] Prior art often uses a method for monitoring the entire work area to identify and detect intrusions into the lifting area of ​​a truck crane. This method typically involves strategically installing multiple surveillance cameras around the crane's work area. These cameras continuously capture images of the worksite and transmit them to a processing system. The processing system then uses a deep learning algorithm to analyze these images, determining the hook's location, the lifting work area, and any intrusions within the lifting area. If intrusion is detected, an alarm is triggered.

[0004] However, the boom of a truck crane is extremely flexible during operation, with its extension length and rotation angle constantly changing, causing the position of the hook and the boundaries of the lifting work area to also fluctuate. Using a method that monitors the entire work area requires first identifying and determining the position of the hook, then determining the lifting work area and identifying whether there is any human intrusion within the lifting work area. Firstly, this method requires the surveillance camera to continuously monitor the entire truck crane operation site, and the hook must appear in the surveillance image to further determine the lifting work area. When the hook is raised too high or obscured by buildings, it cannot be effectively identified, limiting its use cases. Secondly, the lighting conditions at the truck crane operation site are complex, and interference factors such as sand and dust are often present, affecting the quality of the surveillance image, making it difficult to identify the hook and personnel. The method has poor environmental adaptability, and the monitoring and warning of human intrusion are inefficient, failing to effectively meet the real-time safety requirements of truck crane operations. Summary of the Invention

[0005] In response to the problems in the prior art of using the method of overall monitoring of the working area to identify and alarm human intrusion, which has limited usage scenarios, poor environmental adaptability, and low efficiency in monitoring and warning of human intrusion, the invention provides an alarm method, system, equipment and medium for human intrusion in the lifting work area of ​​a truck crane. The method first determines the hook position and the lifting work area, and then obtains the monitoring image of the lifting work area in a targeted manner. The monitoring image is highly targeted, is less restricted by the usage scenario, has high work efficiency, strong environmental adaptability, and high recognition accuracy, which can effectively meet the real-time safety protection needs of truck crane operations.

[0006] In a first aspect, the present invention provides an alarm method for detecting human intrusion in a lifting work area of ​​a truck crane, the steps comprising:

[0007] S1. Set the radius of the crane's lifting work area. The lifting work area is the ground projection area centered on the hook.

[0008] S2. Real-time acquisition of boom status information of the truck crane;

[0009] S3 calculates the position of the hook in real time based on the boom status information, calculates the position of the hook in real time based on the radius of the lifting work area and the lifting work area, and obtains real-time monitoring images within the lifting work area;

[0010] S4. Identify the surveillance images in the lifting work area, determine whether there is any human intrusion in the lifting work area, and issue an alarm when there is any human intrusion in the lifting work area.

[0011] It should be further explained that the boom status information includes the boom rotation angle, boom elevation angle, boom length, and hook height.

[0012] It should be further explained that, in step S3, two sets of intrusion monitoring devices symmetrically arranged on both sides of the truck crane along the longitudinal center line of the truck crane body are used to obtain monitoring images in the lifting work area in real time.

[0013] It should be further explained that the specific steps of step S3 include:

[0014] S301. A rectangular coordinate system is established on the horizontal plane using the projection point of the crane boom rotation center on the ground as the coordinate origin and the projection of the longitudinal centerline of the crane body on the ground as a coordinate axis;

[0015] S302. Calculate the hook projection coordinates in real time based on the hook's position on the rectangular coordinate system;

[0016] S303 calculates the position of the lifting work area in real time based on the projection coordinates of the hook and the radius of the lifting work area;

[0017] S304. Compare the positional relationship between the hook projection coordinates and the longitudinal centerline of the truck crane body, activate the intrusion monitoring device on the same side as the hook projection coordinates, and obtain real-time monitoring images of the lifting work area.

[0018] It should be further explained that, in step S301 , the projection of the longitudinal centerline of the truck crane body on the ground is used as the x-axis.

[0019] It should be further explained that the calculation formula of the hook projection coordinates (x, y) in step S302 is: ; Where, L is the boom length; α is the boom elevation angle; θ is the boom rotation angle. When calculating θ, the positive direction of the x-axis is used as the starting side, and counterclockwise rotation is the positive direction.

[0020] It should be further explained that step S304 also includes adjusting the monitoring angle of the intrusion monitoring device so that the center of the monitoring image always coincides with the hook projection coordinate point.

[0021] It should be further explained that in step S4, an image-based target detection algorithm is used to determine whether there is a person intruding into the lifting work area. The steps include:

[0022] S401. Collect images of people in different scenes, angles, and lighting conditions, wearing different clothes and in different postures. Use annotation tools to annotate the people in each image, frame the rectangular area where the person is located, and mark the category as "person" to form a labeled data set.

[0023] S402. Divide the labeled dataset into a training set, a validation set, and a test set;

[0024] S403. Input the training set images into a model built based on the object detection algorithm for training. The model calculates the difference between the predicted and annotated boxes through forward propagation, measures the difference using a loss function, and then updates the network weights through backpropagation. This process is repeated until the model's precision and recall on the validation set stabilize.

[0025] S404. Use the trained model to identify surveillance images within the lifting work area. The model extracts image features from the surveillance images and predicts the presence of people within them, outputting a series of detection boxes with coordinates and confidence levels. Each of these boxes is examined. If a box is classified as "person" and its confidence level is above a preset threshold, it is determined that a person has intruded within the lifting work area.

[0026] It should be further explained that image-based target detection algorithms include the R-CNN series, the YOLO series, or SSD.

[0027] It should be further explained that, in step S401 , the personnel images include personnel images collected around the actual lifting operation.

[0028] In another embodiment of the present technical solution, step S4 uses a machine learning feature matching method to determine whether there is a human intrusion in the lifting work area, and the steps include:

[0029] S411 collects human images under different postures, angles, and lighting conditions, extracts the feature vectors corresponding to the human body contour features, skin color features, and action posture features of each human body image, and constructs a general human feature template based on the feature vector statistics;

[0030] S412 sets different scales and positions of sliding windows on the monitoring image, and uses the sliding window to slide pixel by pixel and line by line through the entire hoisting work area in the monitoring image. Each time the slide stops, the feature vector of the image area within the sliding window is extracted;

[0031] S413 uses a distance metric algorithm to calculate the similarity between the feature vector extracted within the sliding window and the human feature template. When the calculated distance value is less than the set threshold, the match is successful.

[0032] S414. When the feature vectors of multiple sliding windows in the surveillance image successfully match the human feature template, and the spatial position and scale relationship of these successfully matched sliding windows conform to the human body structure logic, it is determined that there is a human intrusion in the lifting work area.

[0033] It should be further explained that, in step S4, the alarm includes a sound alarm, a light warning, and a voice prompt message. The sound alarm includes a buzzer or a siren, and the voice prompt message includes a danger notification voice and an evacuation guidance voice.

[0034] In a second aspect, the present invention provides an alarm system for detecting human intrusion in the lifting work area of ​​a truck crane, which is used to implement the above-mentioned alarm method for detecting human intrusion in the lifting work area of ​​a truck crane, comprising:

[0035] The parameter setting module is used to set the radius parameters of the lifting working area of ​​the truck crane and store and manage the radius parameters;

[0036] Boom status acquisition module, used to obtain boom status information of the truck crane;

[0037] The hook position calculation module is used to calculate the position of the hook in real time according to the boom status information, and calculate the position of the hoisting work area in real time according to the radius of the hoisting work area at the position of the hook;

[0038] The monitoring image acquisition module includes two sets of intrusion monitoring devices symmetrically arranged on both sides of the truck crane along the longitudinal center line of the truck crane body, which are used to obtain monitoring images within the lifting work area in real time;

[0039] Image recognition and judgment module, used to identify the monitoring images in the lifting work area and determine whether there is any human intrusion in the lifting work area;

[0040] The alarm module is used to issue an alarm when the image recognition and judgment module determines that there is a person intruding into the lifting work area.

[0041] It should be further explained that the boom state acquisition module includes a boom state collector, which is provided with a CAN communication submodule and a wireless communication submodule. The CAN communication module is connected to the boom state sensor signal, and the wireless communication module is used to send the boom state information;

[0042] The boom status sensor includes a rotation angle sensor installed at the connection between the slewing platform and the fixed part of the truck crane, a boom elevation angle sensor installed at the boom joint, a boom length sensor installed parallel to the telescopic cylinder under the boom, and a hook height sensor installed on the drum of the truck crane.

[0043] It should be further explained that the boom status collector is arranged in the operating room of the truck crane.

[0044] It should be further explained that the intrusion monitoring device includes a binocular camera, and the bottom of the binocular camera is connected to the rotation mechanism.

[0045] It should be further explained that the intrusion monitoring device is electrically connected to a human-machine interface, and the human-machine interface is used to display the monitoring screen of the intrusion monitoring device.

[0046] It should be further explained that the human-machine interface is also used to input the radius parameters of the lifting working area of ​​the truck crane.

[0047] It should be further explained that the human-machine interface is also used to display text alarm information or image alarm information when the image recognition and judgment module determines that there is a person intruding into the lifting work area.

[0048] It should be further explained that the alarm module includes a sound alarm device, a light warning device and a voice prompt device. The sound alarm device is used to emit a buzzer or siren sound, and the voice prompt device is used to emit a danger notification voice and an evacuation guidance voice.

[0049] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is configured to implement the steps of the above-mentioned method for alarming human intrusion in the lifting work area of ​​a truck crane when executing the computer program.

[0050] In a fourth aspect, the present invention provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-mentioned method for alarming human intrusion in the lifting work area of ​​a truck crane.

[0051] The beneficial effects of the present invention are:

[0052] 1. The present invention provides an alarm method for detecting human intrusion in the lifting work area of ​​a truck crane. The method sets the lifting work area radius, obtains boom status information, and first calculates the hook position and the location of the lifting work area based on the boom status information. It then acquires and identifies surveillance images within the lifting work area, determines whether there is human intrusion within the lifting work area, and issues an alarm if such intrusion occurs. This method first determines the hook position and the lifting work area, then specifically acquires surveillance images within the lifting work area, resulting in highly targeted surveillance images.

[0053] 2. The present invention calculates the hook position and defines the lifting work area based on the boom status information, eliminating the need to identify the hook in the surveillance image. Even when the hook is raised to a high height or is obscured, the lifting work area can still be determined and human intrusion can be effectively identified and warned. It is less restricted by usage scenarios and has high work efficiency.

[0054] 3. This invention only needs to recognize surveillance images within the lifting work area, which reduces the possibility of irrelevant environmental factors infiltrating the surveillance images. It only needs to recognize personnel, which simplifies the recognition conditions and further reduces the impact of irrelevant interference. It has strong environmental adaptability and high recognition accuracy, effectively meeting the real-time safety requirements of mobile crane operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solution of the present invention, the following is a brief introduction to the drawings required for the description. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0056] Figure 1 The present invention is a flowchart of a method for alarming personnel intrusion in a lifting work area of ​​a truck crane according to an embodiment of the present invention.

[0057] Figure 2It is a top view of the truck crane and the intrusion monitoring device after a plane rectangular coordinate system is established on a horizontal plane in one embodiment of the present invention.

[0058] Figure 3 The present invention is a schematic block diagram of an alarm system for human intrusion in the lifting work area of ​​a truck crane in one embodiment of the present invention.

[0059] Figure 4 FIG. 1 is a schematic diagram of the hardware structure of an electronic device in one embodiment of the present invention.

[0060] In the figure, 1-truck crane, 2-jib, 3-hoisting work area, 4-first intrusion monitoring device, 5-second intrusion monitoring device. DETAILED DESCRIPTION

[0061] In order to make the purpose, features, and advantages of the present invention more obvious and easy to understand, the technical solutions of the present invention will be clearly and completely described below in conjunction with the drawings in the specific embodiments. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0062] The alarm method for human intrusion in the lifting work area of ​​a truck crane involved in this application is mainly aimed at the technical field of safety assurance during the construction process of a truck crane. The technical solution includes setting the radius of the lifting work area, obtaining boom status information, first calculating the hook position and the position of the lifting work area based on the boom status information, then obtaining and identifying the monitoring image in the lifting work area, judging whether there is human intrusion in the lifting work area, and issuing an alarm when there is human intrusion. Compared with the prior art, the present invention first determines the hook position and the lifting work area, and then obtains the monitoring image in the lifting work area in a targeted manner, and the monitoring image is highly targeted; the hook position is calculated and the lifting work area is defined according to the boom status information, and there is no need to identify the hook in the monitoring image. When the hook is raised to a high height and is blocked, the lifting work area can still be judged and human intrusion behavior can be effectively identified and warned. It is less restricted by the usage scenario and has high work efficiency; only the monitoring image in the lifting work area needs to be identified, and the identification area is small, which reduces the possibility of irrelevant environmental factors mixing into the monitoring image; only personnel need to be identified, and the identification conditions are relatively simple, which can further weaken the influence of irrelevant interference. It has strong environmental adaptability and high identification accuracy, and can effectively meet the real-time safety protection needs of automobile crane operations.

[0063] The alarm method for human intrusion in the lifting work area of ​​a truck crane involved in this application mainly addresses the technical problems of limited usage scenarios, poor environmental adaptability, and low efficiency in monitoring and warning of human intrusion when using the method of overall monitoring of the work area to identify and alarm human intrusion behavior.

[0064] The following describes in detail the method for alerting a crane user of intrusion into the lifting area of ​​a mobile crane, as described herein. Specific details, such as specific system structures and techniques, are provided for illustrative purposes, not for limitation, to facilitate a thorough understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the present application may also be implemented in other embodiments without these specific details.

[0065] In the alarm method for detecting human intrusion in the lifting work area of ​​a truck crane involved in this application, the term "comprising" is used to indicate the presence of the described features, entities, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, entities, steps, operations, elements, components, and / or combinations thereof. The terms "including," "comprising," "having," and their variations all mean "including but not limited to," unless otherwise specifically emphasized.

[0066] To facilitate the clear description of the technical solutions of this application, the words "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. Those skilled in the art will understand that the words "first" and "second" do not limit the quantity or order of execution, and the words "first" and "second" do not necessarily mean different.

[0067] The phrases "one embodiment" or "some embodiments" described in this application mean that the specific features, structures, or characteristics described in the embodiment are included in one or more embodiments of the application. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in other embodiments," etc. that appear in different places in this application do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized.

[0068] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0069] The alarm method for personnel intrusion in the lifting work area of ​​a truck crane provided in an embodiment of the present invention is executed by a computer device. Accordingly, the alarm system for personnel intrusion in the lifting work area of ​​a truck crane runs in the computer device.

[0070] Figure 1 This is a flow chart of a method for alerting a person to intrude into a lifting work area of ​​a truck crane according to an embodiment of the present invention. Figure 1 The execution subject can be an alarm system for personnel intrusion in the lifting work area of ​​a truck crane. According to different needs, the order of the steps in the flow chart can be changed, and some can be omitted.

[0071] like Figure 1 As shown, the alarm method for personnel intrusion in the lifting work area of ​​the truck crane includes:

[0072] Step S1, setting the radius of the lifting work area of ​​the truck crane, where the lifting work area is the ground projection area with the hook as the center.

[0073] Clearly setting the radius of the lifting work area can provide a basis for subsequent monitoring and alarm actions, help concentrate monitoring resources, avoid ineffective monitoring, and improve system monitoring efficiency.

[0074] Step S2: Acquire the boom status information of the truck crane in real time.

[0075] Acquire boom status information in real time to provide basic data for subsequent accurate calculation of hook position and determination of dynamic changes in the lifting work area, thus ensuring the continuity and accuracy of the entire monitoring process.

[0076] In some specific embodiments, the boom status information includes boom rotation angle, boom elevation angle, boom length, and hook height.

[0077] The rich parameter dimensions can make the hook position calculation in subsequent steps more accurate, reduce the positioning deviation of the lifting area caused by missing or inaccurate data, and further improve the monitoring accuracy of the entire alarm system.

[0078] Step S3, calculating the position of the hook in real time according to the boom state information, calculating the position of the hoisting work area in real time according to the position of the hook and the radius of the hoisting work area, and acquiring the monitoring image in the hoisting work area in real time.

[0079] By calculating the location of the lifting work area and obtaining the corresponding monitoring images, the system can focus on key monitoring areas and not miss any potential human intrusion, laying the foundation for accurate identification of human intrusion.

[0080] In some specific embodiments, step S3 uses two sets of intrusion monitoring devices symmetrically arranged on both sides of the truck crane along the longitudinal center line of the truck crane body to obtain monitoring images in the lifting work area in real time.

[0081] Two sets of intrusion monitoring devices are symmetrically arranged along the longitudinal center line of the vehicle body to expand the monitoring field of view, reduce visual blind spots, enhance the all-round monitoring capability of the lifting area, and ensure monitoring without blind spots.

[0082] In some specific embodiments, the specific steps of step S3 include:

[0083] Step S301: Establish a rectangular coordinate system on a horizontal plane with the projection point of the slewing center of the truck crane arm on the ground as the coordinate origin and the projection of the longitudinal centerline of the truck crane body on the ground as a coordinate axis;

[0084] Step S302, calculating the hook projection coordinates on the plane rectangular coordinate system in real time according to the position of the hook;

[0085] Step S303, calculating the position of the lifting work area in real time according to the hook projection coordinates and the radius of the lifting work area;

[0086] Step S304 , comparing the positional relationship between the hook projection coordinates and the longitudinal centerline of the truck crane body, activating the intrusion monitoring device on the same side as the hook projection coordinates, and acquiring a monitoring image of the lifting work area in real time.

[0087] By refining step S3 and optimizing the hoisting area positioning and monitoring image acquisition process, the entire monitoring system can respond more quickly, the acquired monitoring images can better meet actual needs, and the system's practicality and reliability can be enhanced.

[0088] In some specific embodiments, step S301 uses the projection of the longitudinal centerline of the truck crane body on the ground as the x-axis.

[0089] In some specific embodiments, in step S302, the calculation formula of the hook projection coordinate (x, y) is: ; Where, L is the boom length; α is the boom elevation angle; θ is the boom rotation angle. When calculating θ, the positive direction of the x-axis is used as the starting side, and counterclockwise rotation is the positive direction.

[0090] In some specific embodiments, step S304 further includes adjusting the monitoring angle of the intrusion monitoring device so that the center of the monitoring image always coincides with the hook projection coordinate point.

[0091] By dynamically adjusting the monitoring angle, the monitoring image is always centered on the hook projection coordinates. No matter how the hook moves, the key monitoring area is always at the core of the picture, capturing the details of human intrusion to the greatest extent and improving recognition accuracy.

[0092] Step S4, identifying the surveillance image in the hoisting work area, judging whether there is a person intruding in the hoisting work area, and issuing an alarm when there is a person intruding in the hoisting work area.

[0093] Timely identification of human intrusion and issuance of alarms can reduce the probability of accidents and protect the life, health and property safety of on-site personnel to the greatest extent.

[0094] In some specific embodiments, step S4 uses an image-based target detection algorithm to determine whether there is a person intruding into the lifting work area. The specific steps include:

[0095] Step S401: Collect images of people in different scenes, angles, and lighting conditions, wearing different clothes and in different postures. Use annotation tools to annotate the people in each image, frame the rectangular area where the people are located, and mark the category as "person" to form a labeled data set.

[0096] Step S402: Divide the labeled data set into a training set, a validation set, and a test set;

[0097] Step S403: Input the training set images into the model built based on the object detection algorithm for training. The model calculates the difference between the predicted box and the labeled box through forward propagation, measures the size of the difference using the loss function, and updates the network weights through backpropagation. The model is iterated repeatedly until the precision and recall rate indicators of the model on the validation set tend to be stable.

[0098] In step S404, the trained model is used to identify the surveillance images within the lifting work area. The model extracts the image features of the surveillance images, predicts the possible human targets in the images, and outputs a series of detection boxes with coordinates and confidence levels. These detection boxes are checked one by one. If the category of a detection box is determined to be "person" and the confidence level is higher than the preset threshold, it is determined that there is a human intrusion in the lifting work area.

[0099] It adopts an image-based target detection algorithm and leverages the powerful image feature learning capabilities of deep learning to accurately judge human intrusion situations and adapt to complex and changeable lifting scenarios with a low false alarm rate and high recognition accuracy.

[0100] In some specific embodiments, the image-based target detection algorithm includes the R-CNN series, the YOLO series, or SSD.

[0101] Among them, the R-CNN series includes R-CNN (Region-based Convolutional Neural Networks), Fast R-CNN, and Faster R-CNN;

[0102] The YOLO series includes YOLO (You Only Look Once), YOLOv2, YOLOv3, YOLOv4, YOLOv5, YOLOv6, YOLOv7, and YOLOv8.

[0103] Different algorithms have their own advantages and disadvantages in terms of accuracy, speed, and resource consumption, making it easy to flexibly select based on the actual project needs and hardware conditions.

[0104] In some specific embodiments, the personnel images in step S401 include personnel images collected around the actual lifting operation.

[0105] By incorporating images of personnel collected around actual lifting operations, the training data can be closely aligned with real-world application scenarios. The trained model can better identify the actual personnel situations on site, further reducing misjudgments and missed judgments in actual use.

[0106] In some other embodiments, step S4 uses a machine learning feature matching method to determine whether there is a person intruding in the lifting work area, and the steps include:

[0107] Step S411, collecting human body images in different postures, angles, and lighting conditions, extracting feature vectors corresponding to human body contour features, skin color features, and motion posture features of each human body image, counting the feature vectors, and constructing a universal human feature template;

[0108] Step S412: Setting sliding windows of different scales and positions on the monitoring image, and using the sliding windows to slide pixel by pixel and row by row across the entire hoisting work area in the monitoring image. Each time the sliding stops, the feature vector of the image area within the sliding window is extracted;

[0109] Step S413: using a distance measurement algorithm to calculate the similarity between the feature vector extracted in the sliding window and the human feature template, and a match is successful when the calculated distance value is less than a set threshold;

[0110] In step S414, when the feature vectors of multiple sliding windows in the monitoring image successfully match the human feature template, and the spatial position and scale relationship of these successfully matched sliding windows conform to the human body structure logic, it is determined that there is a human intrusion in the lifting work area.

[0111] Using the feature matching method of machine learning, human intrusion is judged from the feature level. It does not rely on large amounts of labeled data to train deep learning models. The computational complexity is low and it can efficiently and accurately identify people even in some scenarios with limited computing power.

[0112] In some specific embodiments, the alarm includes a sound alarm, a light warning, and a voice prompt message. The sound alarm includes a beep or a siren, and the voice prompt message includes a danger notification voice and an evacuation guidance voice.

[0113] A combination of various alarm forms is adopted. The sound alarm can quickly attract the auditory attention of on-site personnel, the light warning plays a reminder role in noisy environments or visual blind spots, and the voice prompt information clearly informs of danger and guides evacuation, ensuring that on-site personnel can respond to the alarm in a timely manner from all directions and angles.

[0114] In a specific embodiment, the alarm method for personnel intrusion in the lifting work area of ​​a truck crane includes:

[0115] Step S1, setting the radius of the lifting working area of ​​the truck crane, where the lifting working area is the ground projection area with the hook as the center;

[0116] Step S2, real-time acquisition of boom status information of the truck crane, the boom status information including boom rotation angle, boom elevation angle, boom length, and hook height;

[0117] Step S3: Calculate the hook position in real time based on the boom status information, calculate the position of the lifting work area in real time based on the hook position and the radius of the lifting work area, and use two sets of intrusion monitoring devices symmetrically arranged on both sides of the truck crane along the longitudinal centerline of the truck crane body to obtain monitoring images of the lifting work area in real time. The intrusion monitoring device on the left side of the truck crane in the forward direction is the first intrusion monitoring device, and the intrusion monitoring device on the right side of the truck crane in the forward direction is the second intrusion monitoring device. The specific steps include:

[0118] Step S301: With the projection point of the slewing center of the truck crane's boom on the ground as the coordinate origin and the projection of the longitudinal centerline of the truck crane body on the ground as the x-axis, a plane rectangular coordinate system is established on the horizontal plane. The quadrant to the right rear of the truck crane's forward direction is defined as the first quadrant. After the plane rectangular coordinate system is established, the top view of the truck crane and the intrusion monitoring device is as shown below. Figure 2 As shown;

[0119] Step S302: Calculate the hook projection coordinates on the plane rectangular coordinate system in real time according to the position of the hook. The calculation formula of the hook projection coordinates (x, y) is: ; Where, L is the boom length; α is the boom elevation angle; θ is the boom rotation angle. When calculating θ, the positive direction of the x-axis is taken as the starting side, and counterclockwise rotation is the positive direction;

[0120] Step S303, calculating the position of the lifting work area in real time according to the hook projection coordinates and the radius of the lifting work area;

[0121] Step S304: When the hook projection coordinates are located in the second or third quadrant, the first intrusion monitoring device is activated to obtain a monitoring image of the lifting work area in real time; when the hook projection coordinates are located in the first or fourth quadrant, the second intrusion monitoring device is activated to obtain a monitoring image of the lifting work area in real time;

[0122] During the process of acquiring the monitoring image of the hoisting work area, the monitoring angle of the first intrusion monitoring device or the second intrusion monitoring device is adjusted so that the center of the monitoring image always coincides with the projection coordinate point of the hook;

[0123] Step S4: Identify the surveillance image within the lifting work area and use an image-based target detection algorithm to determine whether there is a person intruding into the lifting work area. The software used is YOLOv5. When there is a person intruding into the lifting work area, an alarm is issued;

[0124] The steps to determine whether there is any intrusion into the lifting work area include:

[0125] Step S401: Collect images of people in different scenes, angles, and lighting conditions, wearing different clothes and in different postures. Use annotation tools to annotate the people in each image, frame the rectangular area where the people are located, and mark the category as "person" to form a labeled data set.

[0126] Step S402: Divide the labeled data set into a training set, a validation set, and a test set;

[0127] Step S403: Input the training set images into the model built based on the object detection algorithm for training. The model calculates the difference between the predicted box and the labeled box through forward propagation, measures the size of the difference using the loss function, and updates the network weights through backpropagation. The model is iterated repeatedly until the precision and recall rate indicators of the model on the validation set tend to be stable.

[0128] In step S404, the trained model is used to identify surveillance images within the lifting work area. The model extracts image features from the surveillance images and predicts possible human targets within the images, outputting a series of detection boxes with coordinates and confidence levels. These detection boxes are then checked one by one. If a detection box is identified as "person" and its confidence level is above a preset threshold, it is determined that a person has intruded within the lifting work area.

[0129] Alarms include sound alarms, light warnings, and voice prompts. Sound warnings include beeps or sirens, and voice prompts include danger notification voices and evacuation guidance voices.

[0130] The following is an embodiment of the alarm system for personnel intrusion in the lifting work area of ​​a truck crane provided by the embodiments of the present disclosure. The active load reduction optimization system and the alarm method for personnel intrusion in the lifting work area of ​​a truck crane in the above-mentioned embodiments belong to the same inventive concept. For details not fully described in the embodiment of the alarm system for personnel intrusion in the lifting work area of ​​a truck crane, reference can be made to the embodiment of the alarm method for personnel intrusion in the lifting work area of ​​the truck crane in the above-mentioned embodiments.

[0131] A mobile terminal implementing various embodiments of the present invention will now be described with reference to the accompanying drawings. In the subsequent description, suffixes such as "module," "component," or "unit" used to denote components are used solely to facilitate description of the embodiments of the present invention and do not inherently have specific meanings. Therefore, "module" and "component" may be used interchangeably.

[0132] like Figure 3 As shown in the figure, the alarm system for personnel intrusion in the lifting work area of ​​the truck crane includes:

[0133] The parameter setting module is used to set the radius parameters of the lifting working area of ​​the truck crane and store and manage the radius parameters;

[0134] Boom status acquisition module, used to obtain boom status information of the truck crane;

[0135] The hook position calculation module is used to calculate the position of the hook in real time according to the boom status information, and calculate the position of the hoisting work area in real time according to the radius of the hoisting work area at the position of the hook;

[0136] The monitoring image acquisition module includes two sets of intrusion monitoring devices symmetrically arranged on both sides of the truck crane along the longitudinal center line of the truck crane body, which are used to obtain monitoring images within the lifting work area in real time;

[0137] Image recognition and judgment module, used to identify the monitoring images in the lifting work area and determine whether there is any human intrusion in the lifting work area;

[0138] The alarm module is used to issue an alarm when the image recognition and judgment module determines that there is a person intruding into the lifting work area.

[0139] In some specific embodiments, the boom state acquisition module includes a boom state collector, the boom state collector is provided with a CAN communication submodule and a wireless communication submodule, the CAN communication module is connected to the boom state sensor signal, and the wireless communication module is used to send the boom state information;

[0140] The boom status sensor includes a rotation angle sensor installed at the connection between the slewing platform and the fixed part of the truck crane, a boom elevation angle sensor installed at the boom joint, a boom length sensor installed parallel to the telescopic cylinder under the boom, and a hook height sensor installed on the drum of the truck crane.

[0141] In some specific embodiments, the boom status collector is disposed in an operating room of the truck crane.

[0142] Placing the boom status collector in the operation room makes it easier for operators to manage and maintain it. It also reduces the erosion of the collector by the harsh outdoor environment, extends the service life of the equipment, and ensures continuous and stable data collection.

[0143] In some specific embodiments, the intrusion monitoring device includes a binocular camera, and the bottom of the binocular camera is connected to the rotation mechanism.

[0144] The system adopts a combination of binocular camera and slewing mechanism. The binocular camera provides more stereoscopic visual information, and the slewing mechanism allows flexible shooting angles, enhancing the adaptability to complex lifting scenes and obtaining high-quality monitoring images.

[0145] In some specific embodiments, the intrusion monitoring device is electrically connected to a human-machine interface, and the human-machine interface is used to display a monitoring screen of the intrusion monitoring device.

[0146] By connecting the intrusion monitoring device to the human-machine interface, operators can intuitively view the monitoring screen and grasp the dynamics of the lifting area in real time, making it convenient to make timely operational adjustments or emergency decisions.

[0147] In some specific embodiments, the human-machine interface is further used to input the radius parameter of the lifting working area of ​​the truck crane.

[0148] The human-machine interface is equipped with the function of inputting radius parameters. The operator can adjust the lifting area range as needed on the human-machine interface, improving the convenience and flexibility of system operation.

[0149] In some specific embodiments, the human-machine interface is further configured to display text alarm information or image alarm information when the image recognition and judgment module determines that there is a human intrusion into the lifting work area.

[0150] When a person intrudes, the human-machine interface displays text or image alarm information, supplemented by sound and light alarms, allowing operators to more clearly and intuitively confirm dangerous conditions and assist in emergency decision-making.

[0151] In some specific embodiments, the alarm module includes a sound alarm device, a light warning device and a voice prompt device. The sound alarm device is used to emit a buzzer sound or a siren sound, and the voice prompt device is used to emit a danger notification voice and an evacuation guidance voice.

[0152] By subdividing the alarm modules, different alarm devices can output buzzers, sirens, danger notification voices, and evacuation guidance voices respectively. The diversified alarm outputs can adapt to the perception habits of different people and increase the probability of alarms being received.

[0153] The present application also provides an electronic device for implementing various embodiments of the present invention. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor.

[0154] Those skilled in the art will understand that the electronic device structure involved in the embodiments of the present invention does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0155] Figure 4 A schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention.

[0156] The electronic device includes, but is not limited to, components such as a processor and a memory. Those skilled in the art will appreciate that the electronic device structures described in the embodiments of the present invention do not limit the electronic device, and the electronic device may include more or fewer components than shown, or may combine certain components or arrange the components differently.

[0157] In the embodiments of the present invention, electronic devices include, but are not limited to, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of the present application described and / or claimed herein.

[0158] In the embodiment of the present application, the processor can be implemented by using at least one of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a processor, a controller, a microcontroller, a microprocessor, and an electronic unit designed to perform the functions described herein. In some cases, such an embodiment can be implemented in a controller. For software implementation, an embodiment such as a process or function can be implemented with a separate software module that allows the execution of at least one function or operation. The software code can be implemented by a software application (or program) written in any appropriate programming language, and the software code can be stored in a memory and executed by a controller.

[0159] In addition, the electronic device includes some functional modules not shown, which will not be described here.

[0160] Those skilled in the art will appreciate that various aspects of the electronic device provided herein may be implemented as a system, method, or program product. Therefore, various aspects of the present disclosure may be implemented in the following forms: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits," "modules," or "systems."

[0161] This application also provides a storage medium storing a program product capable of implementing an alarm method for detecting human intrusion in the lifting work area of ​​a truck crane. In some possible implementations, various aspects of this disclosure may also be implemented in the form of a program product comprising program code. When the program product is executed on a terminal device, the program code is configured to cause the terminal device to execute the steps described in the "Exemplary Methods" section above according to various exemplary embodiments of the disclosure.

[0162] The storage medium can be any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0163] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for alarming personnel intrusion in the lifting work area of ​​a truck crane, characterized by the following steps: include: S1. Set the radius of the crane's lifting work area. The lifting work area is the ground projection area centered on the hook. S2. Real-time acquisition of boom status information of the truck crane; S3 calculates the position of the hook in real time based on the boom status information, calculates the position of the hook in real time based on the radius of the lifting work area and the lifting work area, and obtains real-time monitoring images within the lifting work area; S4. Identify the surveillance images in the lifting work area, determine whether there is any human intrusion in the lifting work area, and issue an alarm when there is any human intrusion in the lifting work area.

2. The alarm method according to claim 1, wherein: The boom status information includes boom rotation angle, boom elevation angle, boom length, and hook height.

3. The alarm method according to claim 1, wherein: In step S3, two sets of intrusion monitoring devices are symmetrically arranged on both sides of the truck crane along the longitudinal center line of the truck crane body to obtain monitoring images in the lifting work area in real time.

4. The alarm method according to claim 3, wherein: The specific steps of step S3 include: S301. A rectangular coordinate system is established on the horizontal plane using the projection point of the crane boom rotation center on the ground as the coordinate origin and the projection of the longitudinal centerline of the crane body on the ground as a coordinate axis; S302. Calculate the hook projection coordinates in real time based on the hook's position on the rectangular coordinate system; S303 calculates the position of the lifting work area in real time based on the projection coordinates of the hook and the radius of the lifting work area; S304. Compare the positional relationship between the hook projection coordinates and the longitudinal centerline of the truck crane body, activate the intrusion monitoring device on the same side as the hook projection coordinates, and obtain real-time monitoring images of the lifting work area.

5. The alarm method according to claim 4, characterized in that: Step S304 also includes adjusting the monitoring angle of the intrusion monitoring device so that the center of the monitoring image always coincides with the hook projection coordinate point.

6. The alarm method according to claim 1, wherein: In step S4, an image-based target detection algorithm is used to determine whether there is a person intruding into the lifting work area. The steps include: S401 collects images of people wearing different clothes and postures in different scenes, angles, and lighting conditions. Use annotation tools to annotate the people in each image, frame the rectangular area where the human body is located, and mark the category as "person" to form a labeled data set; S402. Divide the labeled dataset into a training set, a validation set, and a test set; S403. Input the training set images into a model built based on the object detection algorithm for training. The model calculates the difference between the predicted and annotated boxes through forward propagation, measures the difference using a loss function, and then updates the network weights through backpropagation. This process is repeated until the model's precision and recall on the validation set stabilize. S404. Use the trained model to identify surveillance images within the lifting work area. The model extracts image features from the surveillance images and predicts the presence of people within them. The model then outputs a series of detection boxes with coordinates and confidence levels. The model then examines each of these boxes. If a box is identified as "person" and its confidence level exceeds a preset threshold, it is determined that a person has intruded within the lifting work area.

7. The alarm method according to claim 6, wherein: Image-based target detection algorithms include the R-CNN series, the YOLO series, or SSD.

8. The alarm method according to claim 6, wherein: In step S401 , the personnel images include personnel images collected around the actual lifting operation.

9. The alarm method according to claim 1, wherein: Step S4 uses a machine learning feature matching method to determine whether there is a person intruding into the lifting work area. The steps include: S411 collects human images under different postures, angles, and lighting conditions, extracts the feature vectors corresponding to the human body contour features, skin color features, and action posture features of each human body image, and constructs a general human feature template based on the feature vector statistics; S412 sets different scales and positions of sliding windows on the monitoring image, and uses the sliding window to slide pixel by pixel and line by line through the entire hoisting work area in the monitoring image. Each time the slide stops, the feature vector of the image area within the sliding window is extracted; S413 uses a distance metric algorithm to calculate the similarity between the feature vector extracted within the sliding window and the human feature template. When the calculated distance value is less than the set threshold, the match is successful. S414. When the feature vectors of multiple sliding windows in the surveillance image successfully match the human feature template, and the spatial position and scale relationship of these successfully matched sliding windows conform to the human body structure logic, it is determined that there is a human intrusion in the lifting work area.

10. The alarm method according to claim 1, wherein: In step S4, the alarm includes a sound alarm, a light warning, and a voice prompt message. The sound alarm includes a buzzer or a siren, and the voice prompt message includes a danger notification voice and an evacuation guidance voice.

11. An alarm system for personnel intrusion in the lifting work area of ​​a truck crane, characterized in that: Used to implement the alarm method according to any one of claims 1 to 10, comprising: The parameter setting module is used to set the radius parameters of the lifting working area of ​​the truck crane and store and manage the radius parameters; Boom status acquisition module, used to obtain boom status information of the truck crane; The hook position calculation module is used to calculate the position of the hook in real time according to the boom status information, and calculate the position of the hoisting work area in real time according to the radius of the hoisting work area at the position of the hook; The monitoring image acquisition module includes two sets of intrusion monitoring devices symmetrically arranged on both sides of the truck crane along the longitudinal center line of the truck crane body, which are used to obtain monitoring images within the lifting work area in real time; Image recognition and judgment module, used to identify the monitoring images in the lifting work area and determine whether there is any human intrusion in the lifting work area; The alarm module is used to issue an alarm when the image recognition and judgment module determines that there is a person intruding into the lifting work area.

12. The alarm system according to claim 11, wherein The boom status acquisition module includes a boom status collector, which is provided with a CAN communication submodule and a wireless communication submodule. The CAN communication module is connected to the boom status sensor signal, and the wireless communication module is used to send the boom status information; The boom status sensor includes a rotation angle sensor installed at the connection between the slewing platform and the fixed part of the truck crane, a boom elevation angle sensor installed at the boom joint, a boom length sensor installed parallel to the telescopic cylinder under the boom, and a hook height sensor installed on the drum of the truck crane.

13. The alarm system according to claim 12, wherein: The boom status collector is installed in the operator's room of the truck crane.

14. The alarm system according to claim 11, wherein The intrusion monitoring device comprises a binocular camera, and the bottom of the binocular camera is connected with the rotating mechanism.

15. The alarm system according to claim 14, wherein The intrusion monitoring device is electrically connected to a human-machine interface, and the human-machine interface is used to display the monitoring screen of the intrusion monitoring device.

16. The alarm system according to claim 15, wherein: The human-machine interface is also used to input the radius parameters of the lifting working area of ​​the truck crane.

17. The alarm system according to claim 15, wherein: The human-machine interface is also used to display text alarm information or image alarm information when the image recognition and judgment module determines that there is a person intruding into the lifting work area.

18. The alarm system according to claim 11, wherein The alarm module includes a sound alarm device, a light warning device and a voice prompt device. The sound alarm device is used to emit a buzzer or siren sound, and the voice prompt device is used to emit danger notification voice and evacuation guidance voice.

19. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor is configured to implement the steps of the alarm method according to any one of claims 1 to 10 when executing the computer program.

20. A storage medium, characterized in that The storage medium stores a computer program, which, when executed by a processor, implements the steps of the alarm method according to any one of claims 1 to 10.

Images