Work area person recognition method for unmanned loader and work device

By acquiring rear-view camera images from unmanned loaders and combining them with articulated motion to create hazardous areas, and using hysteresis judgment to determine the risk level, the problem of missed and false alarms in personnel identification under remote control mode is solved, and safe and reliable operation control is achieved.

CN123109245APending Publication Date: 2026-11-17SINOHYDRO BUREAU 5
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611151688.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-07-31
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

In remote control mode, existing personnel identification methods for unmanned loaders have problems such as missed detections, false alarms, and high detection rates under dusty or backlit conditions, making it difficult to guarantee safety, especially under normal loading conditions.

Method used

By acquiring images from the loader's rearview camera for personnel detection, and combining the articulation angle and driving status parameters, a danger zone is dynamically generated in the ground coordinate system. Then, an entry and exit judgment with hysteresis is used to determine the risk level of the personnel target and trigger a speed limit or parking request.

Benefits of technology

It enables accurate and stable identification of personnel in the work area under remote control conditions, reduces missed and false alarms, maintains the continuity of operations, and intervenes in a timely manner when the risk level reaches the preset level.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123109245A_ABST
    Figure CN123109245A_ABST
Patent Text Reader

Abstract

The present application relates to the technical field of engineering machinery control, and in particular to a working area personnel identification method of unmanned loader and a working device, comprising: personnel detection is performed on the image collected by the rear-view camera; in the case that a personnel target is detected, the image position thereof is mapped to the ground coordinate system of the loader to obtain the ground position of the personnel; a dangerous area under the ground coordinate system is dynamically generated according to the hinge angle and the driving state parameters; the position relationship between the ground position of the personnel and the boundary of the dangerous area is adopted to determine the risk level by using the entering and exiting determination with hysteresis; when the risk level reaches a preset intervention level, a speed limiting request or a parking request is sent to the vehicle-mounted controller.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of engineering machinery control technology, specifically to a method and device for identifying personnel in the work area of ​​an unmanned loader. Background Technology

[0002] Large-tonnage articulated loaders have long bodies and high hoods, with the bucket and boom obstructing the downward and forward view, and creating large blind spots at the rear and sides. Unmanned loaders typically have two operating modes.

[0003] In fully unmanned driving mode, the vehicle-mounted millimeter-wave radar automatically stops the machine when it detects personnel entering the work area. This process does not rely on human judgment, ensuring safety.

[0004] In remote control mode, one existing approach is to display all camera feeds from all directions on the remote control console screen, requiring the operator to continuously monitor multiple blind spot feeds in addition to the main operating screen. Another approach is to pop up a notification window when a person is detected, based on a fixed alarm area within the image. However, when an articulated loader turns, the front and rear frames shift relative to each other, leading to missed or false alarms in the fixed alarm area. Furthermore, the false alarm rate increases under dusty or backlit conditions during loading operations, which are typical operating conditions for loaders. Summary of the Invention

[0005] In order to solve the above-mentioned technical problems, the present invention provides a method and device for identifying personnel in the working area of ​​an unmanned loader, which realizes real-time updating of the danger zone and determines the risk level. When the risk level reaches the preset level, the machine speed limit or parking is triggered.

[0006] This invention is achieved through the following technical solution:

[0007] A method for identifying personnel in the work area of ​​an unmanned loader, comprising:

[0008] The image captured by the rear-view camera of the loader is acquired, and personnel detection is performed on the image;

[0009] If a person is detected, the image position of the person is mapped to the ground coordinate system of the loader to obtain the ground position of the person.

[0010] Based on the hinge angle and driving status parameters of the loader, the danger zone in the ground coordinate system is dynamically generated;

[0011] Based on the positional relationship between the personnel's ground location and the boundary of the danger zone, an entry and exit decision with hysteresis is used to determine the risk level of the personnel target;

[0012] When the risk level reaches the preset intervention level, a speed limit request or a parking request is sent to the on-board controller of the loader.

[0013] Optionally, the danger zone is the ground corridor that the loader will sweep during the warning time domain;

[0014] A method for dynamically generating the danger zone in the ground coordinate system includes:

[0015] The lateral boundary of the ground corridor is determined according to the hinge angle and the hinge steering kinematics.

[0016] The warning depth of the ground corridor along the driving direction is determined based on the speed of the loader;

[0017] The driving status parameters include vehicle speed and gear position. When the loader is in reverse gear, the ground corridor extends towards the rear of the loader.

[0018] Optionally, the main body of the ground corridor is a ring-shaped sector centered on the turning center of the loader;

[0019] Let the distance from the hinge center of the loader to the front axle be... Distance from the hinge center to the rear axle The filtered hinge angle is Rear axle turning radius satisfy: ;

[0020] The inner radius of the annular sector With outer radius satisfy: ,in, The width of the loader is [the width of the loader]. The rear overhang length is the distance from the center of the rear axle to the rear of the vehicle. For safety margin;

[0021] When the angle is less than a preset threshold, the ground corridor degenerates into a rectangular corridor along the driving direction.

[0022] Optionally, the round-trip delay is obtained based on the timestamp loopback between the remote operator and the loader. Calculate the one-way delay of the remote control link. : ,in, The preset smoothing coefficient;

[0023] Calculate response time : ,in, The preset operator reaction time;

[0024] Calculate the warning depth : ,in, For the vehicle speed, This is the preset minimum static protection depth. This is the maximum braking deceleration of the loader.

[0025] Furthermore, the method for dynamically generating the danger zone in the ground coordinate system also includes: based on the hinge angular velocity Predicting the duration of the prediction The hinge angle after ,in, The hinge angle after filtering;

[0026] The danger zone is determined based on The generated ground corridor and based on The union of the generated ground corridors.

[0027] Optionally, the risk level includes a first level, a second level, and a third level; the method for determining the risk level of the personnel target includes:

[0028] Determine the signed distance from the personnel's ground location to the boundary of the danger zone. , The value is positive within the danger zone;

[0029] continuous Frame satisfies When the target personnel are identified as having entered the danger zone; This is a preset entry threshold;

[0030] continuous Frame satisfies When the time comes, it is determined that the personnel target has left the danger zone; exit threshold. ,in, The preset base hysteresis width, The preset expansion factor, This is the preset maximum width.

[0031] The personnel target is tracked to obtain its ground velocity. This ground velocity is then projected onto the boundary normal of the danger zone to obtain... rate of change over time ;

[0032] when and At that time, the estimated intrusion time ,otherwise It is denoted as infinity;

[0033] The personnel target is located within the surveillance area extended from the danger zone. and If the value exceeds the first time threshold, it is classified as Level 1.

[0034] Less than or equal to the first time threshold, or the entry determination is valid and If the intrusion distance is less than the preset intrusion distance, it is judged as Level 2;

[0035] Greater than or equal to the preset intrusion distance, or If the time threshold is less than or equal to the second time threshold, it is classified as the third level.

[0036] Furthermore, the personnel detection also outputs the visual confidence score of the personnel target, the method including:

[0037] The radar targets detected by the millimeter-wave radar of the loader are acquired, and the radar targets are associated with the personnel targets according to their ground locations;

[0038] When the association is successful, the visual confidence score and the radar confidence score of the radar target are weighted and fused to obtain the fused confidence score. Personnel targets whose fused confidence scores are greater than or equal to a preset confidence threshold are included in the determination of the risk level.

[0039] When the personnel detection fails to detect any personnel and the radar target remains within the danger zone, a prompt is output according to the preset risk level.

[0040] Wherein, the fusion confidence Calculate using the following formula: ,in, For visual confidence, For radar confidence level, As an indicator of image visibility, The preset equilibrium constant, This is the preset lower limit of the weight; This is the amplitude limiting function.

[0041] Preferably, the method further includes:

[0042] When the personnel target is successfully associated with the radar target and the deviation between the visual ranging and the radar radial distance exceeds a preset distance threshold, the radar radial distance is used to replace the distance component while retaining the visual azimuth angle to reconstruct the personnel's ground position.

[0043] When the personnel target is not associated with a radar target, the safety margin of the danger zone is increased: ,in, Based on the basic safety margin, The distance to the target person. The preset maximum elevation difference of the working surface. The installation height of the rearview camera is [height]. This is the preset magnification factor.

[0044] A remote operation device for an unmanned loader includes a loader body, a rear-view camera, a millimeter-wave radar, an articulation angle sensor, an on-board controller, a wireless communication module, and a remote operation terminal;

[0045] The rear-view camera is mounted on the rear frame of the loader body and is used to collect images behind the loader body;

[0046] The detection field of the millimeter-wave radar covers the monitoring area behind the loader body;

[0047] The hinge angle sensor is used to detect the hinge angle of the loader body;

[0048] The vehicle controller is connected to the rear-view camera, the millimeter-wave radar and the articulated angle sensor respectively, and executes the personnel identification method for the unmanned loader's working area as described above;

[0049] The remote operation terminal communicates with the vehicle controller via the wireless communication module to present personnel targets according to risk level.

[0050] This invention dynamically generates hazardous areas through articulated kinematics and determines risk levels by combining adaptive hysteresis judgment, achieving accurate and stable identification of personnel in the work area under remote control conditions. Compared with existing technologies, it has the following advantages:

[0051] By replacing fixed alarm areas with articulated kinematic sweep corridors, the danger zone is automatically covered by the outward expansion area of ​​the drift and the area far away from the side during reversing and turning operations, reducing missed alarms and false alarms. By combining the hysteresis band of articulated angular velocity adaptive with bidirectional asymmetric multi-frame confirmation, the frequency of alarm state switching is reduced when personnel linger near dynamic boundaries. By classifying risk levels, alarms are only interrupted when personnel actually approach or enter the danger zone, and low-risk situations only receive a light warning, thus maintaining the continuity of operations. Attached Figure Description

[0052] The accompanying drawings illustrate exemplary embodiments of the present invention and, together with the description thereof, serve to explain the principles of the invention. These drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, but do not constitute a limitation on the embodiments of the present invention.

[0053] Figure 1 A flowchart illustrating the personnel identification method for the working area of ​​an unmanned loader provided in an embodiment of the present invention;

[0054] Figure 2 This is a schematic diagram of the structure of the remote operation device for unmanned loaders provided in an embodiment of the present invention;

[0055] Figure 3 This is a top-view schematic diagram showing the dynamic generation of the danger zone in an embodiment of the present invention;

[0056] Figure 4 This is a schematic diagram of the entry and exit determination with hysteresis in an embodiment of the present invention. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0058] It should also be noted that, for ease of description, only the parts relevant to the present invention are shown in the accompanying drawings.

[0059] Where there is no conflict, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0060] Example 1

[0061] This embodiment provides a method for personnel identification in the work area of ​​an unmanned loader, applied to an articulated loader in remote control mode. Each step of the method is executed by processing equipment on the loader's on-board side (such as an on-board edge computing module, or a combination of an on-board edge computing module and an on-board controller). Figure 1 As shown, the method includes steps S1 to S5.

[0062] S1. Acquire images captured by the loader's rearview camera and perform personnel detection on the images.

[0063] Detection is performed on each image frame, and the output is a bounding box for the person target. The detection model can be a lightweight single-stage detection model or an end-to-end detection model. The model can be deployed on an embedded computing platform to meet real-time requirements, and image preprocessing, non-maximum suppression, and other steps are implemented in a conventional manner.

[0064] S2. If a personnel target is detected, map the image position of the personnel target to the ground coordinate system of the loader to obtain the personnel ground position.

[0065] If no personnel are detected, this step and subsequent judgment steps are unnecessary, and the update of the danger zone will not be affected and will continue as the vehicle status changes.

[0066] In this embodiment, the mapping is achieved using ground homography transformation: the midpoint of the bottom edge of the detection frame is taken as the foot point. The ground homography matrix calibrated offline Inverse mapping: After normalization, the position of the personnel in the ground coordinate system is obtained. .

[0067] The preferred ground coordinate system is the rear frame ground coordinate system—the rear-view camera is fixed to the rear frame. It does not change with the hinge movement, so the detection, danger zone generation and judgment can be completed in the same coordinate system. It was obtained by offline calibration of ground markers.

[0068] In other embodiments, the mapping from image location to ground location can also be achieved using inverse perspective transformation based on camera intrinsic and extrinsic parameters, or determined by combining measurements from a ranging sensor. Personnel targets are tracked using α-β filtering to obtain their ground velocity estimate for subsequent judgment.

[0069] S3. Based on the loader's articulation angle and driving status parameters, dynamically generate the danger zone in the ground coordinate system.

[0070] The articulation angle is measured by an articulation angle sensor, and the vehicle speed and gear position are read in via the vehicle bus. The danger zone represents the area of ​​ground that the loader will sweep within the warning time domain. The articulation angle determines its lateral geometry, the vehicle speed determines its depth, and the gear position determines its direction of deployment.

[0071] S4. Based on the positional relationship between the personnel's ground location and the boundary of the danger zone, an entry and exit judgment with hysteresis is used to determine the risk level of the personnel target. The entry and exit conditions are asymmetrical, and the exit threshold adapts to the articulation action to suppress repeated reversals of the judgment results near the dynamic boundary.

[0072] S5. When the risk level reaches the preset intervention level, send a speed limit request or parking request to the loader's on-board controller.

[0073] The preset intervention level is usually the highest risk level. After receiving a speed limit request, the vehicle controller will limit the vehicle speed to a safe value (e.g., below 0.5 m / s), and upon receiving a parking request, it will apply the parking brake. This intervention is completed locally on the vehicle side in a closed loop, without going through a remote link, ensuring backup capability in case of network degradation.

[0074] In the above methods, the risk level of personnel can also be used to control how personnel targets are presented on the remote operating terminal, so that the operator is only interrupted when necessary.

[0075] Example 2

[0076] This embodiment, based on Embodiment 1, illustrates the dynamic generation method of the danger zone, such as... Figure 3 As shown.

[0077] The danger zone is the ground corridor that the loader will sweep during the warning time domain, and it is determined by three parts: the lateral boundary determined by the articulated steering kinematics, the warning depth along the direction of travel, and the union formed by the articulated motion prediction.

[0078] First, establish a notation convention: define the hinge angle. For right hinge, For left-hand hinge, the turning radius is as follows: Calculation, corridor geometry The symbol is mirrored horizontally;

[0079] When in reverse, the corridor extends backward and has full depth; when in forward, the entire machine is far from personnel behind, and the rearward depth degenerates into the minimum stationary protection depth. When turning in place (with the vehicle speed close to zero), the main danger zone is the fishtail sector.

[0080] The hinge angle is obtained by first-order low-pass filtering. To filter out vibrations caused by vehicle body vibration, the filter cutoff frequency ranges from 2 to 5 Hz, for example, 2 Hz, 3 Hz, and 5 Hz; the filtered articulated angular velocity is denoted as... .

[0081] The lateral boundary is determined according to the articulated steering kinematics. Let the distances from the articulation center to the front axle and the rear axle be respectively... , The rear axle turning radius meets the following requirements: .

[0082] The main body of the ground corridor is a ring-shaped sector centered on the turning center, i.e., the curved corridor that the vehicle will sweep through, whose inner and outer radii satisfy the following: ,in, The width of the entire machine. The rear overhang length is the distance from the center of the rear axle to the rear of the vehicle. For safety margin, The value range is 0.3-0.8m.

[0083] In the outer radius The item corresponds to the tail sweep radius being greater than half the width of the vehicle body when an articulated vehicle reverses and turns, such as... Figure 3 The maximum hinge angle state is indicated in the diagram. When the angle is less than a preset threshold, the vehicle will travel in a straight line, and the corridor will degenerate into a rectangle to avoid... Numerical divergence; the value range of this angle threshold is 1°-2°.

[0084] The warning depth consists of braking distance and response distance: ,in, For vehicle speed; The maximum braking deceleration has a value ranging from 2 to 4 m / s². The minimum static protection depth ranges from 1.5 to 3 meters. For response time.

[0085] In the basic implementation, This represents the operator's reaction time, with a value ranging from 0.5 to 1.0 seconds.

[0086] In the preferred embodiment, in The one-way delay of the remote control link, measured online, is estimated and smoothed in real time by the timestamp loopback between the remote operator and the loader: , For round-trip delay, smoothing coefficient The value range is 0.8-0.95.

[0087] When network quality deteriorates, the warning depth automatically increases, ensuring that the alarm lead time always matches the actual link conditions under which the operator sees the screen and completes the braking action.

[0088] The danger zone is also superimposed with the predicted union of articulated movements: according to Predicting the duration of the prediction The hinge angle after Danger zone based on The generated corridor and based on The union of the generated corridors allows the danger zone to cover the area to be swept in advance while the direction is being changed. The value range is 0.5-1.5s.

[0089] Example 3

[0090] This embodiment, based on Embodiment 2, illustrates the entry and exit determination with hysteresis, risk classification, and presentation linkage, such as... Figure 4 .

[0091] As the danger zone changes frame by frame with the articulation angle and vehicle speed, the boundary of the alarm area moves. When personnel are near the boundary, the frame-by-frame judgment results repeatedly flip, causing the alarm status to flash, which exacerbates operator distraction. To address this, a judgment with adaptive hysteresis band is designed.

[0092] remember This represents the signed distance from a person's ground location to the boundary of the danger zone; values ​​are positive within the zone.

[0093] Entry requirement is continuous Frame satisfies ;

[0094] Exit criteria require continuous Frame satisfies .

[0095] The value range is 2-4 frames; The value ranges from 5 to 10 frames. (Entry threshold) The value range is 0.1-0.3m.

[0096] The exit threshold expands adaptively with the hinge angular velocity: Basic hysteresis width The value range is 0.2-0.5m; the width factor The value range is 0.5-1.5 m / (rad / s); upper limit of the expansion. The value range is 1.0-1.5m.

[0097] The more violent the hinge action and the faster the boundary movement, the wider the hysteresis band, and the less likely the judgment state is to be overturned by boundary jitter; when the hinge is stationary, the hysteresis band narrows to maintain judgment sensitivity.

[0098] For personnel outside the danger zone, the estimated intrusion time is calculated based on their tracking speed: the ground velocity of the personnel target is projected onto the normal direction of the danger zone boundary to obtain... rate of change over time ;when and hour, Otherwise, record it as infinity.

[0099] The risk level is divided into three levels:

[0100] Level 1 (Potential): Personnel targets are located within the surveillance area. and Greater than the first time threshold The monitoring area is obtained by expanding outward from the danger zone, and the expansion range is 2-3m.

[0101] Second level (close): Or enter into a judgment that is established and Less than the preset intrusion distance .

[0102] Level 3 (Intrusion): ,or Less than or equal to the second time threshold . The value range is 2-4s; The value range is 0.5-1.5s; The value range is 0.5-1.5m.

[0103] The third level is the preset intervention level, which triggers a speed limit request or a parking request in Implementation Example 1.

[0104] The risk level can only be downgraded after the exit criteria are met, to prevent the level itself from fluctuating.

[0105] The risk level is also linked to the way it is presented on the remote control terminal:

[0106] The first level displays only corner markers and directional arrows at the edge of the screen, without pop-ups or interrupting the main operation screen; the second level displays a semi-transparent rearview image on one side of the screen (e.g., the upper right corner of the left screen), with personnel targets outlined in boxes, and the transparency range is 40%-60%; the third level turns the rearview image opaque and enlarges it to the top, while simultaneously providing audio and visual cues. Operators can also view the rearview image with a single button press on the control panel, without being restricted by the level.

[0107] Example 4

[0108] This embodiment, based on the aforementioned embodiments, illustrates the confidence fusion of vision and millimeter-wave radar and the processing of uneven ground.

[0109] The millimeter-wave radar is configured for fully autonomous driving mode. In remote control mode, it participates in personnel detection, provided its detection field of view covers the rear monitoring area. If the original vehicle radar is only forward-facing, a rearward-facing millimeter-wave radar is added to the rear counterweight, keeping the system composition unchanged. Personnel detection outputs not only position but also visual confidence level. Radar targets and personnel targets are associated based on their nearest neighbors on the ground, with the association threshold ranging from 1 to 2 meters.

[0110] After successful association, the fusion confidence level is calculated using the following formula: .

[0111] in, The radar confidence level is linearly mapped from the signal-to-noise ratio of the radar target to the [0,1] interval.

[0112] The image visibility index is determined by the ratio of the image gradient statistics of the personnel target detection box region to the calibrated reference value and truncated to [0,1]. Dust, fog, and backlighting cause a decrease in image gradient energy. Consequently, the gradient decreases. In this embodiment, the image gradient statistics specifically use the mean Sobel gradient of the detection box region.

[0113] Equilibrium constant The value range is 0.3-0.6; lower limit of weight. The value ranges from 0.15 to 0.3, and its function is to prevent the fusion weight from being completely biased towards a single sensor under extreme conditions.

[0114] Determine trigger requirements The confidence level is greater than or equal to a preset confidence threshold, which ranges from 0.45 to 0.6. When there is no continuous visual detection but the radar target remains within the danger zone, a warning is output according to a preset risk level (e.g., level 2) as a fallback for single-source radar detection.

[0115] On uneven working surfaces such as material piles and ramps, where there are differences in ground elevation... The resulting first-order approximation of the ranging error is: ,in For the target distance, The camera installation height is specified; the processing rules are divided into two cases.

[0116] Personnel targets were successfully correlated with radar targets, and the visual ranging distance was equal to the radar radial distance. The deviation exceeds the preset distance threshold At that time, the radar radial range is used to replace the range component, while the visual azimuth angle is preserved. That is, taking in polar coordinates Reconstructing personnel's ground location—radar ranging is unaffected by ground undulations. The value range is 0.4-0.8m.

[0117] When personnel targets are not associated with radar targets, the safety margin of the danger zone is increased based on elevation difference sensitivity: ,in, A maximum elevation difference is preset for the working surface, with a range of 0.3-0.8m; magnification factor. The value range is 1.0-1.5. The farther the target is and the more uneven the ground, the larger the margin. The ranging error is absorbed by the margin, and no false negatives occur.

[0118] For example, if a loader is operating near a material pile, and a worker is standing on a material slope approximately 0.4m high and about 6m away, the radar distance reading is 6.9m, while the radar radial distance is 6.1m. The deviation of 0.8m exceeds the threshold (taken as 0.5m). Reconstruct personnel positions; if the target is not associated with radar in the same scenario, retrieve... m、 m、 m, then m, the safety margin in the direction corresponding to the target is increased accordingly.

[0119] In addition, if the camera video stream is interrupted or there are no valid frames for 2 consecutive seconds, switch to radar as a backup and report the fault to the operator; if the radar has no associated target but the visual confidence is high, proceed according to visual operation with a minimum weight; if the nighttime illumination is insufficient, activate infrared supplementary lighting and implement the operation according to conventional methods.

[0120] Example 5

[0121] This embodiment provides a remote operation device for an unmanned loader, such as Figure 2 As shown, it includes the loader body, rearview camera, millimeter-wave radar, articulation angle sensor, vehicle controller, wireless communication module and remote operation terminal.

[0122] The rear-view camera is mounted on the rear frame of the loader body to capture images behind the loader body;

[0123] The millimeter-wave radar's detection field of view covers the monitoring area behind the loader body;

[0124] The hinge angle sensor is used to detect the hinge angle of the loader body;

[0125] The vehicle controller is connected to the rearview camera, millimeter-wave radar and articulated angle sensor respectively, and executes the personnel identification method in the work area of ​​any of the above embodiments with its internal vehicle edge computing module;

[0126] The remote control terminal communicates with the vehicle controller via a wireless communication module to present personnel targets according to risk level.

[0127] This invention also provides an electronic device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the work area personnel identification method of any of the foregoing embodiments.

[0128] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method for identifying personnel in a work area.

[0129] Without loss of generality, computer-readable media can include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instruction data structures, program modules, or other data. Computer storage media include RAM, ROM, EPROM, EEPROM, flash memory or other solid-state storage technologies, CD-ROM, DVD or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that computer storage media are not limited to the above-mentioned types. The aforementioned system memories and mass storage devices can be collectively referred to as memory.

[0130] A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of any of the methods described above.

[0131] Computer program products include computer programs or instruction sets used to perform specific tasks or achieve specific functions. These programs or instructions are designed to be executed by a processor to implement a series of predefined steps or operations. The program product may be stored in various forms of computer storage media, such as memory, hard disks, solid-state drives, optical discs, or other forms of digital storage devices. It may exist in the form of compiled binary code or in the form of scripts or bytecode that can be executed by an interpreter. Through carefully designed algorithms and logical instructions, the program product enables the processor to process data in a specific order and manner, performing various functions such as data analysis, user interaction, and device control.

[0132] Those skilled in the art should understand that the above embodiments are merely for illustrating the present invention and are not intended to limit the scope of the invention. Those skilled in the art can make other changes or modifications based on the above invention, and these changes or modifications still fall within the scope of the present invention.

Claims

1. A method of identifying personnel in a work area of an unmanned loader, characterized by, include: The image captured by the rear-view camera of the loader is acquired, and personnel detection is performed on the image; If a person is detected, the image position of the person is mapped to the ground coordinate system of the loader to obtain the ground position of the person. Based on the hinge angle and driving status parameters of the loader, the danger zone in the ground coordinate system is dynamically generated; Based on the positional relationship between the personnel's ground location and the boundary of the danger zone, an entry and exit decision with hysteresis is used to determine the risk level of the personnel target; When the risk level reaches the preset intervention level, a speed limit request or a parking request is sent to the on-board controller of the loader.

2. The work area personnel identification method of the unmanned loader according to claim 1, characterized by, The danger zone is the ground corridor that the loader will sweep during the warning time domain; A method for dynamically generating the danger zone in the ground coordinate system includes: The lateral boundary of the ground corridor is determined according to the hinge angle and the hinge steering kinematics. The warning depth of the ground corridor along the driving direction is determined based on the speed of the loader; The driving status parameters include vehicle speed and gear position. When the loader is in reverse gear, the ground corridor extends towards the rear of the loader.

3. The work area personnel identification method of claim 2, wherein The main body of the ground corridor is a ring-shaped sector with the turning center of the loader as the center; Let the distance from the hinge center of the loader to the front axle be... Distance from the hinge center to the rear axle The filtered hinge angle is Rear axle turning radius satisfy: ; The inner radius of the annular sector With outer radius satisfy: ,in, The width of the loader is [the width of the loader]. The rear overhang length is the distance from the center of the rear axle to the rear of the vehicle. For safety margin; When the angle is less than a preset threshold, the ground corridor degenerates into a rectangular corridor along the driving direction.

4. The method for personnel identification in the work area of ​​an unmanned loader according to claim 2, characterized in that, The round-trip delay is obtained from the timestamp loopback between the remote control terminal and the loader. Calculate the one-way delay of the remote control link. : ,in, This is the preset smoothing coefficient; Calculate response time : ,in, The preset operator reaction time; Calculate the warning depth : ,in, For the vehicle speed, This is the preset minimum static protection depth. This is the maximum braking deceleration of the loader.

5. The method for personnel identification in the work area of ​​an unmanned loader according to claim 2, characterized in that, The method for dynamically generating the danger zone in the ground coordinate system further includes: based on the hinge angular velocity. Predicting the duration of the prediction The hinge angle after ,in, The hinge angle after filtering; The danger zone is determined based on The generated ground corridor and based on The union of the generated ground corridors.

6. The method for personnel identification in the work area of ​​an unmanned loader according to claim 1, characterized in that, Methods for determining the risk level of the personnel targets include: Determine the signed distance from the personnel's ground location to the boundary of the danger zone. , The value is positive within the danger zone. continuous Frame satisfies When the target personnel are identified as having entered the danger zone; This is a preset entry threshold; continuous Frame satisfies When the time comes, it is determined that the personnel target has left the danger zone; exit threshold. ,in, The preset base hysteresis width, The preset expansion factor, This is the preset maximum width.

7. The method for personnel identification in the work area of ​​an unmanned loader according to claim 6, characterized in that, The risk levels include Level 1, Level 2, and Level 3; The personnel target is tracked to obtain its ground velocity. This ground velocity is then projected onto the boundary normal of the danger zone to obtain... rate of change over time ; when and At that time, the estimated intrusion time ,otherwise It is denoted as infinity; The personnel target is located within the surveillance area extended from the danger zone. and If the value exceeds the first time threshold, it is classified as Level 1. Less than or equal to the first time threshold, or the entry determination is valid and If the intrusion distance is less than the preset intrusion distance, it is judged as Level 2; Greater than or equal to the preset intrusion distance, or If the time threshold is less than or equal to the second time threshold, it is classified as the third level.

8. The method for personnel identification in the work area of ​​an unmanned loader according to claim 1, characterized in that, Personnel detection also outputs the visual confidence score of the personnel target, the method including: The radar targets detected by the millimeter-wave radar of the loader are acquired, and the radar targets are associated with the personnel targets according to their ground locations; When the association is successful, the visual confidence score and the radar confidence score of the radar target are weighted and fused to obtain the fused confidence score. Personnel targets whose fused confidence scores are greater than or equal to a preset confidence threshold are included in the determination of the risk level. When the personnel detection fails to detect any personnel and the radar target remains within the danger zone, a prompt is output according to the preset risk level. Wherein, the fusion confidence Calculate using the following formula: ,in, For visual confidence, For radar confidence level, As an indicator of image visibility, The preset equilibrium constant, This is the preset lower limit of the weight; This is the amplitude limiting function.

9. The method for personnel identification in the work area of ​​an unmanned loader according to claim 8, characterized in that, The method further includes: When the personnel target is successfully associated with the radar target and the deviation between the visual ranging and the radar radial distance exceeds a preset distance threshold, the radar radial distance is used to replace the distance component while retaining the visual azimuth angle to reconstruct the personnel's ground position. When the personnel target is not associated with a radar target, the safety margin of the danger zone is increased: ,in, Based on the basic safety margin, The distance to the target person. The preset maximum elevation difference of the working surface. The installation height of the rearview camera. This is the preset magnification factor.

10. A remote operation device for an unmanned loader, characterized in that, It includes the loader body, rearview camera, millimeter-wave radar, articulation angle sensor, vehicle controller, wireless communication module and remote operation terminal; The rear-view camera is mounted on the rear frame of the loader body and is used to collect images behind the loader body; The detection field of the millimeter-wave radar covers the monitoring area behind the loader body; The hinge angle sensor is used to detect the hinge angle of the loader body; The vehicle controller is connected to the rear-view camera, the millimeter-wave radar and the articulation angle sensor respectively, and executes the personnel identification method for the unmanned loader's work area as described in any one of claims 1 to 9; The remote operation terminal communicates with the vehicle controller via the wireless communication module to present personnel targets according to risk level.