Method, device, medium and program for determining unloading state of unmanned vehicle

Abstract

Embodiments of the present application disclose a kind of unmanned vehicle unloading state determination method, equipment, medium and program, comprising: obtaining the multi-source perception data of target unmanned vehicle and the reference geometric configuration parameter of target unmanned vehicle;Wherein, multi-source perception data includes acoustic feature data, oil gas spring pressure data, vehicle pose data and point cloud data;According to acoustic feature data, oil gas spring pressure data, vehicle pose data and reference geometric configuration parameter, first detection confidence is calculated;If first detection confidence is greater than or equal to first pre-set confidence threshold, according to first detection confidence, multi-source perception data and reference geometric configuration parameter, second detection confidence is calculated;According to first detection confidence and second detection confidence, actual unloading state is determined.The technical scheme of the embodiment of the present application can improve the accuracy of unloading station state recognition, and then guarantee the orderly and stable operation of unmanned unloading process.

Description

Technical Field

[0001] This invention relates to the field of vehicle transportation, and in particular to a method, device, medium, and program for determining the unloading status of an unmanned vehicle. Background Technology

[0002] Unloading operations by autonomous vehicles require advanced technologies for condition detection and status recognition. Current solutions primarily utilize LiDAR (Light Detection and Ranging) mounted on the rear of the autonomous vehicle for scanning and perception: first, the LiDAR scans to acquire the original reference information of the limiting reference components; during the vehicle's reversing maneuver, the limiting reference components are repeatedly scanned in real time; and a final scan is performed as the vehicle leaves the workstation after unloading to update the information on the limiting reference components and the unloading status. The availability of the unloading workstation is determined based on whether unloaded material remains within the range of the limiting reference components. However, dust pollution is significant in the unloading area, and the harsh dust environment causes a large amount of noise in the point cloud data collected by the LiDAR, resulting in a low effective point cloud ratio. This leads to insufficient reliability in detecting the limiting reference components and residual material, making it difficult to guarantee the accuracy of unloading workstation identification and condition judgment for autonomous vehicles. Summary of the Invention

[0003] This invention provides a method, device, medium, and program for determining the unloading status of an unmanned vehicle, which can improve the accuracy of unloading station status identification and thus ensure the orderly and stable operation of the unmanned unloading process.

[0004] According to one aspect of the present invention, a method for determining the unloading status of an autonomous vehicle is provided, applicable to an autonomous vehicle, comprising:

[0005] Acquire multi-source perception data of the target unmanned vehicle during the unloading process at the target unloading station and the reference geometric configuration parameters of the target unmanned vehicle; wherein, the multi-source perception data includes acoustic feature data, hydraulic spring pressure data, vehicle pose data and point cloud data;

[0006] The first detection confidence level of the target unmanned vehicle in the unloading state at the target unloading station is calculated based on the acoustic feature data, the oil spring pressure data, the vehicle pose data, and the reference geometric configuration parameters.

[0007] If the first detection confidence level is determined to be greater than or equal to the first preset confidence threshold, the second detection confidence level of the target unmanned vehicle in the unloading state at the target unloading station is calculated based on the first detection confidence level, the multi-source perception data and the reference geometric configuration parameters.

[0008] The actual unloading status of the target unmanned vehicle at the target unloading station is determined based on the first detection confidence level and the second detection confidence level.

[0009] According to another aspect of the present invention, an unloading status determination device for an unmanned vehicle is provided, configured in an unmanned vehicle, comprising:

[0010] The data acquisition module is used to acquire multi-source perception data of the target unmanned vehicle during the unloading process at the target unloading station and the reference geometric configuration parameters of the target unmanned vehicle; wherein, the multi-source perception data includes acoustic feature data, oil spring pressure data, vehicle pose data and point cloud data;

[0011] The first detection confidence calculation module is used to calculate the first detection confidence of the target unmanned vehicle in the unloading state of the target unloading station based on the acoustic feature data, the oil spring pressure data, the vehicle pose data and the reference geometric configuration parameters.

[0012] The second detection confidence calculation module is used to calculate the second detection confidence of the target unmanned vehicle in the unloading state of the target unloading station based on the first detection confidence, the multi-source perception data and the reference geometric configuration parameters, when the first detection confidence is greater than or equal to the first preset confidence threshold.

[0013] The actual unloading status determination module is used to determine the actual unloading status of the target unmanned vehicle at the target unloading station based on the first detection confidence level and the second detection confidence level.

[0014] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0015] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the unloading status determination method for an unmanned vehicle according to any embodiment of the present invention.

[0016] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the unloading state determination method for an unmanned vehicle according to any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer program product is also provided, comprising a computer program that, when executed by a processor, implements the unloading status determination method for an unmanned vehicle as described in any embodiment of the present invention.

[0018] This invention acquires multi-source perception data and baseline geometric parameters of a target unmanned vehicle during the unloading process at a target unloading station. The multi-source perception data includes acoustic feature data, hydraulic spring pressure data, vehicle pose data, and point cloud data. Further, a first detection confidence level of the target unmanned vehicle's unloading state at the target unloading station is calculated based on the acoustic feature data, hydraulic spring pressure data, vehicle pose data, and baseline geometric parameters. If the first detection confidence level is greater than or equal to a first preset confidence threshold, a second detection confidence level is calculated based on the first detection confidence level, the multi-source perception data, and the baseline geometric parameters. Based on this, the actual unloading state of the target unmanned vehicle at the target unloading station is determined according to the first and second detection confidence levels. This solution addresses the reliability limitations of using laser point cloud data for unloading state detection, improves the accuracy of unloading station state recognition, and thus ensures the orderly and stable operation of the unmanned unloading process.

[0019] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart of a method for determining the unloading status of an unmanned vehicle according to Embodiment 1 of the present invention;

[0022] Figure 2 This is a flowchart of a method for determining the unloading status of an unmanned vehicle according to Embodiment 2 of the present invention;

[0023] Figure 3 This is a schematic diagram illustrating the principle of determining the final material dropping position according to Embodiment 2 of the present invention;

[0024] Figure 4 This is a schematic diagram of a rectangular ROI (Region of Interest) for a material provided in Embodiment 2 of the present invention;

[0025] Figure 5 This is a schematic diagram of an unmanned vehicle unloading status determination device provided in Embodiment 3 of the present invention;

[0026] Figure 6 This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] Example 1

[0030] Figure 1 This is a flowchart of a method for determining the unloading status of an unmanned vehicle according to Embodiment 1 of the present invention. This embodiment is applicable to situations where the actual unloading status of the unloading station is determined comprehensively based on acoustic feature data, hydraulic spring pressure data, vehicle pose data, and point cloud data. This method can be executed by an unmanned vehicle unloading status determination device, which can be implemented in software and / or hardware and is generally integrated into an electronic device, which can be an unmanned vehicle. Correspondingly, as... Figure 1 As shown, the method includes the following operations:

[0031] S110. Acquire multi-source perception data of the target unmanned vehicle during the unloading process at the target unloading station and the reference geometric configuration parameters of the target unmanned vehicle; wherein, the multi-source perception data includes acoustic feature data, hydraulic spring pressure data, vehicle pose data and point cloud data.

[0032] The target unmanned vehicle can be an unmanned vehicle unloading transported materials at a target unloading station. For example, the target unmanned vehicle can be an unmanned mining truck used for transporting minerals in an open-pit mine scenario, or an unmanned dump truck used in a slag transportation scenario. This embodiment of the invention does not limit the usage scenario or type of the target unmanned vehicle. The target unloading station can be a work station for unloading transported materials by the unmanned vehicle. Multi-source perception data can be environmental and operational status data acquired by the target unmanned vehicle through onboard sensors, cameras, and other onboard perception devices during material unloading operations at the target unloading station. For example, multi-source perception data can include, but is not limited to, acoustic feature data, hydraulic spring pressure data, vehicle pose data, and point cloud data. This embodiment of the invention does not limit the specific content included in the multi-source perception data. Acoustic feature data can be sound data acquired by the target unmanned vehicle through an onboard microphone during material unloading operations at the target unloading station. Hydraulic spring pressure data can be data collected by an onboard pressure sensor during material unloading operations at the target unloading station, used to characterize the load borne by the hydraulic suspension. Vehicle pose data can be perceived data representing the spatial position and body posture of the target autonomous vehicle. Point cloud data can be point cloud data collected by the vehicle-mounted LiDAR during the process of the target autonomous vehicle leaving the target unloading station after completing the material unloading operation, representing the relative spatial information between the target autonomous vehicle and the limiting reference component. Reference geometric configuration parameters can be inherent parameters of the target autonomous vehicle's body. For example, reference geometric configuration parameters may include, but are not limited to, the cargo box length, hinge point position, wheelbase, and microphone installation position of the target autonomous vehicle. This embodiment of the invention does not limit the specific content included in the reference geometric configuration parameters.

[0033] In this embodiment of the invention, an unloading station typically supports unloading materials from multiple vehicles. When the nth vehicle cannot completely unload at the limit reference point, some residual material will be unloaded to the inside of the limit reference point. At this time, no more vehicles can come to the unloading station for material unloading. Therefore, it is necessary to determine the actual unloading status of the target unmanned vehicle at the target unloading station. Specifically, firstly, multi-source perception data and the reference geometric configuration parameters of the target unmanned vehicle during the unloading process at the target unloading station can be obtained. Specifically, an industrial-grade microphone array can be deployed at the rear of the target unmanned vehicle frame. This microphone array is oriented towards the rear of the cargo box and the limit reference point area of ​​the target unmanned vehicle. Beamforming technology is used to suppress environmental background noise and extract acoustic feature data. At the same time, the pressure sensors of the target unmanned vehicle's hydropneumatic suspension system can be used to obtain the hydropneumatic spring pressure data of the target unmanned vehicle. This acquisition method is not affected by dust or lighting conditions and can infer the cargo box load change and road impact conditions based on the suspension pressure change. Furthermore, the lifting angles of the target unmanned vehicle's cargo box at different operational stages can be collected using the cargo box angle sensor. For example, the lifting angles at four key operational nodes—"first lifting to position," "second lifting to position," "dropping to start," and "dropping to position"—can be collected. Additionally, the target unmanned vehicle's autonomous driving system can be used to obtain real-time centimeter-level position and heading information. This allows acoustic feature data and hydraulic spring pressure data to be mapped to a spatial coordinate system and compared with the position of the limiting reference component to determine whether the material drop point is inside or outside the limiting reference component. It is understood that the vehicle pose data includes the lifting angles and vehicle positioning information corresponding to different operational stages of the target unmanned vehicle's cargo box. Finally, during the process of the target unmanned vehicle leaving the target unloading station after completing the material unloading operation, the onboard LiDAR can collect point cloud data between the target unmanned vehicle and the limiting reference component. Understandably, the limit reference component is the boundary structure that defines the working range of the target unloading station, and is the core reference for determining whether the vehicle unloading operation is compliant and whether there are residual materials at the station.

[0034] S120. Calculate the first detection confidence level of the target unmanned vehicle in the unloading state at the target unloading station based on the acoustic feature data, the oil spring pressure data, the vehicle pose data, and the reference geometric configuration parameters.

[0035] The first detection confidence level can be the confidence level of the unloading state of the target unmanned vehicle at the target unloading station, determined based on the acoustic characteristic data, oil spring pressure data, vehicle pose data, and reference geometric configuration parameters of the target unmanned vehicle during the material unloading operation at the target unloading station.

[0036] Accordingly, after acquiring the multi-source perception data and the baseline geometric configuration parameters of the target unmanned vehicle during the unloading process at the target unloading station, since the multi-source perception data can reflect the execution status of the target unmanned vehicle when performing material unloading operations at the target unloading station, the confidence level of the unloading status of the target unmanned vehicle during the material unloading task at the target unloading station can be determined based on the acoustic feature data, oil spring pressure data, vehicle pose data, and baseline geometric configuration parameters in the multi-source perception data.

[0037] S130. If it is determined that the first detection confidence is greater than or equal to the first preset confidence threshold, calculate the second detection confidence of the target unmanned vehicle in the unloading state of the target unloading station based on the first detection confidence, the multi-source perception data and the reference geometric configuration parameters.

[0038] The first preset confidence threshold can be a pre-set confidence threshold representing whether the inner side of the limiting reference component includes unloading material. For example, the first preset confidence threshold can be 0.2. The second detection confidence can be the confidence of the unloading status of the target unmanned vehicle after completing the unloading operation at the target unloading station, determined based on the first detection confidence, multi-source sensing data, and reference geometric configuration parameters.

[0039] Specifically, a first preset confidence threshold can be set in advance. If the first detection confidence is greater than or equal to the first preset confidence threshold, it can be determined that there may be unloading material between the target autonomous vehicle and the limiting reference component, i.e., inside the limiting reference component. In the case that there may be unloading material inside the limiting reference component, a second detection confidence can be calculated based on the calculated first detection confidence, multi-source sensing data, and reference geometric configuration parameters to determine the unloading state of the target autonomous vehicle at the target unloading station.

[0040] S140. Determine the actual unloading status of the target unmanned vehicle at the target unloading station based on the first detection confidence level and the second detection confidence level.

[0041] The actual unloading status can be used to characterize whether there is residual unloading material inside the limit reference component after the target unmanned vehicle completes the material unloading operation at the target unloading station.

[0042] Accordingly, after calculating the first and second detection confidence levels, the actual unloading status of the target unmanned vehicle at the target unloading station can be determined comprehensively based on the first and second detection confidence levels. In a specific example, the actual unloading status may include, but is not limited to, abnormal material unloading status, pending status, and normal material unloading status. An abnormal material unloading status indicates that there is residual unloading material inside the limiting reference component, the target unloading station is full, and the target unloading station should be disabled; a normal material unloading status indicates that there is no unloading material inside the limiting reference component, the target unloading station is not full, and the target unloading station can continue to be used; a pending status indicates that it is uncertain whether there is unloading material inside the limiting reference component, and a second judgment is required.

[0043] Therefore, the embodiments of the present invention comprehensively judge the actual unloading status of the target unmanned vehicle at the target unloading station by using acoustic feature data, oil spring pressure data, vehicle pose data and point cloud data. This can improve the problems of excessive noise and low effective point cloud ratio that exist when relying solely on point cloud data detection, and avoid the drawback of insufficient reliability in detecting residual material inside the limit reference component. It effectively improves the accuracy and robustness of the unloading status recognition results, and realizes reliable judgment on whether unloading material remains inside the limit reference component. This meets the all-weather, high-reliability monitoring requirements of unmanned vehicle unloading operation scenarios.

[0044] This invention acquires multi-source perception data and baseline geometric parameters of a target unmanned vehicle during the unloading process at a target unloading station. The multi-source perception data includes acoustic feature data, hydraulic spring pressure data, vehicle pose data, and point cloud data. Further, a first detection confidence level of the target unmanned vehicle's unloading state at the target unloading station is calculated based on the acoustic feature data, hydraulic spring pressure data, vehicle pose data, and baseline geometric parameters. If the first detection confidence level is greater than or equal to a first preset confidence threshold, a second detection confidence level is calculated based on the first detection confidence level, the multi-source perception data, and the baseline geometric parameters. Based on this, the actual unloading state of the target unmanned vehicle at the target unloading station is determined according to the first and second detection confidence levels. This solution addresses the reliability limitations of using laser point cloud data for unloading state detection, improves the accuracy of unloading station state recognition, and thus ensures the orderly and stable operation of the unmanned unloading process.

[0045] Example 2

[0046] Figure 2This is a flowchart of a method for determining the unloading state of an unmanned vehicle according to Embodiment 2 of the present invention. This embodiment is a specific embodiment based on the above embodiment. In this embodiment, specific optional implementation methods are given for calculating the first detection confidence level of the target unmanned vehicle at the target unloading station based on acoustic feature data, oil spring pressure data, vehicle pose data and reference geometric configuration parameters, and calculating the second detection confidence level of the target unmanned vehicle at the target unloading station based on the first detection confidence level, multi-source perception data and reference geometric configuration parameters. Correspondingly, as Figure 2 As shown, the method in this embodiment may include:

[0047] S210. Acquire multi-source perception data of the target unmanned vehicle during the unloading process at the target unloading station and the reference geometric configuration parameters of the target unmanned vehicle; wherein, the multi-source perception data includes acoustic feature data, hydraulic spring pressure data, vehicle pose data and point cloud data.

[0048] In an optional embodiment of the present invention, after acquiring the multi-source perception data of the target unmanned vehicle during the unloading process at the target unloading station, the method may further include: performing timing alignment processing on the multi-source perception data based on the unified clock of the vehicle domain controller of the target unmanned vehicle; performing timing interval division processing on the multi-source perception data; wherein the timing interval includes a parking and positioning stage, a staged lifting and unloading stage, a lifting steady-state holding stage, and a low-speed forward re-unloading stage; establishing a reference coordinate system with the rear axle center of the target unmanned vehicle as the origin, and using the position of the limiting reference component when the rear wheel of the target unmanned vehicle abuts against the limiting reference component of the target unloading station as a spatial reference line, and performing spatial alignment processing on the multi-source perception data.

[0049] The reference coordinate system can be a Cartesian coordinate system fixed on the target autonomous vehicle, with the origin at the center of the rear axle, the X-axis pointing longitudinally towards the front of the vehicle, the Y-axis pointing laterally towards the left side of the vehicle, and the Z-axis pointing vertically upwards. The spatial reference line is used to mark the extreme stopping positions of the wheels when the target autonomous vehicle is parked, and serves as the spatial positioning reference for the unloading operation of the target autonomous vehicle.

[0050] In this embodiment of the invention, after acquiring multi-source perception data of the target unmanned vehicle during the unloading process at the target unloading station, the timestamp difference of all multi-source perception data can be resampled using the unified clock of the vehicle's onboard domain controller as a reference. For example, the sampling frequency can be 50Hz. Simultaneously, the unloading process of the target unmanned vehicle at the target unloading station can be divided into a parking and positioning stage, a graded lifting and unloading stage, a lifting steady-state holding stage, and a low-speed forward re-unloading stage, using four key nodes corresponding to the lifting angle of the target unmanned vehicle's cargo box as time interval segmentation anchor points, thereby achieving temporal alignment of features. The parking and positioning phase can be the phase where the target unmanned vehicle comes into contact with the limit reference component and the lifting angle is 0; the staged lifting and unloading phase can be the phase from the start of lifting to the second lifting position, where the material is concentrated and poured out; the lifting steady-state maintenance phase can be the phase where the vehicle stops after the lifting position is reached, the material slippage weakens, and the vehicle waits for the forward command; the low-speed forward re-unloading phase can be the phase where the vehicle is kept at the highest lifting position, moves forward at low speed, and clears the remaining material.

[0051] Furthermore, multi-source sensing data can be mapped to a reference coordinate system established with the rear axle center of the target autonomous vehicle as the origin. Further, the position of the limiting reference component when the rear wheels of the target autonomous vehicle abut against the limiting reference component of the target unloading station can be used as a spatial reference line. Then, based on this spatial reference line, inner and outer attribute labels can be applied to the multi-source sensing data to eliminate data interference caused by vehicle parking position deviations. It can be understood that the area between the target autonomous vehicle and the limiting reference component can be defined as the inner side, and the area away from the target autonomous vehicle and located on the other side of the spatial reference line can be defined as the outer side.

[0052] S220. Calculate the load residual rate of the target unmanned vehicle during the lifting steady-state holding phase based on the oil spring pressure data.

[0053] Among them, the load residual rate can be the ratio of the load corresponding to the remaining material inside the cargo box of the target unmanned vehicle to the initial total load of the unloading operation during the lifting steady-state holding phase, which is used to quantify the degree of material residue inside the cargo box.

[0054] Specifically, the pressure data from the hydraulic springs can be used to quantitatively determine whether there is residual material in the cargo box and when the material is unloaded. Furthermore, under low-speed forward movement conditions, road impact interference is minimal, resulting in higher accuracy in identifying load changes. Under normal material unloading conditions, the pressure of the target autonomous vehicle decreases rapidly and monotonously during the two lifting phases, reaching approximately the empty load baseline after the second lifting. The descent inflection point is highly synchronized with the lifting angle. Under abnormal material unloading conditions, the pressure decrease rate of the target autonomous vehicle is slow, and the inflection point is delayed. In some stuck conditions, there is no significant pressure decrease during the first lifting phase, with a sharp pressure drop only occurring during the forward movement phase.

[0055] Based on the above principles, after obtaining the hydraulic spring pressure data of the target unmanned vehicle during the unloading process at the target unloading station, the load residual rate of the target unmanned vehicle during the lifting steady-state holding phase can be calculated based on the hydraulic spring pressure data. In a specific example, the load residual rate of the target unmanned vehicle during the lifting steady-state holding phase can be calculated based on the following formula:

[0056] ;

[0057] in, The load residual rate of the target autonomous vehicle during the lifting steady-state holding phase. This data represents the pressure of the air spring during the steady-state holding phase of the lift. The data on the pressure of the pneumatic springs when the cargo box of the target unmanned vehicle is completely empty of materials. This data represents the hydraulic spring pressure when the target unmanned vehicle is fully loaded with materials. It is understandable that... The larger the value, the greater the probability of residual material within the limit reference component.

[0058] Optionally, the rate of decrease and cumulative decrease of the hydraulic spring pressure during the staged lifting and unloading phase can be calculated based on the following formula to help determine the actual unloading status of the target unmanned vehicle at the target unloading station:

[0059] ;

[0060] ;

[0061] in, The rate of decrease in pressure of the gas spring during the staged lifting and unloading phase. The data pertains to the hydraulic spring pressure after the target autonomous vehicle has been lifted into position for the second time. To increase the total duration, This represents the cumulative decrease in the pressure of the gas spring during the staged lifting and unloading phase. When When the value is greater than or equal to 0.9, it indicates that the material is basically unloaded during the lifting process of the target unmanned vehicle; when When the value is less than 0.8, it indicates that there is still obvious material residue in the cargo box after the target unmanned vehicle has been lifted into position.

[0062] Optionally, the pressure drop can be calculated based on the following formula after the vehicle moves forward at low speed, to help determine the actual unloading status of the target unmanned vehicle at the target unloading station:

[0063] ;

[0064] in, This refers to the second drop in pressure of the air spring after the vehicle moves forward at low speed. The pressure of the air spring is adjusted to prepare for low-speed forward movement after the steady-state lifting phase ends. This represents the pressure of the pneumatic spring after the low-speed forward unloading action is completed. When the material unloading is normal, the pressure of the pneumatic spring during the low-speed forward movement of the target unmanned vehicle is approximately 0, with no significant secondary drop. When the material unloading is abnormal, i.e., there is residual material inside the limit reference component, the forward movement causes the residual material to become unstable and be unloaded, resulting in a significant secondary drop in pressure.

[0065] S230. Determine the acoustic timing characteristic indicators of the target unmanned vehicle during the unloading process at the target unloading station based on the acoustic characteristic data.

[0066] Among them, the acoustic temporal characteristic index can be the acoustic index corresponding to different stages of the target unmanned vehicle performing the unloading operation.

[0067] In this embodiment of the invention, when material unloading is normal, the acoustic characteristic data of the target unmanned vehicle shows a "single-peak gradual descent" trend; when material unloading is abnormal, i.e., there is residual material inside the limiting reference component, the acoustic characteristic data of the target unmanned vehicle shows a "double-peak structure of first decreasing and then increasing". Therefore, the actual unloading state of the target unmanned vehicle at the target unloading station can be determined based on the acoustic characteristic data of the target unmanned vehicle. Specifically, after obtaining the acoustic characteristic data of the target unmanned vehicle during the unloading process at the target unloading station, the envelope of the acoustic characteristic data can be extracted using Hilbert transform to generate the amplitude time-series curve corresponding to the acoustic characteristic data. Furthermore, the acoustic time-series characteristic indicators of the target unmanned vehicle during the unloading process at the target unloading station can be extracted based on the amplitude time-series curve corresponding to the acoustic characteristic data. In a specific example, acoustic timing characteristic indicators may include, but are not limited to, the amplitude main peak (i.e., the first cascading peak of the amplitude timing curve) during the staged lifting and unloading phase, the amplitude trough during the steady-state holding phase of the lifting phase, the amplitude secondary peak (i.e., the second cascading peak) during the low-speed forward unloading phase, the starting moment when the amplitude begins to rise from the trough during the low-speed forward unloading phase, the moment when the vehicle starts moving forward during the low-speed forward unloading phase, and the moment when unloading is completed during the low-speed forward unloading phase.

[0068] S240. Calculate the amplitude recovery rate and timing matching degree of the target unmanned vehicle during the unloading process at the target unloading station based on the acoustic timing characteristic index.

[0069] Accordingly, after obtaining the acoustic timing characteristic indicators of the target unmanned vehicle during the unloading process at the target unloading station, the amplitude recovery rate and timing matching degree of the target unmanned vehicle during the unloading process at the target unloading station can be calculated based on the acoustic timing characteristic indicators:

[0070] ;

[0071] ;

[0072] in, The amplitude recovery rate, This is the secondary peak of the amplitude during the low-speed forward unloading phase. To boost the amplitude trough during the steady-state maintenance phase, This represents the main peak of amplitude during the staged lifting and unloading phase. For time series matching degree, This marks the starting point at which the amplitude of the low-speed forward unloading phase begins to recover from its trough. This refers to the moment when the vehicle starts moving forward during the low-speed unloading phase. This serves as the forward observation window. It should be noted that the greater the amplitude recovery rate, the higher the probability of abnormal material unloading; the higher the timing matching degree, the higher the credibility that the recovery is triggered by vehicle movement.

[0073] S250. Calculate the sound source energy ratio of the target area based on the acoustic feature data, the vehicle pose data, and the reference geometric configuration parameters.

[0074] The target area sound source energy ratio can be the ratio of the total sound source signal energy in the target area to the total acoustic energy of the entire field, after dividing the collected acoustic feature data into the target area and the remaining background area.

[0075] Correspondingly, the azimuth and distance of the sound source can be solved by beamforming the onboard microphone array of the target autonomous vehicle. Then, combined with the reference geometric configuration parameters of the target autonomous vehicle and the real-time pose data of the vehicle, the proportion of sound source energy inside the limiting reference component can be calculated, and this proportion can be used as the sound source energy proportion of the target area to verify the spatial position of the material landing point.

[0076] S260. The load residual rate, amplitude recovery rate, temporal matching degree and sound source energy ratio of the target unmanned vehicle during the lifting steady-state holding phase are weighted and summed to obtain the first detection confidence level of the target unmanned vehicle in the unloading state of the target unloading station.

[0077] Accordingly, after calculating the above-mentioned feature indicators, the first detection confidence of the target unmanned vehicle in the unloading state at the target unloading station can be calculated based on the following formula:

[0078] ;

[0079] in, The first detection confidence level, The load residual rate of the target autonomous vehicle during the lifting steady-state holding phase. for The corresponding weights The amplitude recovery rate, for The corresponding weights For time series matching degree, for The weight, The percentage of sound source energy in the target area. for The corresponding weights.

[0080] S270. Determine whether the first detection confidence level is greater than or equal to the first preset confidence threshold. If yes, execute S280; otherwise, end the judgment process.

[0081] In this embodiment of the invention, if the first detection confidence level is less than the first preset confidence threshold, it can be directly determined that there is no unloading material residue between the target unmanned vehicle and the limiting reference component, and the target unloading station can continue to be used; if the first detection confidence level is greater than or equal to the first preset confidence threshold, it can be determined that there may be unloading material between the target unmanned vehicle and the limiting reference component, and it is necessary to further determine the actual unloading status of the target unmanned vehicle at the target unloading station.

[0082] S280. Determine the unloading end time of the target unmanned vehicle within the limiting reference component based on the acoustic feature data and the oil spring pressure data.

[0083] Correspondingly, if it is determined that there may be unloading material between the target unmanned vehicle and the limiting reference component, the unloading end time of the target unmanned vehicle within the limiting reference component can be identified based on acoustic feature data and oil spring pressure data.

[0084] S290. Determine the material circumscribed rectangle ROI based on the vehicle pose data at the end of unloading within the limiting reference member and the reference geometric configuration parameters.

[0085] Among them, the ROI (Region of Interest) of the material can be the smallest axis-aligned rectangle that can enclose the area of ​​residual unloading material inside the limiting reference component.

[0086] Accordingly, after determining the unloading end time of the target unmanned vehicle within the limit reference component, the final material dropping position can be determined based on the vehicle pose data and reference geometric configuration parameters at the unloading end time. Figure 3 This is a schematic diagram illustrating the principle of determining the final material dropping position according to Embodiment 2 of the present invention. Specifically, as shown... Figure 3As shown, high-precision positioning information of the target unmanned vehicle, with its rear axle center as the positioning reference point, can be obtained based on the moment of unloading completion, such as the cargo box lifting angle. Vertical reference height of the positioning reference point Horizontal distance from the positioning reference point to the hinge axis of the carriage The horizontal projection distance from the hinge shaft of the carriage to the point where the material falls. Wait, then calculate the final material drop position.

[0087] Figure 4 This is a schematic diagram of a material circumscribed rectangle (ROI) provided in Embodiment 2 of the present invention. After calculating the final material drop position, coordinate transformation can be performed based on the positional relationship between the lower rear edge of the cargo box and the positioning reference point to solve for the spatial position corresponding to the lower rear edge of the cargo box. Simultaneously, the same coordinate transformation method can be used to vertically project the lower rear edge of the cargo box onto the ground as the material pile boundary point. Based on this, by combining all boundary points, a fit can be obtained as shown below. Figure 4 The diagram shows the bounding rectangle (ROI) of the material in the geodetic coordinate system. For example, the bounding rectangle (ROI) of the material can be represented as... ,in , , as well as These are the boundary points of the defined projection area.

[0088] S2100: Determine the target point cloud data based on the first detection confidence level, the material circumscribed rectangle ROI, and the point cloud data.

[0089] The target point cloud data can be point cloud data used to calculate the second detection confidence level.

[0090] Accordingly, after obtaining the material's bounding rectangle ROI, it can be adjusted based on the first detection confidence level, and the point cloud data within the adjusted material's bounding rectangle ROI can be used as the target point cloud data.

[0091] In an optional embodiment of the present invention, determining the target point cloud data based on the first detection confidence level, the material circumscribed rectangle (ROI), and the point cloud data may include: when it is determined that the first detection confidence level is greater than or equal to the first preset confidence threshold and when it is determined that the first detection confidence level is less than or equal to the second preset confidence threshold, extending the material circumscribed rectangle (ROI) outward by a first preset distance to obtain the target material circumscribed rectangle (ROI); when it is determined that the first detection confidence level is greater than the second preset confidence threshold, extending the material circumscribed rectangle (ROI) outward by a second preset distance to obtain the target material circumscribed rectangle (ROI); and using the point cloud data within the target material circumscribed rectangle (ROI) as the target point cloud data.

[0092] The first preset distance and the second preset distance can be pre-set distance values. For example, the first preset distance can be 1m and the second preset distance can be 0.5m. This embodiment of the invention does not limit the specific values ​​of the first and second preset distances. The ROI (Region of Interest) of the target material can be obtained by adjusting the ROI based on the first detection confidence level. The second preset confidence threshold can be a pre-set confidence threshold. For example, the second preset confidence threshold can be 0.8. This embodiment of the invention does not limit the specific value of the second preset confidence threshold.

[0093] In this embodiment of the invention, when determining the target point cloud data based on the first detection confidence level, the circumscribed rectangle (ROI) of the material, and the point cloud data, the first detection confidence level can be used as the basis for judgment. If the first detection confidence level is greater than or equal to a first preset confidence threshold and less than or equal to a second preset confidence threshold, indicating that there is likely residual unloaded material inside the limiting reference component, the circumscribed rectangle (ROI) of the material can be extended outward by a first preset distance to serve as the target circumscribed rectangle (ROI) of the material, thus covering the uncertain range. If the first detection confidence level is greater than a second preset confidence threshold, indicating that there is a high probability of residual unloaded material inside the limiting reference component, the circumscribed rectangle (ROI) of the material can be extended outward by a second preset distance to serve as the target circumscribed rectangle (ROI) of the material, thereby reserving a margin while ensuring the accuracy of identification. After determining the target circumscribed rectangle (ROI), the point cloud data within the target circumscribed rectangle (ROI) of the material can be used as the target point cloud data.

[0094] S2110. Determine the second detection confidence level of the target unmanned vehicle in the unloading state of the target unloading station based on the target point cloud data.

[0095] Accordingly, after determining the target point cloud data, the unloading status of the target unmanned vehicle at the target unloading station can be identified based on the point cloud data to obtain the corresponding second detection confidence level.

[0096] In an optional embodiment of the present invention, determining the second detection confidence level of the target unmanned vehicle in the unloading state of the target unloading station based on the target point cloud data may include: determining the maximum height of the point cloud, the projected area of ​​the point cloud, and the effective proportion of the point cloud based on the target point cloud data; and calculating the second detection confidence level of the target unmanned vehicle in the unloading state of the target unloading station based on the maximum height of the point cloud, the projected area of ​​the point cloud, and the effective proportion of the point cloud.

[0097] The maximum height of the point cloud can be the maximum value of the height dimension coordinate among all 3D point coordinates within the target point cloud data. The point cloud projection area can be the area of ​​the 2D coverage region formed by all projected points after orthogonally projecting the effective 3D points in the target point cloud onto the target plane along a specified projection direction. The effective point cloud percentage can be the ratio of the number of effective point clouds within the bounded rectangle ROI space of the target material to the total number of point clouds within that area.

[0098] In this embodiment of the invention, when determining the second detection confidence level of the unloading state of the target unmanned vehicle at the target unloading station based on the target point cloud data, the ground plane can first be fitted within the ROI bounded by the target material using RANSAC (RANdomSample Consensus) to extract the maximum height, projected area, and effective percentage of the point cloud data. Further, the second detection confidence level of the target unmanned vehicle at the target unloading station can be calculated based on the following formula using the maximum height, projected area, and effective percentage of the point cloud data:

[0099] ;

[0100] in, The second detection confidence level, The maximum height of the point cloud. For example, a height-based threshold is used to determine the height. It can be 2m, but this embodiment of the invention is not applicable. The specific values ​​to be taken are limited. The weight corresponding to the maximum height of the point cloud. The point cloud projection area, This is the area determination value. The weights corresponding to the projected area of ​​the point cloud. For the effective proportion of point cloud, The weights corresponding to the effective percentage of point clouds.

[0101] Optionally, before determining the second detection confidence level of the unloading status of the target unmanned vehicle at the target unloading station based on the target point cloud data, outliers caused by dust can be removed by statistical filtering, and high-altitude floating artifacts exceeding 2.5m above the ground can be filtered out using height constraints. The target point cloud data retained after multi-layer condition filtering can effectively filter out noise points while completely preserving the original geometry of the material pile.

[0102] S2120. Determine the actual unloading status of the target unmanned vehicle at the target unloading station based on the first detection confidence level and the second detection confidence level.

[0103] In an optional embodiment of the present invention, determining the actual unloading status of the target unmanned vehicle at the target unloading station based on the first detection confidence level and the second detection confidence level may include: determining a dust assessment coefficient based on the effective proportion of the point cloud; determining the weight of each detection confidence level based on the dust assessment coefficient; performing a weighted summation of the first detection confidence level and the second detection confidence level based on the weight of each detection confidence level to obtain a comprehensive detection confidence level; and determining the actual unloading status of the target unmanned vehicle at the target unloading station based on the comprehensive detection confidence level.

[0104] Among them, the dust assessment coefficient can be used as an indicator to quantify the proportion of false point clouds caused by suspended dust within the circumscribed rectangle ROI of the target material and the severity of dust interference.

[0105] In this embodiment of the invention, when determining the actual unloading status of the target unmanned vehicle at the target unloading station based on the first detection confidence level and the second detection confidence level, the dust assessment coefficient can first be determined based on the effective proportion of the point cloud mentioned above. It should be noted that the dust assessment coefficient ranges from 0 to 1; the larger the dust assessment coefficient, the heavier the dust. Furthermore, the weight of each detection confidence level can be determined based on the dust assessment coefficient using the following formula:

[0106] ;

[0107] ;

[0108] in, The weight corresponding to the first detection confidence level. This is the dust assessment coefficient. The weights corresponding to the second detection confidence level are defined as follows: Understandably, in a clear, lightly dusty environment, the second detection confidence level has a higher weight, ensuring geometric measurement accuracy; in a heavily dusty environment, the first detection confidence level has a higher weight, relying on acoustic and pressure characteristics unaffected by the environment to ensure recognition reliability. The above scheme dynamically adjusts the weights of each detection confidence level based on the quality of the LiDAR point cloud, effectively adapting to complex and harsh working conditions such as dust and rain / fog, significantly improving the robustness of material unloading status recognition.

[0109] After determining the weights of each detection confidence level, the overall detection confidence level can be calculated based on the following formula:

[0110] ;

[0111] in, To comprehensively test the confidence level.

[0112] Based on this, the actual unloading status of the target unmanned vehicle at the target unloading station can be determined by the comprehensive detection confidence level.

[0113] In an optional embodiment of the present invention, determining the actual unloading status of the target unmanned vehicle at the target unloading station based on the comprehensive detection confidence level may include: if the comprehensive detection confidence level is greater than or equal to a third preset confidence threshold, determining the actual unloading status of the target unmanned vehicle at the target unloading station as an abnormal material unloading status, and outputting material space geometric parameters; if the comprehensive detection confidence level is less than the third preset confidence threshold and greater than a fourth preset confidence threshold, determining the actual unloading status of the target unmanned vehicle at the target unloading station as a pending status; re-acquiring target point cloud data and updating the comprehensive detection confidence level based on the target point cloud data; determining the actual unloading status of the target unmanned vehicle at the target unloading station based on the updated comprehensive detection confidence level; and if the comprehensive detection confidence level is less than or equal to the fourth preset confidence threshold, determining the actual unloading status of the target unmanned vehicle at the target unloading station as a normal material unloading status.

[0114] The third and fourth preset confidence thresholds are pre-set confidence thresholds. For example, the third preset confidence threshold can be 0.7, and the fourth preset confidence threshold can be 0.3. This embodiment of the invention does not limit the specific values ​​of the third and fourth preset confidence thresholds. The material space geometric parameters can be a set of parameters such as the location, height, and area of ​​the material within the bounding rectangle (ROI) of the target material.

[0115] In this embodiment of the invention, when determining the actual unloading status of the target unmanned vehicle at the target unloading station based on the comprehensive detection confidence level, if the comprehensive detection confidence level is greater than or equal to the third preset confidence threshold, it can be determined that the actual unloading status of the target unmanned vehicle at the target unloading station is an abnormal material unloading status, there is unloading material residue inside the limit reference component, the target unloading station is full, the material space geometric parameters corresponding to the material residue are output, and the scheduling center can be notified to perform a disable operation on the target unloading station.

[0116] If the overall detection confidence level is less than the third preset confidence threshold but greater than the fourth preset confidence threshold, then the actual unloading status of the target autonomous vehicle at the target unloading station can be determined as pending. Furthermore, the target point cloud data can be reacquired, and the overall detection confidence level can be updated based on the updated overall detection confidence level. This updated overall detection confidence level can then be used to re-identify the actual unloading status of the target autonomous vehicle at the target unloading station.

[0117] If the overall detection confidence level is less than or equal to the fourth preset confidence threshold, it can be determined that the actual unloading status of the target unmanned vehicle at the target unloading station is a normal material unloading status, and there is no unloaded material residue inside the limit reference component. The target unloading station can continue to be used. Therefore, the dispatch center can be notified to release the target unloading station for other vehicles to carry out material unloading operations.

[0118] This invention acquires multi-source perception data and baseline geometric parameters of the target unmanned vehicle during the unloading process at the target unloading station. The multi-source perception data includes acoustic feature data, hydraulic spring pressure data, vehicle pose data, and point cloud data. Further, the load residual rate of the target unmanned vehicle during the lifting and steady-state holding phase is calculated based on the hydraulic spring pressure data. Simultaneously, acoustic temporal characteristic indicators of the target unmanned vehicle during the unloading process at the target unloading station are determined based on the acoustic feature data. Further, the amplitude recovery rate and temporal matching degree of the target unmanned vehicle during the unloading process at the target unloading station are calculated based on the acoustic temporal characteristic indicators. Simultaneously, the proportion of sound source energy in the target area is calculated based on the acoustic feature data, vehicle pose data, and baseline geometric parameters. Based on this, a weighted sum is performed on the load residual rate, amplitude recovery rate, temporal matching degree, and proportion of sound source energy in the target area during the lifting and steady-state holding phase of the target unmanned vehicle to obtain the first detection confidence level of the target unmanned vehicle in the unloading state at the target unloading station. After obtaining the first detection confidence level of the target unmanned vehicle's unloading status at the target unloading station, it is determined whether the first detection confidence level is greater than or equal to a first preset confidence threshold. If not, the judgment process ends; if so, the unloading end time of the target unmanned vehicle within the limiting reference component is determined based on acoustic feature data and hydraulic spring pressure data. Then, the circumscribed rectangle (ROI) of the material is determined based on the vehicle pose data and reference geometric configuration parameters at the unloading end time within the limiting reference component. Further, target point cloud data is determined based on the first detection confidence level, the circumscribed rectangle (ROI) of the material, and point cloud data. On this basis, a second detection confidence level of the target unmanned vehicle's unloading status at the target unloading station is determined based on the target point cloud data. Then, the actual unloading status of the target unmanned vehicle at the target unloading station is determined based on the first and second detection confidence levels. The above scheme solves the problem of insufficient reliability in using laser point cloud data for unloading status detection, improves the accuracy of unloading station status recognition, and thus ensures the orderly and stable operation of the unmanned unloading process.

[0119] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information in this technical solution comply with relevant laws and regulations and do not violate public order and good morals.

[0120] It should be noted that all information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) involved in this disclosure are information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data comply with the relevant laws, regulations and standards of the relevant regions.

[0121] It should be noted that any arrangement or combination of the technical features in the above embodiments also falls within the protection scope of this invention.

[0122] Example 3

[0123] Figure 5 This is a schematic diagram of an unmanned vehicle unloading status determination device provided in Embodiment 3 of the present invention, as shown below. Figure 5 As shown, the device is configured in an autonomous vehicle and includes: a data acquisition module 310, a first detection confidence calculation module 320, a second detection confidence calculation module 330, and an actual unloading status determination module 340, wherein:

[0124] The data acquisition module 310 is used to acquire multi-source perception data of the target unmanned vehicle during the unloading process at the target unloading station and the reference geometric configuration parameters of the target unmanned vehicle; wherein, the multi-source perception data includes acoustic feature data, oil spring pressure data, vehicle pose data and point cloud data.

[0125] The first detection confidence calculation module 320 is used to calculate the first detection confidence of the target unmanned vehicle in the unloading state of the target unloading station based on the acoustic feature data, the oil spring pressure data, the vehicle pose data and the reference geometric configuration parameters.

[0126] The second detection confidence calculation module 330 is used to calculate the second detection confidence of the target unmanned vehicle in the unloading state of the target unloading station based on the first detection confidence, the multi-source perception data and the reference geometric configuration parameters, when the first detection confidence is determined to be greater than or equal to the first preset confidence threshold.

[0127] The actual unloading status determination module 340 is used to determine the actual unloading status of the target unmanned vehicle at the target unloading station based on the first detection confidence level and the second detection confidence level.

[0128] This invention acquires multi-source perception data and baseline geometric parameters of a target unmanned vehicle during the unloading process at a target unloading station. The multi-source perception data includes acoustic feature data, hydraulic spring pressure data, vehicle pose data, and point cloud data. Further, a first detection confidence level of the target unmanned vehicle's unloading state at the target unloading station is calculated based on the acoustic feature data, hydraulic spring pressure data, vehicle pose data, and baseline geometric parameters. If the first detection confidence level is greater than or equal to a first preset confidence threshold, a second detection confidence level is calculated based on the first detection confidence level, the multi-source perception data, and the baseline geometric parameters. Based on this, the actual unloading state of the target unmanned vehicle at the target unloading station is determined according to the first and second detection confidence levels. This solution addresses the reliability limitations of using laser point cloud data for unloading state detection, improves the accuracy of unloading station state recognition, and thus ensures the orderly and stable operation of the unmanned unloading process.

[0129] Optionally, the above-mentioned device may further include a data preprocessing module, used to perform time-series alignment processing on the multi-source sensing data based on the unified clock of the on-board domain controller of the target autonomous vehicle; to perform time-series interval division processing on the multi-source sensing data; wherein the time-series interval includes a parking and positioning stage, a staged lifting and unloading stage, a lifting steady-state holding stage, and a low-speed forward re-unloading stage; to establish a reference coordinate system with the rear axle center of the target autonomous vehicle as the origin, and to perform spatial alignment processing on the multi-source sensing data with the position of the limiting reference component when the rear wheel of the target autonomous vehicle abuts against the limiting reference component of the target unloading station as the spatial reference line.

[0130] Optionally, the first detection confidence calculation module 320 is specifically used for: calculating the load residual rate of the target unmanned vehicle during the lifting steady-state holding phase based on the oil spring pressure data; determining the acoustic timing characteristic index of the target unmanned vehicle during the unloading process at the target unloading station based on the acoustic feature data; calculating the amplitude recovery rate and timing matching degree of the target unmanned vehicle during the unloading process at the target unloading station based on the acoustic timing characteristic index; calculating the sound source energy proportion of the target area based on the acoustic feature data, the vehicle pose data, and the reference geometric configuration parameters; and performing a weighted summation of the load residual rate, amplitude recovery rate, timing matching degree, and sound source energy proportion of the target area during the lifting steady-state holding phase of the target unmanned vehicle to obtain the first detection confidence of the target unmanned vehicle in the unloading state at the target unloading station.

[0131] Optionally, the second detection confidence calculation module 330 is specifically used for: determining the unloading end time of the target unmanned vehicle within the limiting reference component based on the acoustic feature data and the oil spring pressure data; determining the material circumscribed rectangular area (ROI) based on the vehicle pose data at the unloading end time within the limiting reference component and the reference geometric configuration parameters; determining target point cloud data based on the first detection confidence, the material circumscribed rectangular area (ROI), and the point cloud data; and determining the second detection confidence of the target unmanned vehicle in the unloading state at the target unloading station based on the target point cloud data.

[0132] Optionally, the second detection confidence calculation module 330 is further configured to: when it is determined that the first detection confidence is greater than or equal to the first preset confidence threshold, and when it is determined that the first detection confidence is less than or equal to the second preset confidence threshold, extend the material circumscribed rectangle ROI outward by a first preset distance to obtain a target material circumscribed rectangle ROI; when it is determined that the first detection confidence is greater than the second preset confidence threshold, extend the material circumscribed rectangle ROI outward by a second preset distance to obtain the target material circumscribed rectangle ROI; and use the point cloud data within the target material circumscribed rectangle ROI as the target point cloud data.

[0133] Optionally, the second detection confidence calculation module 330 is further configured to: determine the maximum height of the point cloud, the projected area of ​​the point cloud, and the effective proportion of the point cloud based on the target point cloud data; and calculate the second detection confidence of the target unmanned vehicle in the unloading state at the target unloading station based on the maximum height of the point cloud, the projected area of ​​the point cloud, and the effective proportion of the point cloud.

[0134] Optionally, the actual unloading status determination module 340 is specifically used for: determining a dust assessment coefficient based on the effective proportion of the point cloud; determining the weight of each detection confidence level based on the dust assessment coefficient; performing a weighted summation of the first detection confidence level and the second detection confidence level based on the weight of each detection confidence level to obtain a comprehensive detection confidence level; and determining the actual unloading status of the target unmanned vehicle at the target unloading station based on the comprehensive detection confidence level.

[0135] Optionally, the actual unloading status determination module 340 is further configured to: determine the actual unloading status of the target unmanned vehicle at the target unloading station as an abnormal material unloading status when the comprehensive detection confidence level is greater than or equal to a third preset confidence threshold, and output material space geometric parameters; determine the actual unloading status of the target unmanned vehicle at the target unloading station as a pending status when the comprehensive detection confidence level is less than the third preset confidence threshold and greater than a fourth preset confidence threshold; reacquire target point cloud data and update the comprehensive detection confidence level based on the target point cloud data; determine the actual unloading status of the target unmanned vehicle at the target unloading station based on the updated comprehensive detection confidence level; and determine the actual unloading status of the target unmanned vehicle at the target unloading station as a normal material unloading status when the comprehensive detection confidence level is less than or equal to the fourth preset confidence threshold.

[0136] The aforementioned unloading status determination device for autonomous vehicles can execute the unloading status determination method for autonomous vehicles provided in any embodiment of the present invention, and possesses the corresponding functional modules and beneficial effects of the method. Technical details not described in detail in this embodiment can be found in the unloading status determination method for autonomous vehicles provided in any embodiment of the present invention.

[0137] Since the unmanned vehicle unloading status determination device described above is an apparatus capable of executing the unmanned vehicle unloading status determination method in the embodiments of the present invention, those skilled in the art can understand the specific implementation and various variations of the unmanned vehicle unloading status determination device in this embodiment based on the unmanned vehicle unloading status determination method described in the embodiments of the present invention. Therefore, how the unmanned vehicle unloading status determination device implements the unmanned vehicle unloading status determination method in the embodiments of the present invention will not be described in detail here. Any apparatus used by those skilled in the art to implement the unmanned vehicle unloading status determination method in the embodiments of the present invention falls within the scope of protection of this application.

[0138] Example 4

[0139] Figure 6A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0140] like Figure 6 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as ROM 12, RAM 13, etc., communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded into the RAM 13 from the storage unit 18. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An I / O interface 15 is also connected to the bus 14.

[0141] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0142] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as the unloading status determination method for an autonomous vehicle.

[0143] In some embodiments, the autonomous vehicle unloading status determination method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the autonomous vehicle unloading status determination method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the autonomous vehicle unloading status determination method by any other suitable means (e.g., by means of firmware).

[0144] Optionally, the method for determining the unloading status of an autonomous vehicle, applied to an autonomous vehicle, may include: acquiring multi-source perception data of the target autonomous vehicle during the unloading process at a target unloading station and the reference geometric configuration parameters of the target autonomous vehicle; wherein, the multi-source perception data includes acoustic feature data, hydraulic spring pressure data, vehicle pose data, and point cloud data; calculating a first detection confidence level of the unloading status of the target autonomous vehicle at the target unloading station based on the acoustic feature data, the hydraulic spring pressure data, the vehicle pose data, and the reference geometric configuration parameters; if the first detection confidence level is determined to be greater than or equal to a first preset confidence threshold, calculating a second detection confidence level of the unloading status of the target autonomous vehicle at the target unloading station based on the first detection confidence level, the multi-source perception data, and the reference geometric configuration parameters; and determining the actual unloading status of the target autonomous vehicle at the target unloading station based on the first detection confidence level and the second detection confidence level.

[0145] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0146] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0147] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0148] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0149] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0150] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0151] This application also discloses a computer program product, which includes a computer program that, when executed by a processor, implements the unloading status determination method for autonomous vehicles provided in any embodiment of this application. This program product shares the same inventive concept as the unloading status determination method for autonomous vehicles disclosed in the embodiments of this application, and therefore will not be described in detail here.

[0152] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0153] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for determining the unloading status of an unmanned vehicle, characterized in that, Applications in autonomous vehicles include: Acquire multi-source perception data of the target unmanned vehicle during the unloading process at the target unloading station and the reference geometric configuration parameters of the target unmanned vehicle; wherein, the multi-source perception data includes acoustic feature data, hydraulic spring pressure data, vehicle pose data and point cloud data; The first detection confidence level of the target unmanned vehicle in the unloading state at the target unloading station is calculated based on the acoustic feature data, the oil spring pressure data, the vehicle pose data, and the reference geometric configuration parameters. If the first detection confidence level is determined to be greater than or equal to the first preset confidence threshold, the second detection confidence level of the target unmanned vehicle in the unloading state at the target unloading station is calculated based on the first detection confidence level, the multi-source perception data and the reference geometric configuration parameters. The actual unloading status of the target unmanned vehicle at the target unloading station is determined based on the first detection confidence level and the second detection confidence level.

2. The method according to claim 1, characterized in that, After acquiring the multi-source perception data of the target unmanned vehicle during the unloading process at the target unloading station, the method further includes: Using the unified clock of the on-board domain controller of the target autonomous vehicle as a reference, timing alignment processing is performed on the multi-source sensing data; The multi-source sensing data is subjected to time-series interval division processing; wherein, the time-series interval includes the docking and positioning stage, the staged lifting and unloading stage, the lifting steady-state holding stage, and the low-speed forward re-unloading stage; A reference coordinate system is established with the rear axle center of the target unmanned vehicle as the origin, and the position of the limit reference component when the rear wheel of the target unmanned vehicle abuts against the limit reference component of the target unloading station is used as a spatial reference line to perform spatial alignment processing on the multi-source sensing data.

3. The method according to claim 1, characterized in that, The calculation of the first detection confidence level of the target unmanned vehicle in the unloading state at the target unloading station based on the acoustic feature data, the oil spring pressure data, the vehicle pose data, and the reference geometric configuration parameters includes: The load residual rate of the target unmanned vehicle during the lifting steady-state holding phase is calculated based on the oil spring pressure data. Based on the acoustic feature data, determine the acoustic timing feature indicators of the target unmanned vehicle during the unloading process at the target unloading station; The amplitude recovery rate and timing matching degree of the target unmanned vehicle during the unloading process at the target unloading station are calculated based on the acoustic timing characteristic index. The energy percentage of the sound source in the target area is calculated based on the acoustic feature data, the vehicle pose data, and the reference geometric configuration parameters. The first detection confidence level of the target unmanned vehicle in the unloading state at the target unloading station is obtained by weighted summing of the load residual rate, amplitude recovery rate, temporal matching degree, and sound source energy ratio of the target unmanned vehicle during the lifting steady-state holding phase.

4. The method according to claim 1, characterized in that, The step of calculating the second detection confidence level of the target unmanned vehicle at the target unloading station based on the first detection confidence level, the multi-source sensing data, and the reference geometric configuration parameters includes: The unloading end time of the target unmanned vehicle within the limiting reference component is determined based on the acoustic feature data and the oil spring pressure data. The region of interest (ROI) of the material's outer rectangle is determined based on the vehicle's pose data at the end of unloading within the limiting reference component and the reference geometric configuration parameters. The target point cloud data is determined based on the first detection confidence level, the material's circumscribed rectangle ROI, and the point cloud data. The second detection confidence level of the target unmanned vehicle in the unloading state of the target unloading station is determined based on the target point cloud data.

5. The method according to claim 4, characterized in that, The step of determining the target point cloud data based on the first detection confidence level, the material's circumscribed rectangle (ROI), and the point cloud data includes: If it is determined that the first detection confidence level is greater than or equal to the first preset confidence threshold, and the first detection confidence level is less than or equal to the second preset confidence threshold, the material circumscribed rectangle ROI is extended outward by a first preset distance to serve as the target material circumscribed rectangle ROI. If the first detection confidence level is determined to be greater than the second preset confidence threshold, the outer bounding rectangle ROI of the material is extended outward by a second preset distance to serve as the outer bounding rectangle ROI of the target material. The point cloud data within the bounding rectangle ROI of the target material is used as the target point cloud data.

6. The method according to claim 4, characterized in that, The second detection confidence level for determining the unloading status of the target unmanned vehicle at the target unloading station based on the target point cloud data includes: The maximum height of the point cloud, the projected area of ​​the point cloud, and the effective proportion of the point cloud are determined based on the target point cloud data. The second detection confidence level of the target unmanned vehicle in the unloading state at the target unloading station is calculated based on the maximum height of the point cloud, the projected area of ​​the point cloud, and the effective proportion of the point cloud.

7. The method according to claim 6, characterized in that, Determining the actual unloading status of the target unmanned vehicle at the target unloading station based on the first detection confidence level and the second detection confidence level includes: The dust assessment coefficient is determined based on the effective proportion of point clouds. The weight of each detection confidence level is determined based on the dust assessment coefficient. The first detection confidence and the second detection confidence are weighted and summed according to the weight of each detection confidence to obtain the comprehensive detection confidence. The actual unloading status of the target unmanned vehicle at the target unloading station is determined based on the comprehensive detection confidence level.

8. The method according to claim 7, characterized in that, Determining the actual unloading status of the target unmanned vehicle at the target unloading station based on the comprehensive detection confidence level includes: If the comprehensive detection confidence level is greater than or equal to the third preset confidence threshold, the actual unloading state of the target unmanned vehicle at the target unloading station is determined to be an abnormal material unloading state, and the material space geometric parameters are output. If the overall detection confidence level is determined to be less than the third preset confidence threshold and greater than the fourth preset confidence threshold, the actual unloading status of the target unmanned vehicle at the target unloading station is determined to be pending; the target point cloud data is reacquired, and the overall detection confidence level is updated based on the target point cloud data; the actual unloading status of the target unmanned vehicle at the target unloading station is determined based on the updated overall detection confidence level. If the overall detection confidence level is less than or equal to the fourth preset confidence threshold, the actual unloading status of the target unmanned vehicle at the target unloading station is determined to be the normal material unloading status.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the unloading status determination method for an unmanned vehicle as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the unloading status determination method for any one of claims 1-8.

11. A computer program product, characterized in that, It includes a computer program / instruction, wherein when the computer program / instruction is executed by a processor, it implements the unloading status determination method for an unmanned vehicle as described in any one of claims 1-8.

Images