A garbage can precise snapshot triggering method based on mechanical rod lifting motion feature recognition

Abstract

The application provides a garbage can accurate snapshot triggering method based on mechanical lifting rod motion feature recognition, collects mechanical lifting rod posture data and continuous image frames and unifies to the same time reference, performs continuous and direction normalization processing on the posture angle sequence, generates continuous motion phase and determines frame phase; establishes a dumping process in combination with vehicle state and mechanical lifting rod state, evaluates image frame quality in the candidate snapshot phase area and updates the candidate snapshot image; confirms the garbage can dumping action in the end phase area according to the complete motion path and action constraint, confirms the solidified candidate snapshot image after backtracking and executes the reset locking. The application can reduce false triggering and invalid snapshot, improve the image quality and reliability of the dumping snapshot record.

Description

Technical Field

[0001] This invention belongs to the field of vehicle intelligent sensing technology, and in particular relates to a method for accurately capturing and triggering images of trash cans based on the recognition of mechanical lever motion features. Background Technology

[0002] Garbage collection trucks typically use mechanical booms, lifting arms, or tipping mechanisms to hold garbage bins, sequentially lifting, tipping, dumping, and returning them to their original position. To create collection records, performance evaluation records, or evidence for handling abnormalities, images clearly showing the relative relationships between the garbage bins, mechanical booms, and the garbage container are needed during each actual dumping.

[0003] Existing image capture methods typically include manual button capture, fixed-period capture, single-position switch triggering, single-angle threshold triggering, and continuous image recognition triggering. Manual button capture is easily affected by missed operations; fixed-period capture generates a large number of images unrelated to tilting; single-position switch triggering or single-angle threshold triggering has difficulty distinguishing between complete tilting, boundary jitter, mid-trip retraction, and mechanical jamming; continuous image recognition triggering is easily limited by lighting, stains, occlusion, and onboard processing capabilities.

[0004] To avoid capturing images where the trash can is obscured by the garbage container or has moved from its optimal framing position at the end of the dumping process, images can be cached during the process and saved afterward. However, simply saving the latest image before the action occurs cannot guarantee image sharpness, exposure effectiveness, or the visibility of the trash can, nor can it guarantee that the action to which the cached image belongs will ultimately complete the full dumping process.

[0005] Installing attitude sensors on the mechanical lever can acquire attitude angles, acceleration, and angular velocity. However, attitude angles are often expressed in terms of -180 degrees to +180 degrees. When mechanical movement crosses this boundary, there can be a near 360-degree jump in value. Different installation orientations may also cause the same tilting action to appear as an increasing or decreasing angle. Directly judging based on the absolute angle range will affect the calculation of motion direction, trajectory length, retraction amount, and phase velocity. In addition, when the vehicle is moving smoothly, the combined amplitude of the three-axis acceleration may also be close to the acceleration due to gravity. Judging whether the vehicle is stationary solely based on whether the combined amplitude is close to a gravitational acceleration is difficult to reliably rule out false triggering during movement. Continuing to use the stationary condition after the action has started may mistakenly identify the acceleration generated by the mechanical lever's own movement as vehicle movement, thus stopping the normal action.

[0006] Therefore, this technical solution designs a method for precise capture triggering of trash cans based on the recognition of mechanical lifting lever motion features. Summary of the Invention

[0007] To achieve the objectives of this invention, the following technical solution is adopted: This invention provides a precise method for triggering garbage bin image capture based on the motion feature recognition of a mechanical lifting lever, comprising the following steps: S1, acquiring mechanical lifting lever posture data and continuous image frames, and unifying the posture data and image frames to the same time reference; S2, performing continuous and direction normalization processing on the posture angle sequence to generate continuous motion phases of the mechanical lifting lever and determining the frame phase of each image frame; S3, determining the operation enable based on the vehicle status and mechanical lifting lever status, and establishing the current dumping process after meeting the pre-enablement and initial motion conditions; S4, evaluating the image frame quality within the candidate capture phase area, and continuously updating and saving candidate capture images based on frame phase reliability; S5, confirming the garbage bin dumping action within the end phase area based on the complete motion path and action constraints, and retrospectively solidifying the candidate capture images; S6, encapsulating dumping event data after confirming the complete garbage bin dumping action; executing a reset lock after confirming the complete garbage bin dumping action or after abnormal termination, wherein, in the case of abnormal termination, no formal dumping event data is formed to control the activation of the next garbage bin dumping recognition.

[0008] Preferably, in step S2, during the installation and calibration stage, the sensitive attitude axis with the highest correlation to the mechanical lifting and tilting motion is selected from multiple attitude axes of the inertial measurement unit (IMU). During the operation phase, the k-th original attitude angle θk is read from the sensitive attitude axis, and the original angle difference Δθk = θk - θ(k-1) between adjacent samples is calculated. When Δθk>180°, Δθ′k=Δθk-360°; When Δθk<-180°, Δθ′k=Δθk+360°; When -180°<Δθk<180°, Δθ′k=Δθk; When |Δθk|=180° or the actual angular displacement of adjacent samples may reach 180°, the corresponding sample is marked as having an uncertain angular direction, and the correction result is determined by combining the gyroscope angular velocity sign or the motion direction of multiple samples before and after. The corrected angle difference is accumulated to obtain the continuous angle uk=u(k-1)+Δθ′k, and the direction is normalized according to φk=d·(uk-uref), where uref represents the continuous angle reference value when the mechanical lifting rod is in the reset position, and d takes +1 or -1, so that φk increases in total during the normal tilting process.

[0009] Preferably, in step S1, the vehicle-mounted processor uses a monotonic clock as a unified time reference; when the inertial measurement unit (IMU) and the vehicle-mounted camera are directly triggered by the vehicle-mounted processor, the monotonic clock time is recorded at the triggering moment; when the IMU and the vehicle-mounted camera have their own clocks, the corresponding points between the device time and the monotonic clock time are periodically acquired, and a linear time mapping or a piecewise linear time mapping is established based on the most recent multiple corresponding points to convert the device time into monotonic clock time; when the vehicle-mounted camera can output the exposure start time and the exposure end time, the image frame representative time tf is determined according to the midpoint of the exposure time interval. When the vehicle-mounted camera can only output the image acquisition completion time, encoding completion time, or the time tc when the vehicle-mounted processor receives the image frame, the representative time of the image frame is determined according to tf = tc - δc, where δc is the fixed delay of the camera determined during the installation and calibration stage; assuming ti ≤ tf ≤ t(i+1), the frame phase φf is calculated according to φf = φi + [(tf - ti) / (t(i+1) - ti)]·(φ(i+1) - φi), and interpolation is performed only when t(i+1) > ti and t(i+1) - ti does not exceed the maximum interpolation interval Tinterp.

[0010] Preferably, the frame phase uncertainty εf is calculated as follows: εf=|vf|·(em+ec+texp / 2)+(a(φ,max) / 8)·(Δti) 2 +eφ is calculated, where vf is the continuous motion phase velocity near time tf of the image frame, em is the upper bound of the IMU time mapping error, ec is the upper bound of the camera time mapping error, texp is the exposure time, a(φ,max) is the upper bound of the absolute value of the phase angular acceleration within the interpolation time period, Δti is the time interval between adjacent attitude samples, and eφ is the upper bound of the error after the attitude measurement and installation errors are converted to the continuous motion phase; the actual phase of the image frame is represented as [φf-εf,φf+εf]; When the actual phase does not intersect with the candidate capture phase region, the image frame is not included in the candidate evaluation; when the actual phase intersects with the candidate capture phase region but εf is greater than the upper limit of allowable uncertainty εallow, the image frame is marked as frame phase unreliable and is not used as a candidate capture image.

[0011] Preferably, in step S3, the root mean square ωr of the gravity residual rg, acceleration dispersion σa, and triaxial angular velocity composite value is calculated within the sliding time window, and the job enable confidence Ce is calculated according to Ce = wgFg + waFa + wωFω + wvFv + wsFs, where wg + wa + wω + wv + ws = 1, and each weight is a non-negative number; Fg = max(0, 1 - rg / Rg), Fa = max(0, 1 - σa / Ag), Fω = max(0, 1 - ωr / Ωg), where Rg, Ag, and Ωg are the corresponding normalization thresholds; the vehicle speed confidence value Fv is determined based on the vehicle speed |vveh| and preset speed thresholds v0 and v1: when |vveh|≤v0, Fv = 1; when v0 < |vveh| < v1, Fv = (v1 - |vveh|) / (v1 - v0); when |vveh|≥v1, Fv = 0; the discrete state confidence value Fs is determined based on one or more of the following: parking brake, gear position, mechanical lever in-situ state, and hydraulic operation enabled state; when any data is unavailable, the weight corresponding to that item is set to 0, and the remaining weights are renormalized; The system sets an independent operation gate state Gop. When any of the hydraulic operation enable state, power take-off state, or mechanical lever control command is valid and clearly indicates that the tilting operation is allowed, Gop = 1. Alternatively, when the aforementioned operation enable signal cannot be obtained, but the vehicle speed is within the preset low speed range or the vehicle is in a parked state, and the mechanical lever in place state is valid, Gop = 1. When Gop = 1, Ce reaches the pre-enabling threshold, and the continuous motion phase φk is in the initial phase region and continues to meet the pre-enabling holding time, the pre-enabling state is entered, and the continuous motion phase when entering the pre-enabling state is recorded as the initial continuous motion phase φ0. In the pre-enabling state, when φk - φ0 ≥ Δφstart, the continuous motion phase velocity v(φ,k) ≥ vstart, and the motion holding time Tstart is continuously met, the start time of the action and the current calibration parameter version are recorded, and the current tilting process is established.

[0012] Preferably, in step S4, an image frame is determined as a valid candidate image frame when it simultaneously satisfies the following conditions: target visibility index Vf ≥ target visibility minimum threshold Vmin, occlusion index Of ≤ maximum allowable occlusion ratio Omax, motion blur index Bf ≤ maximum allowable motion blur degree Bmax, and frame phase uncertainty εf ≤ allowable uncertainty upper limit εallow. For image frames that meet the candidate validity conditions, the comprehensive quality score Qf is calculated according to: Qf=αsSf+αeEf+αvVf+αb(1-Bf)+αo(1-Of)+αp(1-Df)+αu(1-Uf); where αs+αe+αv+αb+αo+αp+αu=1, Df=min(|φf-φ*| / Wc,1), Uf=min(εf / εallow,1), Sf is the sharpness index, Ef is the exposure effectiveness index, φ* is the preferred capture phase, and Wc>0.

[0013] Preferably, before candidate images are formed during the current dumping process, the candidate state is set to invalid, and the first image frame that simultaneously meets the candidate validity condition and Qf≥Qmin is saved as the current candidate image, and its comprehensive quality score is set to Qbest. Subsequently, when a new effective candidate image frame simultaneously meets Qf≥Qmin and Qf>Qbest, the current candidate image is replaced by the new effective candidate image frame, and Qbest is updated to Qf. When the comprehensive quality scores of the new effective candidate image frame and the current candidate image are the same, the image frame with the smaller occlusion index Of is retained first. When the occlusion indices are the same, the image frame with the smaller frame phase uncertainty εf is retained first. When the frame phase uncertainty is still the same, the current candidate image is retained.

[0014] Preferably, in step S5, when the current tilting process is established, the record is recorded as having entered the starting phase zone, and when the continuous motion phase or frame phase interval intersects with the candidate capture phase zone, the record is recorded as having passed the candidate capture phase zone. Only after all the aforementioned records are valid can the entry into the ending phase zone be qualified for confirmation. The motion direction consistency rate Rd is calculated according to Rd = Nsame / Neff, where Nsame represents the number of samples in the effective phase increment whose signs are consistent with the tilting direction, and Neff represents the total number of effective phase increments whose absolute value is greater than the phase noise threshold. Rd is calculated only when Neff ≥ Nmin > 0, and Rd is not lower than the direction consistency rate threshold. The motion duration Ta is between the preset shortest motion duration Tmin and the preset longest motion duration Tmax. After the continuous motion phase enters the ending phase region, it continues to maintain the ending duration Te; the maximum phase backoff amount Rmax is calculated according to Rmax=max(1≤q≤K)[max(1≤p≤q)(φp)-φq] and does not exceed the backoff threshold Rth, where K is the number of effective phase samples in this tilting process, and φp and φq are the continuous motion phases corresponding to the p-th and q-th effective phase samples, respectively; this action is divided into M sampling positions according to the normalized time progress, τm=(tm-t s) / (te-ts), the trajectory deviation Dtr is calculated according to Dtr=(1 / M)·Σ(m=1)^M|φ(τm)-φref(τm)| / L and does not exceed the trajectory deviation threshold, where M is the number of sampling positions in the normalized time progress, ts and te are the start time and provisional end time of the action respectively, τm is the normalized time progress of the m-th sampling position, φref(τm) is the continuous motion phase of the reference trajectory at τm, and L is the normal tilting reference full stroke and L>0; Before confirming the complete trash can emptying action, the image content, unified timestamp, frame phase, frame phase uncertainty, and image quality sub-items of the candidate captured image are all complete and valid; when the aforementioned confirmation qualifications, motion direction consistency rate, action duration, end hold duration, maximum phase backoff amount, trajectory deviation, and candidate captured image validity are all satisfied, the complete trash can emptying action is confirmed.

[0015] Preferably, the current tilting process is terminated when the continuous missing time of the attitude data sample exceeds the attitude missing threshold, the unified timestamp becomes non-incremental, or the attitude data cannot be used to generate continuous motion phases. When no valid candidate image has been formed and the continuous missing time of image frames exceeds the image missing threshold, the current tilting process is terminated; when a valid candidate image has been formed and subsequent image frames are temporarily missing, but the image content, unified timestamp, frame phase, frame phase uncertainty and image quality sub-item results of the candidate image remain complete and valid, the complete action confirmation is continued based on the attitude data. After the complete dumping action is confirmed, a dumping event data packet is generated. The persistence task first writes to a temporary file or temporary record. After the candidate captured image, metadata and digital digest are all successfully written, the atomic commit is performed. The system enters reset lockout after a complete tilting action is confirmed or the current tilting process is abnormally terminated. When abnormally terminated, no formal tilting event data packet is generated, but abnormal diagnostic information including the reason for termination, the last continuous motion phase, the maximum phase back-off amount, and data missing information is saved. The reset lockout is released only when the continuous motion phase returns to the reset phase area and continues to reach the reset hold time Tr, or after the cooling time Tc and the mechanical lifting lever is detected to be in its original position.

[0016] The beneficial effects of the present invention are as follows: by unifying the mechanical lifting pole attitude data and continuous image frames to the same time reference, and performing continuous and direction normalization processing on the attitude angle sequence, it is possible to reduce the impact of attitude angles crossing the boundary from negative 180 degrees to positive 180 degrees and different installation directions on the calculation of motion direction, trajectory length, backtracking amount and phase velocity. By combining vehicle status, mechanical lever status, operation enable confidence, and independent operation gating status to establish the dumping process, false triggering caused by smooth vehicle driving, road impact, and mechanical vibration can be reduced. By calculating frame phase and frame phase uncertainty, candidate validity is first judged within the candidate capture phase area, and then the candidate capture image with the highest comprehensive quality score is selected based on sharpness, exposure effectiveness, target visibility, motion blur, occlusion, phase deviation, and phase uncertainty. This can improve the visibility of the relative relationship between the trash can, mechanical lever, and trash container. By confirming the complete trash can dumping action based on the phase zone passing sequence, motion direction consistency, action duration, end hold, and maximum phase back-off amount in the end phase zone, and by retrospectively fixing the previously saved candidate capture images after confirmation, it is possible to avoid midway back-off, mechanical jamming, or incomplete action forming a formal dumping capture record. By encapsulating event data, atomic committing, and resetting lockouts, the integrity of dumping event data can be improved and duplicate identification of adjacent dumping processes can be reduced. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the overall process of a method for accurately capturing and triggering images of trash cans based on the recognition of the motion features of a mechanical lifting lever. Detailed Implementation

[0018] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0019] The following parameter ranges, thresholds, and calculation methods are used to illustrate possible implementation schemes and do not constitute a limitation on the scope of protection; while maintaining the technical relationship between each step, those skilled in the art can make adjustments based on vehicle model, mechanical lifting structure, camera perspective, and processor capability.

[0020] In this manual, the Inertial Measurement Unit (IMU) is the Chinese equivalent of the English term. The IMU is used to output attitude angles, three-axis acceleration, and three-axis angular velocity. A monotonic clock is a timekeeping reference that increases only in one direction and does not regress or jump due to system date correction. All letters appearing thereafter are symbols for calculation variables or parameters, and will be explained one by one after the corresponding formula.

[0021] Example 1 This invention provides a method for accurately capturing images of trash cans based on the recognition of mechanical lever motion features, specifically including the following steps: S1. Collect mechanical lever posture data and continuous image frames, and unify the posture data and image frames to the same time reference; like Figure 1As shown, an IMU is installed on a mechanical lifting arm or a rigid structural component that moves synchronously with the mechanical lifting arm, and a vehicle-mounted camera is installed at a position that can capture the relative relationship between the trash can, the mechanical lifting arm, and the trash container; wherein, the IMU preferably outputs data at a frequency of 50 to 200 times per second, and the vehicle-mounted camera preferably continuously acquires images at a frequency of 10 to 30 frames per second.

[0022] For example, this embodiment can use an IMU sampling period of 20 milliseconds and a camera frame rate of 25 frames per second.

[0023] Each attitude data point includes at least the attitude angle, three-axis acceleration, three-axis angular velocity, sampling sequence number, and timestamp; each image frame includes at least the image content, frame number, and image timestamp; the image timestamp can be the exposure start time and exposure end time, the image acquisition completion time, the encoding completion time, or the time when the on-board processor receives the image frame; for non-exposure timestamps, time compensation is performed according to the fixed camera delay obtained by calibration.

[0024] Preferably, the midpoint of the exposure time interval is used as the image frame acquisition time, so that the mechanical position corresponding to the image content is more accurate.

[0025] Furthermore, the vehicle processor uses a monotonic clock as a unified time reference. If the IMU and the vehicle camera are directly triggered by the vehicle processor, the monotonic clock time is recorded at the trigger time. If the devices have their own clocks, the corresponding points between the device time and the monotonic clock time are periodically obtained, and a linear or piecewise linear time mapping is established based on the most recent multiple corresponding points to convert the device time into monotonic clock time.

[0026] When the vehicle-mounted camera can output the start and end times of exposure, the image frame representative time tf is determined according to the midpoint of the exposure time interval.

[0027] When the vehicle-mounted camera cannot directly output the exposure timestamp, but can only output the time when image acquisition is completed, the time when encoding is completed, or the time when the vehicle-mounted processor receives the image frame tc, the fixed camera delay δc relative to the midpoint of the image exposure is determined during the installation and calibration phase, and the time represented by the image frame is calculated according to the following formula: tf = tc - δc.

[0028] The fixed camera delay δc can be determined based on the median value of multiple synchronous acquisition results. The maximum deviation or upper quantile deviation of the multiple synchronous acquisition results relative to the median value is denoted as camera delay jitter jc, and jc is included in the upper bound of the camera time mapping error ec. When the vehicle-mounted camera uses a rolling shutter mode for exposure, the exposure midpoint corresponding to the preset image area where the trash can is located can be used as the representative time of the image frame; when line-by-line exposure information cannot be obtained, the line-by-line exposure time span of a frame or its upper bound is included in the upper bound of the camera time mapping error ec.

[0029] By compensating for the fixed camera delay and incorporating camera delay jitter and rolling shutter error into ec, the frame phase uncertainty εf can reflect the error between the actual exposure time of the image and the image timestamp.

[0030] It should be noted that the time mapping process saves the average mapping residual, the maximum mapping residual, and the most recent synchronization time. The average mapping residual is used to reflect the time error under normal circumstances, the maximum mapping residual is used to identify abnormal synchronization points, and the most recent synchronization time is used to limit long-term drift in the time mapping. Synchronization points exceeding the allowable residual are not included in the mapping parameter update.

[0031] Attitude data acquisition and image frame acquisition can be performed through independent processing tasks; for example, attitude data enters an attitude buffer sorted by time, and image frames enter an image circular buffer sorted by time; the circular buffer refers to a storage space with a fixed capacity that is used cyclically in a manner that deletes the earliest data after it is full.

[0032] It is particularly important to note that each cache uses a unified timestamp as an index to avoid data mismatch caused by inconsistent processing speeds of different collection tasks.

[0033] Furthermore, when the duration of consecutive missing attitude data samples exceeds the attitude missing threshold, the unified timestamp exhibits a non-incremental phenomenon, or the attitude data cannot be used to generate continuous motion phases, the system marks the current tilting process as incomplete and terminates the current tilting process.

[0034] If a valid candidate image has not yet been generated, and the continuous time of missing image frames exceeds the image missing threshold, the current tilting process is terminated and temporary candidate data is deleted.

[0035] Once a valid candidate image has been formed, if subsequent image frames are temporarily missing, but the image content, unified timestamp, frame phase, frame phase uncertainty, and image quality sub-items of the candidate image remain complete and valid, the current tilting process will not be terminated due to the missing subsequent image frames, and the system will continue to perform complete action confirmation based on the attitude data.

[0036] When the candidate image that has been formed is damaged, the time or phase data corresponding to the candidate image is invalid, or the image data is missing and there are no valid candidate images, the current tilting process is terminated and the candidate data that has not yet been solidified is deleted. Preferably, the attitude loss threshold Timu is set based on the IMU nominal sampling period Ts, and is preferably 3Ts to 5Ts; The image missing threshold Timg is set based on the camera's nominal frame period Tf, preferably 3Tf-5Tf. This threshold can also be determined by adding a safety margin to the upper quantile of the time interval between adjacent valid samples in normal operating data. Timu, Ts, Timg, and Tf are all in seconds (s) and are stored as calibration parameters. To facilitate subsequent verification, the system records the IMU installation identifier, vehicle-mounted camera identifier, sampling frequency, time mapping version, and calibration parameter version each time it runs. This recording ensures that even if hardware is replaced or parameters are updated, the acquisition conditions corresponding to a specific snapshot can still be clearly identified.

[0037] S2. Perform continuous and direction normalization processing on the attitude angle sequence to generate the continuous motion phase of the mechanical lifting lever and determine the frame phase of each image frame. Specifically, during the installation and calibration phase, the sensitive attitude axis with the highest correlation to the mechanical lever tilting motion is selected from multiple attitude axes of the IMU; during the operation phase, the k-th original attitude angle θk is read from the sensitive attitude axis, and the original attitude angle is represented in the range of [-180°, 180°].

[0038] Calculate the original angle difference between adjacent samples: Δθk = θk - θ(k-1) In the formula, k represents the attitude sample number, θk represents the original angle of the sensitive attitude axis of the kth attitude sample, θ(k-1) represents the original angle of the sensitive attitude axis of the previous attitude sample, and Δθk represents the difference between the two original angles. The angle unit is degrees (°).

[0039] To eliminate the numerical jump caused when the attitude angle crosses the ±180° boundary, the original angle difference is corrected as follows, provided that the attitude sampling frequency is sufficient to ensure that the absolute value of the true angular displacement between two adjacent samples of the mechanical lifting lever is less than 180°: When Δθk > 180°, Δθ′k = Δθk - 360°; When Δθk < -180°, Δθ′k = Δθk + 360°; When -180° < Δθk < 180°, Δθ′k = Δθk.

[0040] When |Δθk|=180° or the actual angular displacement of adjacent samples may reach 180°, the sample is marked as having an uncertain angular direction, and the correction result is determined by combining the gyroscope angular velocity sign or the motion direction of multiple samples before and after.

[0041] In the formula, Δθ′k represents the angle difference after boundary correction, in degrees (°); when the original angle jumps from close to +180° to close to -180°, the original difference is close to -360°, so 360° should be added; when the original angle jumps from close to -180° to close to +180°, the original difference is close to +360°, so 360° should be subtracted.

[0042] For example, when the original angle changes from +179° to -179°, the original difference is -358°, and adding 360° results in +2°; when the original angle changes from -179° to +179°, the original difference is +358°, and subtracting 360° results in -2°.

[0043] The corrected angle differences are accumulated to obtain continuous angles, and then normalized according to the direction of the tilting process: uk = u(k-1) + Δθ′k φk = d · (uk - uref); In the formula, uk, u(k-1), uref, and φk are all in degrees (°); uref represents the continuous angular reference value when the mechanical lifting rod is in the reset position; d is a dimensionless direction sign, which is +1 or -1, so that φk increases overall during the normal tilting process.

[0044] To suppress mechanical vibration and quantization noise, Δθ′k can be filtered using a three- to seven-point median filter, or a maximum variation limit can be set for a single sample. Angle differences with absolute values ​​less than the phase noise threshold can be considered zero, but the untrunculated original data should be retained for anomaly analysis.

[0045] The continuous motion phase velocity is calculated using the continuous motion phase difference between adjacent attitude samples. Let tk be the unified timestamp of the k-th attitude sample, then: v(φ,k)=(φk-φ(k-1)) / (tk-t(k-1)); In the formula, v(φ,k) represents the continuous motion phase velocity corresponding to the k-th attitude sample, in degrees per second (° / s). The continuous motion phase velocity is calculated only when tk > t(k-1); to reduce the influence of noise in a single sample, the calculation results of three to seven consecutive phase velocities can be processed by median filtering or moving average.

[0046] In this manual, φ0 represents the initial value of the continuous motion phase recorded when entering the pre-enabled state. After entering the pre-enabled state, φ0 remains unchanged during this pre-enabled judgment and the established tilting process. After establishing this tilting process, φ0 remains unchanged during this tilting process.

[0047] The frame phase of an image frame is interpolated based on the exposure time of the image frame; let the start time of the exposure of the image frame be t(f,s) and the end time be t(f,e), then the time represented by the image frame is: tf = (t(f,s) + t(f,e)) / 2 Let ti ≤ tf ≤ t(i+1), and Δti = t(i+1) - ti > 0. The frame phase φf is calculated as follows: φf = φi + [(tf - ti) / (t(i+1) - ti)] · (φ(i+1) - φi); In the formula, tf, ti, t(i+1), and Δti are in seconds (s), and φf, φi, and φ(i+1) are in degrees (°). Interpolation is performed only when t(i+1) > ti and the time interval between them does not exceed the maximum interpolation interval Tinterp; otherwise, the image frame is marked as frame phase unavailable. The maximum interpolation interval Tinterp is preset based on the IMU's nominal sampling period Ts, preferably satisfying that Tinterp is no greater than 3Ts. When the IMU uses non-fixed period sampling, Tinterp is determined based on the upper quantile value of the time interval between adjacent valid attitude samples under normal operating conditions and the time safety margin. The units of Tinterp and Ts are both seconds (s).

[0048] Since the frame phase is not an absolutely accurate value, this embodiment further calculates the frame phase uncertainty εf: εf = |vf| · (em + ec + texp / 2) + (a(φ,max) / 8) · (Δti) 2 + eφ In the formula, vf is the continuous motion phase velocity near tf, in degrees per second (° / s), em is the upper bound of IMU time mapping error, in seconds (s), ec is the upper bound of camera time mapping error, in seconds (s), and texp=t(f,e)-t(f,s) is the exposure time, in seconds (s). a(φ,max) is the upper bound of the absolute value of the phase angular acceleration within the interpolation time interval, in degrees per second squared (° / s). 2 eφ is the upper bound of the attitude measurement and installation error converted to the continuous motion phase, in degrees (°), and εf is in degrees (°).

[0049] Among them, (a(φ,max) / 8)·(Δti) 2 Used to limit the curvature error of linear interpolation under non-uniform motion conditions.

[0050] The actual phase of an image frame is represented as an interval [φf-εf, φf+εf]. When this interval does not intersect with the candidate capture phase region, the image frame is not included in the candidate evaluation. When the frame phase interval intersects with the candidate capture phase region, but εf is greater than the upper limit of the allowable uncertainty εallow, the image frame is marked as having an unreliable frame phase and is not used as a candidate capture image. When εf is not greater than εallow, the phase mapping error is reflected in the comprehensive quality score through the phase uncertainty index Uf.

[0051] Furthermore, the installation calibration can also generate a reference phase velocity range; when the phase velocity during operation significantly exceeds the reference range, εf can be increased or the corresponding time period can be marked as abnormal to avoid incorrect phase mapping caused by mechanical shock, sensor loosening or sampling loss.

[0052] S3. Determine the operation enable based on the vehicle status and mechanical lever status, and establish the current tilting process after meeting the pre-enablement and initial motion conditions; Specifically, before the mechanical boom begins to lift, it is determined whether the vehicle and the mechanical boom are in a state that allows the tilting process to begin. Unlike simply determining whether the combined amplitude of the three-axis acceleration is close to a gravitational acceleration, this step simultaneously calculates the gravity residual, acceleration fluctuation, and angular velocity within a sliding time window.

[0053] For the i-th IMU sample in the sliding time window, the composite triaxial acceleration amplitude mi is calculated using the following formula: mi = √(a(x,i) 2 + a(y,i) 2 + a(z,i) 2 ) In the formula, a(x,i), a(y,i), and a(z,i) represent the acceleration of the i-th sample on the three orthogonal measurement axes, respectively, and mi represents the combined amplitude of the triaxial acceleration of the sample; the unit of each acceleration measure is meters per second squared (m / s²). 2 ).

[0054] Within a sliding time window containing n samples, the gravity residual rg and acceleration dispersion σa are calculated using the following formula: rg = |(1 / n) · Σ(i=1)^n mi - g| σa = √{(1 / n) · Σ(i=1)^n [mi - (1 / n) · Σ(q=1)^n mq] 2} In the formula, n represents the number of samples within the sliding time window, g represents the local reference value of gravitational acceleration, and rg and σa are both in meters per second squared (m / s²). 2). The local gravitational acceleration g may adopt the calibrated value at the installation site, and the standard gravitational acceleration of 9.80665 m / s may be used when no calibration is performed 2 .

[0055] The root mean square ωr of the triaxial angular velocity resultant value within the sliding time window is: ωr = √{(1 / n) · Σ(i=1)^n (ω(x,i) 2 + ω(y,i) 2 + ω(z,i) 2 )} In the formula, ω(x,i), ω(y,i) and ω(z,i) are all in radians per second (rad / s), and the unit of ωr is radians per second (rad / s); the vehicle state interface may provide one or more of vehicle speed, gear position, parking brake state, mechanical lifting rod in-situ state and hydraulic enable state.

[0056] The operation enable confidence Ce adopts the following formula: Ce = wgFg + waFa + wωFω + wvFv + wsFs wg + wa + wω + wv + ws = 1 In the formula, the value range of Ce is 0-1; all weights are non-negative numbers. When certain data is not available, the corresponding weight is set to 0 and the remaining weights are re-normalized.

[0057] Fg = max(0, 1 - rg / Rg) Fa = max(0, 1 - σa / Ag) Fω = max(0, 1 - ωr / Ωg) In the formula, the units of Rg and Ag are meters per second squared (m / s 2 ), the unit of Ωg is radians per second (rad / s), and Fg, Fa and Fω are all dimensionless numbers.

[0058] The vehicle speed credibility value Fv may be determined according to the following piecewise function: When |vveh|≤v0, Fv=1; When v0<|vveh|<v1, Fv=(v1-|vveh|) / (v1-v0); When |vveh|≥v1, Fv=0.

[0059] Herein, vveh, v0 and v1 shall uniformly adopt meters per second (m / s) or kilometers per hour (km / h), and no mixed use is allowed in the same formula.

[0060] The discrete state reliability value Fs can be determined based on parking brake, gear position, mechanical lever in-situ position, and hydraulic operation enable. For example, when parking or low speed state, in-situ state, and hydraulic enable state must be satisfied simultaneously, Fs = Ipark·Ihome·Ihyd, where each I is a dimensionless state quantity that takes the value 0 or 1.

[0061] When vehicle speed, parking status, gear position, or independent operation input cannot be obtained, rg, σa, and ωr can only characterize that the IMU is in a low vibration stable state and cannot be used alone to determine that the vehicle is absolutely stationary.

[0062] To prevent false enablement caused by relying solely on the low vibration state of the IMU when the vehicle is driving smoothly, the system further sets up an independent operation gating state, Gop.

[0063] Gop = 1 when any of the following conditions are met: (1) At least one of the following is valid: hydraulic operation enable status, power take-off status, or mechanical lever control command, and it is clearly indicated that the tilting operation is permitted; (2) The aforementioned operation enable signal cannot be obtained, but the vehicle speed is in the preset low speed range or the vehicle is in the parking state, and the mechanical lever is in the original position.

[0064] If any of the above conditions are not met, Gop = 0.

[0065] Entering the pre-enabled state requires not only that the operation enable confidence level Ce reach the pre-enabled threshold, but also that Gop = 1. The gravity residual rg, acceleration dispersion σa, and root mean square angular velocity ωr are only used to evaluate the stability of the environment in which the IMU is located, and cannot individually change Gop from 0 to 1.

[0066] When the vehicle cannot provide any independent operating status, vehicle status, or mechanical lever in place status, the system can continue to calculate continuous motion phases and save diagnostic information, but will not automatically generate a formal garbage bin dumping capture record.

[0067] The operation process employs a two-level establishment mechanism: When Gop = 1, Ce reaches the pre-enable threshold, and φk is in the initial phase region and continues to reach the pre-enable holding time, the pre-enable state is entered. In the pre-enable state, when φk - φ0 ≥ Δφstart, phase velocity v(φ,k) ≥ vstart, and the motion holding time Tstart is continuously satisfied, the start time of the action, the initial continuous motion phase, and the current calibration parameter version are recorded to establish this tilting process.

[0068] Where Δφstart is in degrees (°), vstart is in degrees per second (° / s), and Tstart is in seconds (s). For example, with an IMU sampling period of 20 milliseconds, it is required that 10 consecutive samples meet the motion conditions to form a motion holding time of 200 milliseconds.

[0069] The starting phase zone is set with an entry boundary and an exit boundary, forming a hysteresis band between them. A stricter entry boundary is used when entering the starting phase zone, and a different exit boundary is used when leaving the starting phase zone, in order to avoid mechanical shaking causing the state to switch repeatedly on both sides of the boundary. Hysteresis bands can also be set for the candidate capture phase zone and the end phase zone.

[0070] It should be noted that once the tilting process is established, the system freezes the pre-enabled operation results, meaning the action will no longer be terminated due to changes in acceleration or angular velocity caused by the lever lifting movement. Whether the action continues to be effective thereafter is determined by the continuous motion phase, directional consistency, action duration, retraction amount, and data integrity. Only when the vehicle status clearly indicates that continued operation is not permitted, such as when the vehicle is moving at high speed or the hydraulic system is shut down, can this be used as a forced termination condition.

[0071] If, during the pre-enablement period, the continuous motion phase leaves the initial phase region, the operation enable confidence drops below the exit threshold, or the attitude data is incomplete, the pre-enablement hold count is cleared, and the tipping process is not established. This process helps eliminate short-term entry caused by transient vibrations, road impacts, or sensor noise.

[0072] S4. Evaluate the image frame quality within the candidate capture phase area, and continuously update and save the candidate capture images based on the frame phase reliability. Specifically, after the dumping process is established, image frames are continuously stored in a circular buffer. Each buffer entry records at least the image content, a unified timestamp, frame phase φf, frame phase uncertainty εf, and image quality sub-items. Only image frames whose frame phase interval [φf-εf, φf+εf] intersects with the candidate capture phase region participate in the candidate evaluation. Preferably, the image quality evaluation is conducted from at least four aspects: image sharpness, exposure effectiveness, visibility of the trash can or its opening, and phase position. To improve comparability under different cameras and lighting conditions, each indicator is first converted into a normalized value between 0 and 1.

[0073] It should be noted that the sharpness index Sf is determined based on the gradient energy of adjacent pixels or the variance of the Laplacian operator response in the grayscale image, and the larger the value, the sharper the edges; the exposure effectiveness index Ef is determined inversely based on the proportion of overexposed pixels and the proportion of underexposed pixels, and the value is higher when the brightness distribution of the image is within the identifiable range.

[0074] Secondly, the target visibility index Vf can be determined through two implementation methods: The first method does not deploy a target recognition model; instead, visibility is determined within a pre-defined image region based on foreground area, edge density, contour closure, vertical structure, and texture of the bin opening area. The second method deploys a bin detection or keypoint detection model, determining visibility based on bin detection confidence, bounding box integrity, visibility of keypoints at the bin opening, and target occlusion ratio. When using the model, the input should be clearly defined as an image frame or a pre-defined region image, and the output should include at least one of the following: bin detection bounding box, detection confidence, keypoints at the bin opening, or occlusion ratio.

[0075] For the motion blur index Bf, it is determined based on the degree of loss of high-frequency information in the image or the degree of target edge trailing in consecutive frames; the occlusion index Of is determined based on the proportion of the area of ​​the trash can detection frame covered by the vehicle body or the box body; the phase deviation index Df is determined based on the distance between the frame phase and the preferred capture phase φ*; and the phase uncertainty index Uf is determined based on the ratio of εf to the upper limit of allowable uncertainty.

[0076] All image quality metrics are normalized to [0,1]. Among them, larger values ​​for Sf, Ef, and Vf indicate better quality; larger values ​​for Bf, Of, Df, and Uf indicate a higher degree of deterioration.

[0077] Before calculating the overall quality score Qf, a candidate validity assessment is performed on the image frames. An image frame is determined to be a valid candidate image frame only if it simultaneously meets the following conditions: Vf ≥ Vmin; Of ≤ Omax; Bf ≤ Bmax; εf ≤ εallow.

[0078] In the formula, Vmin represents the minimum threshold for target visibility, Omax represents the maximum allowable occlusion ratio, Bmax represents the maximum allowable motion blur, and εallow represents the upper limit of allowable frame phase uncertainty. All of the above thresholds are determined through installation calibration or engineering configuration.

[0079] Image frames that do not meet any of the candidate validity criteria will not be included in the candidate capture image update, and their image quality sub-results can be saved only as diagnostic data. For image frames that meet the candidate validity criteria, the better image will be selected according to the comprehensive quality score Qf.

[0080] The overall quality score Qf is calculated using the following formula: Qf = αsSf + αeEf + αvVf + αb(1-Bf) + αo(1-Of) + αp(1-Df) + αu(1-Uf) αs + αe + αv + αb + αo + αp + αu = 1 All weights are greater than or equal to 0, therefore the value range of Qf is [0,1]. Each weight can be determined by installation calibration or engineering configuration; the comprehensive quality score Qf is only used to optimize among image frames that have met the candidate validity conditions, and cannot replace the candidate validity judgment of target visibility, occlusion degree, motion blur degree and frame phase uncertainty.

[0081] Df = min(|φf - φ*| / Wc, 1) Uf = min(εf / εallow, 1) In the formula, φf, φ*, Wc, εf and εallow are all in degrees (°), Wc>0, εallow>0; Df and Uf are dimensionless numbers.

[0082] One implementation of the exposure effectiveness index is: Ef=max(0,1-pdark-pbright), where pdark represents the proportion of pixels with gray values ​​below the underexposure threshold, and pbright represents the proportion of pixels with gray values ​​above the overexposure threshold.

[0083] One implementation of the occlusion index is: Of=min(Aocc / Abin,1), where Aocc is the area of ​​the trash can detection area that is occluded by the vehicle body or the container, and Abin is the area of ​​the complete trash can detection area. Both use the same pixel area unit.

[0084] The current candidate capture image and its overall quality score Qbest are saved. Before a candidate capture image is formed in this tilting process, the candidate status is set to invalid; the first image frame that simultaneously meets the candidate validity condition and Qf≥Qmin is saved as the current candidate capture image, and its overall quality score is set to Qbest.

[0085] Subsequently, when a new valid candidate image frame simultaneously satisfies Qf≥Qmin and Qf>Qbest, the current candidate capture image is replaced with the new valid candidate image frame, and Qbest is updated to Qf.

[0086] When the comprehensive quality scores of new valid candidate image frames are the same as those of the current candidate capture images, the image frames with smaller occlusion indices Of are retained first; if the occlusion indices are still the same, the image frames with smaller frame phase uncertainty εf are retained first; if they are still the same, the original candidate capture images are retained.

[0087] Both Qmin and Qbest are dimensionless numbers. Through the above update method, the candidate images ultimately retained in this dumping process are the image frames with the highest overall quality score among the image frames that meet the candidate validity criteria.

[0088] When the speed of the mechanical lifting lever changes significantly, the candidate capture phase area can be divided into multiple phase sub-intervals. Within each sub-interval, the image frame with the highest score is retained. When confirming the end, the image with the highest total score and complete target is selected from multiple image frames. This method can prevent a single short-term high-score error from occupying the entire candidate space.

[0089] For example, three consecutive image frames that meet the candidate validity criteria are obtained, with overall quality scores of 0.64, 0.88, and 0.73 respectively, and Qmin of 0.60. The first image frame is saved as the initial candidate capture image; the second image frame has a higher overall quality score than the first image frame, so it replaces the current candidate capture image; the third image frame has a lower overall quality score than the second image frame, so no replacement is performed.

[0090] Even if the third image frame is acquired later, the system still retains the second image frame with an overall quality score of 0.88 to avoid mistaking the latest image for the best image.

[0091] It is important to note that candidate images are only stored in a volatile buffer or temporary candidate area in this step and are not immediately used to generate a formal capture record. The selection time of the best image and the confirmation time of the complete tilting action are separate, but they are kept in correspondence through a unified timestamp and continuous motion phase.

[0092] S5. Within the end phase zone, confirm the trash can tipping action based on the complete motion path and action constraints, and backtrack and solidify the candidate captured image; Specifically, when a continuous motion phase enters the ending phase zone, the dumping action is not immediately confirmed. Instead, the sequence of phase zones, consistency of motion direction, action duration, end hold, maximum retraction amount, degree of deviation from the reference trajectory, and validity of candidate captured images are continuously checked. Only when all necessary conditions are met is a complete trash can dumping action confirmed.

[0093] The phase zone is recorded sequentially by the status flags. When the current tilting process is established, the record indicates that the phase zone has entered the initial phase zone. When the phase zone of continuous motion or frame intersects with the candidate capture phase zone, the record indicates that the phase zone has passed the candidate capture phase zone. Only after both of the aforementioned flags are valid can the phase zone be confirmed.

[0094] The motion direction consistency rate Rd is calculated using the following formula: Rd = Nsame / Neff In the formula, Nsame represents the number of samples in the effective phase increment whose sign is consistent with the tilting direction, and Neff represents the total number of effective phase increments whose absolute value is greater than the phase noise threshold. Rd is calculated only when Neff≥Nmin>0; when Neff<Nmin, the current action is marked as insufficient valid motion samples, and the dumping action is not confirmed.

[0095] The action duration Ta is the difference between the current confirmation moment and the action start moment; the time constraint is satisfied only when Ta is between the preset minimum action duration Tmin and the preset maximum action duration Tmax. The units of Ta, Tmin and Tmax are all seconds (s).

[0096] The end hold condition requires that the continuous motion phase continues to reach the end hold duration Te after entering the end phase region; if the continuous motion phase crosses the exit boundary of the end phase region during the hold period, the end hold count is reset to zero. The unit of Te is second (s). End hold can eliminate transient triggers that cross the end boundary only within one sampling period.

[0097] The maximum phase fallback amount Rmax is defined by the following formula: Rmax = max(1≤q≤K) [ max(1≤p≤q)(φp) - φq ] In the formula, K is the number of valid phase samples in the current dumping process, and the units of Rmax and φ are both degrees (°); it is considered that no significant mid-way fallback occurs in the dumping outbound path only when Rmax does not exceed the fallback threshold Rth; Rth is generated based on the fallback distribution of multi-cycle data of normal actions and the safety margin.

[0098] Optionally, the system also compares the current normalized phase trajectory with the reference trajectory envelope. The current action is divided into M sampling positions according to the normalized time progress: τm = (tm - ts) / (te - ts), 0≤τm≤1 Dtr = (1 / M) · Σ(m=1)^M |φ(τm) - φref(τm)| / L In the formula, ts and te are respectively the action start moment and the tentative end moment, with the unit of second (s); the units of φ(τm), φref(τm) and L are degrees (°); L is the full reference stroke of normal dumping, and L>0; Dtr is a dimensionless number. It is considered that the current trajectory is consistent with the normal dumping trajectory only when Dtr does not exceed the trajectory deviation threshold.

[0099] When the candidate snapshot phase region and the end phase region have overlapping boundaries due to the short stroke of the vehicle model, the image frame candidate evaluation and candidate image update are performed first in the same sampling period, and then the end hold count and complete action confirmation are performed, so as to avoid that the best image exactly located in the overlapping region is not included in the candidates because the action is confirmed first.

[0100] It should be noted that the system will determine that the tilting process is abnormally terminated under any of the following circumstances: continuous motion phase returns to the starting phase region; direction consistency rate is lower than the threshold; maximum backtracking amount exceeds the threshold; action duration exceeds Tmax; attitude data missing time exceeds the attitude missing threshold; unified timestamp is abnormal or continuous motion phase cannot continue to be generated; image data missing time exceeds the image missing threshold before a valid candidate capture image has been formed; calibration parameter version changes during the action; the formed candidate capture image is damaged, or the unified timestamp, frame phase, frame phase uncertainty, or image quality sub-item result corresponding to the candidate capture image is invalid.

[0101] Once valid candidate images have been generated, a brief loss of subsequent image frames is not a necessary condition for abnormal termination of the tilting process. In the event of abnormal termination, temporary candidate data is deleted, and no formal tilting capture record is generated.

[0102] When all confirmation conditions are met, the previously retained candidate images are written from the temporary candidate area to persistent storage. The acquisition time of the candidate image is retained as the original unified timestamp of that image frame, and the action confirmation time is recorded as the moment when the end holding condition is met. The two times represent the "actual acquisition time of the best image" and the "time when the complete tilting action is confirmed", respectively, thus enabling verification of the temporal relationship between the two.

[0103] For example, if a normal tilting process sequentially passes through the initial phase zone, the candidate capture phase zone, and the final phase zone, with a directional consistency rate of 0.92, an action duration of 4.8 seconds, a final hold duration of 0.30 seconds, a maximum phase back-off amount below the back-off threshold, and there are candidate capture images with scores reaching the valid threshold, then the tilting is confirmed as complete and the candidate images are fixed. If another action returns to the initial phase zone after the candidate capture phase zone, even if a clear candidate image has been obtained, a tilting capture record is not formed.

[0104] S6. After confirming the complete trash can emptying action, encapsulate the emptying event data; after confirming the complete trash can emptying action or after abnormal termination, execute a reset lock. In the case of abnormal termination, no formal emptying event data is generated, so as to control the activation of the next trash can emptying recognition. Specifically, after the complete dumping action is confirmed, a dumping event data packet is generated.

[0105] The event data packet includes at least the candidate captured image, the candidate image acquisition time, the action confirmation time, the action duration, the sensitive attitude axis identifier, the continuous motion phase trajectory summary, the direction consistency rate, the maximum backoff amount, the trajectory deviation, the frame phase and its uncertainty, the image quality sub-results, the calibration parameter version, the vehicle identifier, and the optional geographical location.

[0106] To ensure consistency among event data, digital digests can be calculated for the data content, phase trajectory summary, and calibration parameter version of candidate captured images, and the digest values ​​can be written into the event data packet. A digital digest is a fixed-length checksum obtained after performing deterministic operations on the input data, used to detect whether event data has been accidentally altered during storage or transmission. This invention does not limit the specific digest algorithm.

[0107] After the event data packet is generated, it is asynchronously written to non-volatile memory by the persistent task. Asynchronous writing means that the capture and recognition task only submits the confirmed complete event, without waiting for the image file and database records to be fully written, thereby avoiding the impact of storage speed fluctuations on the real-time processing of pose data and image frames. The persistent task should first write to a temporary file or temporary record, and then atomically commit after the image, metadata, and digital digest have all been successfully written.

[0108] When the network is available, event data packets can be uploaded to the management platform; when the network is unavailable, they are stored in a local queue awaiting upload. The queue is sorted by event confirmation time and records the number of uploads and the most recent upload result. Upload failures do not affect the validity of the local event or trigger duplicate dumping detection.

[0109] After the action is confirmed, the system enters the reset lockout. During the reset lockout, a new tilting process is not allowed, but attitude data continues to be collected to determine whether the mechanical lever has returned to the reset phase zone.

[0110] The reset lock is released only when the continuous motion phase returns to the reset phase region and remains there for the reset hold time Tr, or after the cooling time Tc and the mechanical lever is detected to be in its original position. The units for Tr and Tc are both seconds (s).

[0111] For actions that terminate abnormally, a reset lock is also executed, but no formal event data packet is generated. The abnormal termination log can only save diagnostic information, including the cause of termination, the last continuous motion phase, the maximum backoff amount, and data missing information, for maintenance personnel to check the IMU fixation status, mechanical lever jamming, or camera connection abnormalities.

[0112] To generate the parameters used in the above steps, installation calibration is performed after initial installation, IMU re-fixing, camera perspective adjustment, vehicle model change, or mechanical lifter repair. During the calibration phase, attitude data and image frames are simultaneously acquired for K normal tilting cycles, where K represents the number of normal tilting cycles, which can be 5-30, and preferably includes both no-load and loaded actions.

[0113] For the j-th candidate pose axis, the sensitivity axis score Scorej is calculated as follows: Scorej = [Aj / (Nj + εN)] · Mj · C(comp,j) · C(ind,j) In the formula, Aj is the phase travel of the axis during normal tilting, in degrees (°); Nj is the angular noise amplitude of the axis when the mechanical lever is stationary, in degrees (°); εN is the noise regularization amount set to prevent the denominator from being zero, in degrees (°); Mj is the monotonicity of the outgoing direction; C(comp,j) is the data integrity rate; C(ind,j) is the independence or discriminability of the axis relative to other attitude axes, all three taking values ​​in the range [0,1]; and Scorej is a dimensionless number. The attitude axis with the highest Scorej that exceeds the minimum score threshold is selected as the sensitive attitude axis. Preferably, the monotonicity Mj in the outbound direction can be calculated using the following formula: Mj = Nj,same / Nj,eff; In the formula, Nj,same represents the number of angle increments in the j-th candidate attitude axis whose sign is consistent with the main outgoing direction of the axis and whose absolute value is greater than the phase noise threshold, and Nj,eff represents the total number of effective angle increments with an absolute value greater than the phase noise threshold for the axis. When Nj,eff is 0, Mj is set to 0.

[0114] The data integrity rate C(comp,j) is the ratio of the number of valid attitude samples for the j-th candidate attitude axis to the number of required attitude samples calculated according to the nominal sampling frequency, and the calculation result is limited to 0-1.

[0115] Independence or identifiability C(ind,j) can be determined based on the maximum absolute correlation coefficient between the j-th candidate attitude axis and other candidate attitude axes: C(ind,j)=1-max(|ρj,l|), l≠j; In the formula, ρj,l represents the correlation coefficient between the j-th candidate attitude axis and the l-th candidate attitude axis during the calibration tilt process. When only one candidate attitude axis is available, C(ind,j) is set to 1.

[0116] For each calibration cycle, the sensitive attitude axis angles are made continuous and oriented normalized. The starting change point, candidate image quality peaks, and ending plateaus are detected. The starting change point can be determined based on the phase velocity continuously exceeding a start threshold combined with angular velocity changes. The ending plateau can be determined based on the phase velocity remaining below a stop threshold. By aligning multiple cycles according to the starting change point and ending plateau, the median, lower quantile, and upper quantile values ​​of the phase trajectory are statistically analyzed on a unified phase grid to form a reference trajectory envelope.

[0117] The percentile value refers to the value that is located at a specified proportion after multiple samples are arranged numerically. For example, the 5th percentile means that about 5% of the samples are not greater than this value, and the 95th percentile means that about 95% of the samples are not greater than this value.

[0118] The median or lower quantile of the overall image quality score is statistically analyzed on a unified phase grid, and the position with the highest statistical score among the phase positions where the target visibility reaches a threshold is selected as the preferred capture phase φ*. By using multiple cycles of median or lower quantile values, instead of a single highest score, parameter shifts caused by occasional sharp frames or occasional no occlusion can be reduced.

[0119] Based on the distribution quantiles of the starting change point, the preferred capture phase, and the ending platform in multiple loops, and with the addition of phase safety margin, the starting phase area, candidate capture phase area, and ending phase area are generated; Tmin and Tmax are generated based on the normal action duration distribution; and the phase noise threshold, back-off threshold, and hold duration are generated based on the static noise, normal back-off amount, and phase velocity distribution.

[0120] The calibration confidence level Ccal can be calculated using the following formula: Ccal=β1Caxis+β2Crep+β3Cdir+β4Cimg+β5Ctime+β6Csync; In the formula, β1 to β6 are non-negative weights, and the sum of the weights is 1. Each term is normalized to 0 to 1 and can be calculated as follows: Caxis=max{0, min[1, (Scoremax-Scoresecond) / (Scoremax+εS)]}; Where Scoremax is the score of the highest sensitivity axis, Scoresecond is the score of the second highest sensitivity axis, and εS is a positive number set to prevent the denominator from being zero; Crep=max[0,1-Dcal / Drep]; Where Dcal is the average trajectory deviation relative to the reference trajectory after multiple calibration cycles, and Drep is the upper limit of the allowable calibration trajectory deviation; Cdir is the average of the direction consistency rate Rd of each effective calibration cycle; Cimg is the ratio of the number of calibration loops to obtain valid candidate capture images to the total number of valid calibration loops; Ctime=max[0,1-σT / Tavg]; Where σT is the standard deviation of the duration of multiple effective calibration cycles, Tavg is the average duration of the cycle, and Tavg is greater than 0; Csync=max[0,1-emap / esync]; Where emap is the maximum residual of time mapping during the calibration period, and esync is the upper limit of the allowed residual of time mapping; both have the same unit.

[0121] Drep, εS, and esync are all pre-defined calibration parameters. Through the above calculations, each component of the calibration confidence level has a clear data source and calculation method.

[0122] When Ccal falls below the calibrated threshold, or any key component falls below the minimum component threshold, the system refuses to enable automatic image capture and provides prompts to re-fix the IMU, adjust the camera angle, check time synchronization, or supplement with normal tilting samples. This rejection mechanism prevents the system from entering operation even when parameters are clearly unreliable.

[0123] Example 2: Numerical Calculation Example This embodiment provides a set of calculable data for frame phase, frame phase uncertainty, and overall image quality score to illustrate the specific usage of each formula. The following values ​​are for illustrative purposes only and do not constitute a limitation on the scope of protection.

[0124] Let the uniform times of two adjacent IMU samples be ti = 1.000 s and t(i+1) = 1.020 s, respectively, and the corresponding continuous motion phases be φi = 63.0° and φ(i+1) = 64.2°.

[0125] If the exposure of a certain image frame starts at 1.007 s and ends at 1.011 s, then tf = (1.007 + 1.011) / 2 = 1.009 s.

[0126] φf = 63.0° + [(1.009-1.000) / (1.020-1.000)]×(64.2°-63.0°) = 63.54° Assume vf=60° / s, em=0.001 s, ec=0.002 s, texp=0.004 s, a(φ,max)=300° / s 2 , Δti=0.020 s, eφ=0.15°, then: εf = 60×(0.001+0.002+0.004 / 2) + (300 / 8)×0.020 2 + 0.15 = 0.465°; Therefore, the actual phase interval corresponding to this image frame is [63.075°, 64.005°]. If the candidate capture phase interval is [55°, 75°], then this image frame enters the candidate evaluation.

[0127] Furthermore, setting Sf=0.82, Ef=0.90, Vf=0.88, Bf=0.12, and Of=0.08, the preferred capture phase φ*=65°, Wc=10°, and the upper limit of the allowable uncertainty εallow=2°, Df=0.146Uf=0.2325. Taking weights αs=0.20, αe=0.15, αv=0.25, αb=0.10, αo=0.10, αp=0.10, αu=0.10, we get: Qf = 0.20×0.82 + 0.15×0.90 + 0.25×0.88 + 0.10×0.88 + 0.10×0.92 +0.10×0.854 + 0.10×0.7675 ≈ 0.861; If Qmin = 0.70 and the current Qbest = 0.80, since Qf = 0.861 > Qbest, the system replaces the current candidate capture image with this image frame and updates Qbest to 0.861.

[0128] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and non-volatile computer storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0129] Without altering the technological chain of this invention, the attitude data of the mechanical lever can also be obtained from an absolute angle sensor, an encoder, or the fusion of multiple attitude sensors; image quality evaluation can use lightweight image metrics, or it can use target detection or key point detection results; event data can be stored locally or uploaded to a remote platform. All of the above alternative methods should maintain the interrelationship between "continuous motion phase generation, image frame phase mapping, candidate phase region selection, complete action confirmation of the end phase region, backtracking and solidification after confirmation, and reset locking."

[0130] The above embodiments provide a complete description of the technical solution of the present invention. For those skilled in the art, any equivalent adjustments made to the threshold, weight, sampling frequency, phase region boundary, and data storage format without departing from the principles of the present invention should fall within the protection scope of the present invention.

Claims

1. A method for precise image capture triggering of trash cans based on mechanical lever motion feature recognition, characterized in that, Includes the following steps: S1. Collect mechanical lever posture data and continuous image frames, and unify the posture data and image frames to the same time reference; S2. Perform continuous and direction normalization processing on the posture angle sequence to generate continuous motion phases of the mechanical lever and determine the frame phase of each image frame; S3. Determine the operation enable based on the vehicle status and mechanical lever status, and establish the current dumping process after meeting the pre-enablement and initial motion conditions; S4. Evaluate the image frame quality within the candidate capture phase area, and continuously update and save the candidate capture images based on the frame phase reliability; S5. Confirm the trash can dumping action based on the complete motion path and action constraints within the end phase area, and backtrack and solidify the candidate capture images; S6. Encapsulate the dumping event data after confirming the complete trash can dumping action; execute a reset lock after confirming the complete trash can dumping action or after abnormal termination. In the case of abnormal termination, no formal dumping event data is generated to control the activation of the next trash can dumping recognition.

2. The method for precise capture and triggering of trash can images based on mechanical lever motion feature recognition as described in claim 1, characterized in that, In step S2, during the installation and calibration phase, the sensitive attitude axis with the highest correlation to the tilting motion of the mechanical lever is selected from multiple attitude axes of the inertial measurement unit (IMU). During the operation phase, the k-th original attitude angle θk is read from the sensitive attitude axis, and the original angle difference Δθk = θk - θ(k-1) between adjacent samples is calculated. When Δθk>180°, Δθ′k=Δθk-360°; When Δθk<-180°, Δθ′k=Δθk+360°; When -180°<Δθk<180°, Δθ′k=Δθk; When |Δθk|=180° or the actual angular displacement of adjacent samples may reach 180°, the corresponding sample is marked as having an uncertain angular direction, and the correction result is determined by combining the gyroscope angular velocity sign or the motion direction of multiple samples before and after. The corrected angle difference is accumulated to obtain the continuous angle uk=u(k-1)+Δθ′k, and the direction is normalized according to φk=d·(uk-uref), where uref represents the continuous angle reference value when the mechanical lifting rod is in the reset position, and d takes +1 or -1, so that φk increases in total during the normal tilting process.

3. The method for precise capture and triggering of trash can images based on mechanical lever motion feature recognition as described in claim 1, characterized in that, In step S1, the vehicle processor uses a monotonic clock as a unified time reference; when the inertial measurement unit (IMU) and the vehicle camera are directly triggered by the vehicle processor, the monotonic clock time is recorded at the trigger moment. When the inertial measurement unit (IMU) and the vehicle-mounted camera have their own clocks, the corresponding points between the device time and the monotonic clock time are periodically acquired, and a linear time mapping or a piecewise linear time mapping is established based on the most recent multiple corresponding points to convert the device time into the monotonic clock time; when the vehicle-mounted camera can output the exposure start time and exposure end time, the image frame representative time tf is determined according to the midpoint of the exposure time interval. When the vehicle-mounted camera can only output the image acquisition completion time, encoding completion time, or the time tc when the vehicle-mounted processor receives the image frame, the representative time of the image frame is determined according to tf = tc - δc, where δc is the fixed delay of the camera determined during the installation and calibration stage; assuming ti ≤ tf ≤ t(i+1), the frame phase φf is calculated according to φf = φi + [(tf - ti) / (t(i+1) - ti)]·(φ(i+1) - φi), and interpolation is performed only when t(i+1) > ti and t(i+1) - ti does not exceed the maximum interpolation interval Tinterp.

4. The method for precise capture and triggering of trash can images based on mechanical lever motion feature recognition as described in claim 3, characterized in that, The frame phase uncertainty εf is calculated as: εf=|vf|·(em+ec+texp / 2)+(a(φ,max) / 8)·(Δti) 2 +eφ is calculated, where vf is the continuous motion phase velocity near time tf of the image frame, em is the upper bound of the IMU time mapping error, ec is the upper bound of the camera time mapping error, texp is the exposure time, a(φ,max) is the upper bound of the absolute value of the phase angular acceleration within the interpolation time period, Δti is the time interval between adjacent attitude samples, and eφ is the upper bound of the error after the attitude measurement and installation errors are converted to the continuous motion phase; the actual phase of the image frame is represented as [φf-εf,φf+εf]; When the actual phase does not intersect with the candidate capture phase region, the image frame is not included in the candidate evaluation; when the actual phase intersects with the candidate capture phase region but εf is greater than the upper limit of allowable uncertainty εallow, the image frame is marked as frame phase unreliable and is not used as a candidate capture image.

5. The method for precise capture and triggering of trash can images based on mechanical lever motion feature recognition as described in claim 1, characterized in that, In step S3, the root mean square ωr of the gravity residual rg, acceleration dispersion σa, and triaxial angular velocity composite value is calculated within the sliding time window, and the job enable confidence Ce is calculated according to Ce = wgFg + waFa + wωFω + wvFv + wsFs, where wg + wa + wω + wv + ws = 1, and all weights are non-negative numbers. Fg = max(0, 1 - rg / Rg), Fa = max(0, 1 - σa / Ag), Fω = max(0, 1 - ωr / Ωg), where Rg, Ag, and Ωg are the corresponding normalization thresholds; the vehicle speed confidence value Fv is determined based on the vehicle speed |vveh| and preset speed thresholds v0 and v1: when |vveh|≤v0, Fv = 1; when v0 < |vveh| < v1, Fv = (v1 - |vveh|) / (v1 - v0); when |vveh|≥v1, Fv = 0; the discrete state confidence value Fs is determined based on one or more of the following: parking brake, gear position, mechanical lever in-situ state, and hydraulic operation enabled state; when any data is unavailable, the weight corresponding to that item is set to 0, and the remaining weights are renormalized; The system sets an independent operation gate state Gop. When any of the hydraulic operation enable state, power take-off state, or mechanical lever control command is valid and clearly indicates that the tilting operation is allowed, Gop = 1. Alternatively, when the aforementioned operation enable signal cannot be obtained, but the vehicle speed is within the preset low speed range or the vehicle is in a parked state, and the mechanical lever in place state is valid, Gop = 1. When Gop = 1, Ce reaches the pre-enabling threshold, and the continuous motion phase φk is in the initial phase region and continues to meet the pre-enabling holding time, the pre-enabling state is entered, and the continuous motion phase when entering the pre-enabling state is recorded as the initial continuous motion phase φ0. In the pre-enabling state, when φk - φ0 ≥ Δφstart, the continuous motion phase velocity v(φ,k) ≥ vstart, and the motion holding time Tstart is continuously met, the start time of the action and the current calibration parameter version are recorded, and the current tilting process is established.

6. The method for precise capture and triggering of trash can images based on mechanical lever motion feature recognition as described in claim 1, characterized in that, In step S4, an image frame is determined as a valid candidate image frame when it simultaneously satisfies the following conditions: target visibility index Vf ≥ target visibility minimum threshold Vmin, occlusion index Of ≤ maximum allowable occlusion ratio Omax, motion blur index Bf ≤ maximum allowable motion blur degree Bmax, and frame phase uncertainty εf ≤ allowable uncertainty upper limit εallow. For image frames that meet the candidate validity conditions, the comprehensive quality score Qf is calculated according to: Qf=αsSf+αeEf+αvVf+αb(1-Bf)+αo(1-Of)+αp(1-Df)+αu(1-Uf); where αs+αe+αv+αb+αo+αp+αu=1, Df=min(|φf-φ*| / Wc,1), Uf=min(εf / εallow,1), Sf is the sharpness index, Ef is the exposure effectiveness index, φ* is the preferred capture phase, and Wc>0.

7. The method for precise capture and triggering of trash can images based on mechanical lever motion feature recognition as described in claim 6, characterized in that, Before candidate images are generated during this dumping process, the candidate status is set to invalid. The first image frame that simultaneously meets the candidate validity condition and Qf≥Qmin is saved as the current candidate image and its comprehensive quality score is set to Qbest. Subsequently, when a new valid candidate image frame simultaneously satisfies Qf≥Qmin and Qf>Qbest, the current candidate capture image is replaced by the new valid candidate image frame, and Qbest is updated to Qf. When the comprehensive quality scores of the new valid candidate image frame and the current candidate capture image are the same, the image frame with the smaller occlusion index Of is retained first. When the occlusion indices are the same, the image frame with the smaller frame phase uncertainty εf is retained first. When the frame phase uncertainty is still the same, the current candidate capture image is retained.

8. The method for precise capture and triggering of trash can images based on mechanical lever motion feature recognition as described in claim 1, characterized in that, In step S5, when the current tilting process is established, it is recorded that the starting phase area has been entered. When the continuous motion phase or frame phase interval intersects with the candidate capture phase area, it is recorded that the candidate capture phase area has been passed. Only after all the aforementioned records are valid can the entry into the ending phase area be qualified for confirmation. The motion direction consistency rate Rd is calculated according to Rd = Nsame / Neff, where Nsame represents the number of samples in the effective phase increment whose signs are consistent with the tilting direction, and Neff represents the total number of effective phase increments whose absolute value is greater than the phase noise threshold. Rd is calculated only when Neff ≥ Nmin > 0, and Rd is not lower than the direction consistency rate threshold. The motion duration Ta is between the preset shortest motion duration Tmin and the preset longest motion duration Tmax. After the continuous motion phase enters the ending phase region, it continues to maintain the ending duration Te; the maximum phase backoff amount Rmax is calculated according to Rmax=max(1≤q≤K)[max(1≤p≤q)(φp)-φq] and does not exceed the backoff threshold Rth, where K is the number of effective phase samples in this tilting process, and φp and φq are the continuous motion phases corresponding to the p-th and q-th effective phase samples, respectively; this action is divided into M sampling positions according to the normalized time progress, τm=(tm-t s) / (te-ts), the trajectory deviation Dtr is calculated according to Dtr=(1 / M)·Σ(m=1)^M|φ(τm)-φref(τm)| / L and does not exceed the trajectory deviation threshold, where M is the number of sampling positions in the normalized time progress, ts and te are the start time and provisional end time of the action respectively, τm is the normalized time progress of the m-th sampling position, φref(τm) is the continuous motion phase of the reference trajectory at τm, and L is the normal tilting reference full stroke and L>0; Before confirming the complete trash can emptying action, the image content, unified timestamp, frame phase, frame phase uncertainty, and image quality sub-items of the candidate captured image are all complete and valid; when the aforementioned confirmation qualifications, motion direction consistency rate, action duration, end hold duration, maximum phase backoff amount, trajectory deviation, and candidate captured image validity are all satisfied, the complete trash can emptying action is confirmed.

9. The method for precise capture and triggering of trash can images based on mechanical lever motion feature recognition as described in claim 1, characterized in that, The current tilting process is terminated when the continuous missing time of attitude data samples exceeds the attitude missing threshold, the unified timestamp becomes non-incremental, or the attitude data cannot be used to generate continuous motion phases. When no valid candidate image has been formed and the continuous missing time of image frames exceeds the image missing threshold, the current tilting process is terminated; when a valid candidate image has been formed and subsequent image frames are temporarily missing, but the image content, unified timestamp, frame phase, frame phase uncertainty and image quality sub-item results of the candidate image remain complete and valid, the complete action confirmation is continued based on the attitude data. After the complete dumping action is confirmed, a dumping event data packet is generated. The persistence task first writes to a temporary file or temporary record. After the candidate captured image, metadata and digital digest are all successfully written, the atomic commit is performed. The system enters reset lockout after a complete tilting action is confirmed or the current tilting process is abnormally terminated. When abnormally terminated, no formal tilting event data packet is generated, but abnormal diagnostic information including the reason for termination, the last continuous motion phase, the maximum phase back-off amount, and data missing information is saved. The reset lockout is released only when the continuous motion phase returns to the reset phase area and continues to reach the reset hold time Tr, or after the cooling time Tc and the mechanical lifting lever is detected to be in its original position.

Images