A Passenger Transport Inspection Method Based on Object Detection

Abstract

This invention belongs to the field of action recognition and monitoring technology, and relates to a passenger transport operation inspection method based on target detection. The method includes: acquiring a continuous video frame sequence of the interior environment of an operating vehicle; extracting a two-dimensional bounding box of the target human body, and loading a static threshold mask array coinciding with the physical edge of the vehicle door in the corresponding image region; calculating a morphological exposure rate variable based on the distribution of moving pixels within the two-dimensional bounding box in the mask array, generating a temporal evolution sequence and calculating a first-order difference sequence; determining a positive crossing state based on the first-order difference sequence, and then concatenating them to generate a spatiotemporal feature tensor; generating action confidence by matching with pre-stored feature fingerprints, and outputting a warning signal in conjunction with vehicle operating parameters. This invention reveals the nonlinear evolution law of a human body entering the vehicle by introducing a morphological exposure rate variable, eliminating false alarms caused by monocular visual trajectory isomorphism, improving the accuracy of action recognition, and solidifying the inspection evidence chain.

Description

Technical Field

[0001] This invention belongs to the field of motion recognition and monitoring technology, and in particular relates to a passenger transport operation inspection method based on target detection. Background Technology

[0002] Currently, passenger vehicles are typically equipped with video surveillance equipment, using target detection algorithms combined with tracking logic to analyze pedestrian movement trajectories. Boarding behavior is identified by determining the degree of overlap between the pedestrian and the vehicle door. The projection mapping process from a 3D scene to a 2D plane inevitably results in depth information collapse, causing pixel displacement in the image coordinate system to only represent the projection features of the target on the imaging plane. This imaging principle creates significant detection ambiguity in actual operational inspection scenarios. When a target generates a displacement vector on the imaging plane, its absolute depth change in physical space is not visible. The projection process of a pedestrian walking parallel to the vehicle body and passing through the door's field of view, and the projection process of a passenger approaching and crossing the threshold to enter the vehicle, exhibit highly isomorphic characteristics in terms of 2D coordinate changes and bounding box scaling. Existing processing methods rely on the geometric shape of the trajectory line to determine the action. Since it is impossible to reconstruct the lost physical rigid boundary constraints within a single imaging source, it is difficult to distinguish, from the principle level, random passing behavior in the environmental background from the actual boarding process.

[0003] Control algorithms that analyze the evolution of complex behaviors have limitations. For example, Chinese invention patent application CN117830970A discloses a method and intelligent device for detecting illegal passenger pick-up and drop-off on shuttle buses based on GPS data. This technology relies on the comparison of passenger counts before and after a vehicle triggered by a geofence. In environments with dense crowds causing physical obstruction or severe fluctuations in light and shadow, the accumulated error in counting can lead to false alarms or missed alarms. This technology bypasses the analysis of the depth features of transient actions through overall statistical methods. It cannot decouple the ambiguity of the mapping between two-dimensional trajectory and depth change under monocular vision, and it cannot build an evidence chain based on the physical interaction process of the human body in non-station passenger pick-up scenarios.

[0004] Therefore, how to utilize the physical rigid boundary of the passenger vehicle door as a spatial reference system in the image processing domain, decouple the depth ambiguity of the two-dimensional trajectory and construct an evidence chain of action evolution features in a monocular vision environment, has become the technical problem to be solved by this invention. Summary of the Invention

[0005] The present invention aims to solve the problems of depth ambiguity and high false alarm rate of inspection and early warning in complex working conditions when using two-dimensional trajectory features to determine boarding actions.

[0006] In this technical solution, a passenger transport operation inspection method based on target detection includes the following steps:

[0007] Step 101: Obtain a continuous video frame sequence representing the internal environment of the operating vehicle;

[0008] Step 102: Extract the target detection results of each video frame in the continuous video frame sequence. The target detection results include the two-dimensional bounding box corresponding to the target human body.

[0009] Step 103: Load a static threshold mask array within the image region where the two-dimensional bounding box is located. The static threshold mask array coincides with the physical edge image of the vehicle door in the image coordinate system.

[0010] Step 104: Identify pixels with motion attributes within the two-dimensional bounding box to construct a feature pixel set. Based on the distribution of the feature pixel set within the static threshold mask array, calculate the morphological exposure rate variable corresponding to each video frame and generate a temporal evolution sequence characterizing the physical boundary truncation of the target human body morphology.

[0011] Step 105: Calculate the first-order difference sequence of the time-series evolution sequence;

[0012] Step 106: If the first-order difference sequence maintains positive difference within a preset number of consecutive frames and the cumulative difference value is greater than a preset action trigger threshold, the target human body is marked as a positive crossing state.

[0013] Step 107: The determination result representing the positive crossing state is concatenated with the displacement vector of the two-dimensional bounding box in the time domain to generate a spatiotemporal feature tensor representing the evolution process of the target human boarding action.

[0014] Step 108: Calculate the feature similarity distance between the spatiotemporal feature tensor and each pre-stored action fingerprint in the action fingerprint database, and output an early warning signal based on the action confidence generated by the feature similarity distance and the real-time operating parameters of the operating vehicle.

[0015] Preferably, step 107 involves concatenating the determination result representing the positive crossing state with the displacement vector of the two-dimensional bounding box in the time domain to generate a spatiotemporal feature tensor representing the evolution of the target human's boarding action. This includes: step 1071, taking the coordinate evolution trajectory of the geometric center of the two-dimensional bounding box in the image coordinate system; step 1072, calculating the motion direction component and velocity change envelope of the coordinate evolution trajectory within the corresponding time period of the temporal evolution sequence to construct the displacement vector; and step 1073, using a one-dimensional convolutional layer to perform dimensional mapping between the binary identifier representing the positive crossing state and the motion direction component and velocity change envelope to generate the spatiotemporal feature tensor.

[0016] Preferably, before loading the static threshold mask array within the image region where the two-dimensional bounding box is located in step 103, the method further includes: step 1031, taking the reference boundary of the static threshold mask array and the pixel gradient magnitude within its extended pixel neighborhood; step 1032, using the pre-stored reference mask boundary as a template, determining the two-dimensional affine translation vector corresponding to the extreme value of gradient magnitude overlap in the extended pixel neighborhood using normalized cross-correlation matching operation; and step 1033, performing coordinate axis translation calibration on the static threshold mask array based on the two-dimensional affine translation vector.

[0017] Preferably, the morphology exposure rate variable is calculated through the following steps: Step 1041, count the number of pixels in the feature pixel set that fall inside the static threshold mask array, and determine it as the first feature statistic; Step 1042, count the total number of pixels in the feature pixel set, and determine it as the second feature statistic; Step 1043, the ratio of the first feature statistic to the second feature statistic is determined as the morphology exposure rate variable.

[0018] Preferably, step 104, calculating the morphological exposure rate variable corresponding to each video frame based on the distribution of the feature pixel set's landing points within the static threshold mask array, further includes: step 1044, taking the pixel-level optical flow vector of each pixel in the feature pixel set between adjacent video frames; step 1045, using the directional consistency constraint of the pixel-level optical flow vector to cluster and segment the feature pixel set, separating the effective pixel subset that conforms to the feature of crossing into the vehicle from the overlapping human pixel clusters; step 1046, mapping the effective pixel subset to the static threshold mask array, and calculating the morphological exposure rate variable corresponding to the effective pixel subset.

[0019] Preferably, before step 101, acquiring the continuous video frame sequence representing the internal environment of the operating vehicle, the method further includes: step 1011, measuring the physical state signal of the vehicle door; step 1012, when the physical state signal of the door represents that the door is in the open state, triggering the acquisition of the continuous video frame sequence.

[0020] Preferably, step 108, calculating the feature similarity distance between the spatiotemporal feature tensor and each pre-stored action fingerprint in the action fingerprint database, and outputting a warning signal based on the action confidence generated by the feature similarity distance and the real-time operating parameters of the operating vehicle, includes: step 1081, obtaining the real-time operating parameters of the operating vehicle, including real-time geographic location parameters, operating speed signal, and preset geofence reference coordinates; step 1082, calculating the spatial distance offset between the real-time geographic location parameters and the preset geofence reference coordinates; step 1083, outputting a warning signal when the action confidence is greater than a preset probability threshold, the spatial distance offset is greater than 50m, and the operating speed signal is less than 5km / h.

[0021] Preferably, step 102, taking the target detection results of each video frame in the continuous video frame sequence, includes: step 1021, using a convolutional neural network model to perform feature mapping on each video frame to generate a multi-channel feature map; step 1022, performing target candidate region regression on the multi-channel feature map to output the vertex coordinates of the two-dimensional bounding box.

[0022] Preferably, after issuing the warning signal in step 1083, the method further includes: step 1084, which associates and stores the feature evidence chain characterizing the evolution of the target human's boarding action with the warning signal.

[0023] Compared with existing technologies, the passenger transport operation inspection method based on target detection of this invention has the following beneficial effects:

[0024] Firstly, in the passenger transport operation inspection method for target detection, by transforming the physical gate structure of passenger vehicles into a static threshold mask array within the computational domain, and combining the spatial intersection and union relationship between the target feature pixel set and the static threshold mask array to determine the morphological exposure rate variable, this invention changes the logic of traditional technology that relies solely on the displacement and area changes of two-dimensional bounding boxes to determine actions. Since the temporal evolution sequence of the morphological exposure rate variable can objectively reflect the nonlinear evolution process of human pixels being truncated and exposed by physical boundaries, this mechanism fundamentally eliminates the trajectory isomorphism problem caused by the lack of depth information in monocular vision, effectively eliminates false alarms caused by interference behaviors such as pedestrians passing by or lingering outside the vehicle, and improves the evidentiary value of action recognition results in the operation inspection scenario.

[0025] Secondly, in response to the mechanical vibrations generated during vehicle idling and door opening and closing, this invention extracts the pixel gradient amplitude of the preset area of ​​the mask and performs template matching to determine the two-dimensional affine translation vector to achieve coordinate calibration of the static threshold mask array. This mechanism uses the inherent physical edge features of the door as visual anchor points to transform the mechanical disturbances of the physical world into dynamic compensation of the coordinate system in the image processing domain, ensuring that the extraction accuracy of the shape exposure rate variable does not produce a systematic shift under uncontrolled harsh vehicle conditions, thereby enhancing the operational stability of the entire action recognition architecture.

[0026] Third, when dealing with extreme conditions of multiple overlapping passengers during peak hours, this invention forcibly shrinks the feature extraction domain to the coverage area of ​​a static threshold mask array and combines it with the directional constraints of pixel-level optical flow vectors to extract effective pixels that meet the inward crossing feature from the entangled two-dimensional pixel clusters. This decoupling method, which combines spatial mask constraints with kinematic vector filtering, avoids the interruption of action recognition caused by bounding box failure when the overlap rate is high in traditional target detection, and ensures the system's continuous monitoring capability and data output determinism in densely populated environments. Attached Figure Description

[0027] Figure 1This invention relates to a passenger operation audit flowchart of the morphological exposure rate evolution characteristics;

[0028] Figure 2 This invention relates to a diagram of the phenotypic tensor construction and behavioral confidence early warning architecture. Detailed Implementation

[0029] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0030] It should be noted that all directional and positional terms used in this invention, such as: up, down, left, right, front, back, vertical, horizontal, inner, outer, top, bottom, transverse, longitudinal, center, etc., are only used to explain the relative positional relationship and connection between components in a specific state (as shown in the accompanying drawings). They are only for the convenience of describing this invention and do not require that this invention be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention. In addition, the descriptions of "first," "second," etc., in this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated.

[0031] In the description of this invention, unless otherwise explicitly specified and limited, the terms installation, connection, and linking should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections; they can refer to direct connections or indirect connections through an intermediate medium; they can refer to the internal connection of two components. For those skilled in the art, the specific meaning of the above terms in this invention can be understood according to the specific circumstances.

[0032] In the description of this specification, references to the terms "an embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example, and the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0033] Example 1: This example relates to a passenger transport operation inspection method based on target detection, which includes the following steps:

[0034] Step 101: Obtain a continuous video frame sequence representing the internal environment of the operating vehicle;

[0035] Step 102: Extract the target detection results of each video frame in the continuous video frame sequence. The target detection results include the two-dimensional bounding box corresponding to the target human body.

[0036] Step 103: Load a static threshold mask array within the image region where the two-dimensional bounding box is located. The static threshold mask array coincides with the physical edge image of the vehicle door in the image coordinate system.

[0037] Step 104: Identify pixels with motion attributes within the two-dimensional bounding box to construct a feature pixel set. Based on the distribution of the feature pixel set within the static threshold mask array, calculate the morphological exposure rate variable corresponding to each video frame and generate a temporal evolution sequence characterizing the physical boundary truncation of the target human body morphology.

[0038] Step 105: Calculate the first-order difference sequence of the time-series evolution sequence;

[0039] Step 106: If the first-order difference sequence maintains positive difference within a preset number of consecutive frames and the cumulative difference value is greater than a preset action trigger threshold, the target human body is marked as a positive crossing state.

[0040] Step 107: The determination result representing the positive crossing state is concatenated with the displacement vector of the two-dimensional bounding box in the time domain to generate a spatiotemporal feature tensor representing the evolution process of the target human boarding action.

[0041] Step 108: Calculate the feature similarity distance between the spatiotemporal feature tensor and each pre-stored action fingerprint in the action fingerprint database, and output an early warning signal based on the action confidence generated by the feature similarity distance and the real-time operating parameters of the operating vehicle.

[0042] Step 107 in this embodiment, which involves concatenating the determination result of the positive crossing state with the displacement vector of the two-dimensional bounding box in the time domain to generate a spatiotemporal feature tensor representing the evolution of the target human's boarding action, includes: Step 1071, taking the coordinate evolution trajectory of the geometric center of the two-dimensional bounding box in the image coordinate system; Step 1072, calculating the motion direction component and velocity change envelope of the coordinate evolution trajectory within the corresponding time period of the temporal evolution sequence to construct the displacement vector; Step 1073, using a one-dimensional convolutional layer to perform dimensional mapping between the binary identifier representing the positive crossing state and the motion direction component and velocity change envelope to generate the spatiotemporal feature tensor.

[0043] In this embodiment, before loading the static threshold mask array within the image region where the two-dimensional bounding box is located, step 103 further includes: step 1031, taking the reference boundary of the static threshold mask array and the pixel gradient magnitude within its extended pixel neighborhood; step 1032, using the pre-stored reference mask boundary as a template, determining the two-dimensional affine translation vector corresponding to the extreme value of gradient magnitude overlap within the extended pixel neighborhood using normalized cross-correlation matching operation; and step 1033, performing coordinate axis translation calibration on the static threshold mask array based on the two-dimensional affine translation vector.

[0044] The morphology exposure rate variable described in this embodiment is calculated through the following steps: Step 1041, the number of pixels in the feature pixel set that fall inside the static threshold mask array is determined as the first feature statistical value; Step 1042, the total number of pixels in the feature pixel set is determined as the second feature statistical value; Step 1043, the ratio of the first feature statistical value to the second feature statistical value is determined as the morphology exposure rate variable.

[0045] Step 104 of this embodiment, which calculates the morphological exposure rate variable corresponding to each video frame based on the distribution of the feature pixel set's landing points within the static threshold mask array, further includes: Step 1044, extracting the pixel-level optical flow vector of each pixel in the feature pixel set between adjacent video frames; Step 1045, using the directional consistency constraint of the pixel-level optical flow vector to cluster and segment the feature pixel set, separating the effective pixel subset that conforms to the feature of crossing into the vehicle from the overlapping human pixel clusters; Step 1046, mapping the effective pixel subset to the static threshold mask array, and calculating the morphological exposure rate variable corresponding to the effective pixel subset.

[0046] Before acquiring the continuous video frame sequence characterizing the internal environment of the operating vehicle in step 101 of this embodiment, the method further includes: step 1011, monitoring the physical state signal of the vehicle door; and step 1012, triggering the acquisition of the continuous video frame sequence when the physical state signal of the door indicates that the door is in an open state.

[0047] Step 108 in this embodiment calculates the feature similarity distance between the spatiotemporal feature tensor and each pre-stored action fingerprint in the action fingerprint database, and outputs a warning signal based on the action confidence generated by the feature similarity distance and the real-time operating parameters of the operating vehicle. This includes: Step 1081, obtaining the real-time operating parameters of the operating vehicle, which include real-time geographic location parameters, operating speed signals, and preset geofence reference coordinates; Step 1082, calculating the spatial distance offset between the real-time geographic location parameters and the preset geofence reference coordinates; Step 1083, outputting a warning signal when the action confidence is greater than a preset probability threshold, the spatial distance offset is greater than 50m, and the operating speed signal is less than 5km / h.

[0048] Step 102 in this embodiment, which extracts the target detection results of each video frame in a continuous video frame sequence, includes: Step 1021, using a convolutional neural network model to perform feature mapping on each video frame to generate a multi-channel feature map; Step 1022, performing target candidate region regression based on the multi-channel feature map to output the vertex coordinates of the two-dimensional bounding box.

[0049] After outputting the warning signal, step 1083 of this embodiment further includes: step 1084, which associates and stores the feature evidence chain characterizing the evolution of the target human's boarding action with the warning signal.

[0050] Example 2: In the scenario of vehicle inspection at a passenger transport hub in Hangzhou, when a vehicle stops and opens its doors for passengers to board and alight, the system suffers from depth information collapse due to the monocular vision imaging principle. Pedestrians around the station walk parallel to the vehicle and pass through the door's field of view, or stop outside the door, approach, and then turn away. The 2D pixel displacement vectors and bounding box area expansion features of these interfering targets are isomorphic to the boarding action, causing the judgment logic based on trajectory geometry or single bounding box scale changes to misjudge and make it difficult to distinguish between passing behavior and the boarding process.

[0051] The processor acquires the target detection results of each video frame in a continuous video frame sequence of the interior environment of the operating vehicle, determines the 2D bounding box corresponding to the target human body, and retrieves a pre-calibrated static threshold mask array from the storage unit. This static threshold mask array is a binary data matrix determined based on the geometric projection of the physical border of the passenger vehicle door in the image coordinate system. It coincides with the physical edge image of the door in the image coordinate system. The processor identifies pixels with motion attributes within the 2D bounding box, constructs a feature pixel set, counts the number of pixels in the feature pixel set that fall inside the static threshold mask array, determines the first feature statistical value, counts the total number of pixels in the feature pixel set, determines the second feature statistical value, and determines the ratio of the first feature statistical value to the second feature statistical value as the morphological exposure rate variable R. A temporal evolution sequence representing the truncation of the target human body's morphology by the physical boundary is generated, and the first-order difference sequence ΔR of this temporal evolution sequence is calculated. If the first-order difference sequence ΔR maintains positive difference within a preset number of consecutive frames and the cumulative difference value is greater than a preset action trigger threshold, the target human body is... The body is labeled as a positive traversal state. The coordinate evolution trajectory of the geometric center of the 2D bounding box in the image coordinate system is extracted. The motion direction component and velocity change envelope of the coordinate evolution trajectory in the corresponding time period of the temporal evolution sequence are calculated to construct a displacement vector. A 1D convolutional layer is used to map the binary identifier representing the positive traversal state with the motion direction component and velocity change envelope to generate a spatiotemporal feature tensor representing the evolution process of the target human body boarding the vehicle. The feature similarity distance between this spatiotemporal feature tensor and each pre-stored action fingerprint in the action fingerprint database is calculated. For the calculation and generation of the above velocity change envelope, the system performs first-order discrete sampling of the instantaneous pixel displacement rate of the geometric center at a preset fixed time step. The local extremum retrieval algorithm is used to extract the sequence of maximum points of the velocity matrix in the current evolution period. The spline interpolation function is applied to connect all the separated maximum points to construct a continuous and smooth upper edge envelope. This mathematical process filters out the high-frequency disordered jitter caused by random limb shaking and extracts the core kinematic potential energy change trend representing the overall traversal action.

[0052] This method utilizes a static threshold mask array as a spatial reference system within the image processing domain. It translates the boarding action into a spatial intersection and stripping logic between the target feature pixel set and the mask array in the computational domain. Since the boarding action is accompanied by physical truncation and spatial crossing of the human body and the door boundary, the resulting morphological exposure rate variable R corresponds to a temporal evolution sequence with monotonic topological mutation characteristics. However, interfering targets that linger outside the vehicle or pass by in parallel do not mathematically satisfy this mask truncation characteristic. This mechanism decouples the depth ambiguity caused by single 2D bounding box tracking and combines real-time geographic location parameters, running speed signals, and preset geofence reference coordinates to determine inspection and warning signals. In complex scenarios with dense crowds and occlusion, it outputs feature evidence with causal chains.

[0053] Example 3: In this example, the current simulated physical test platform includes an image acquisition unit installed 2.1m above the door of an operating vehicle. This unit has a resolution of 1920×1080 pixels, a sampling frequency set to 30fps, and a top-down shooting angle of 15 degrees with the vertical center line. ∘ The test video data was collected in real time by the platform. To simulate electromagnetic interference and light and shadow fluctuations in the operating environment, Gaussian white noise with a signal-to-noise ratio of 20dB was superimposed on the original image signal, and a 50Hz light intensity sinusoidal disturbance was introduced.

[0054] The sampling window length N is selected as 15, corresponding to a time span of 0.5s. This parameter is determined based on the physical displacement rate of the target crossing the car door. When the human body's movement speed is between 0.5m / s and 1.5m / s, the 15-frame sampling window covers the critical action phase from the target touching the car door mask boundary to the center of gravity completely crossing it. This ensures that the morphological exposure rate variable R and its first-order difference sequence ΔR exhibit stable monotonicity characteristics in the time domain, filtering out single-frame pixel-level detection errors. In the sample group of this invention, the target human body performs the action of boarding the vehicle. When the target's head touches the static threshold mask array... When the target is at the edge, the processor determines the 2D bounding box of the target human body and counts the moving pixels falling within the static threshold mask array. The initial measured value of the shape exposure variable R is 0.082. As the action evolves, the R value increases to 0.217 in the 5th frame and to 0.425 in the 10th frame. The first-order difference sequence ΔR is greater than 0.031 in these 10 consecutive frames, and the calculated cumulative difference value is 0.342. This value is greater than the preset action trigger threshold of 0.25, and the processor marks the target as in a positive crossing state. When the target's center of gravity enters the center region of the mask, The value shows a non-linear growth trend and tends to level off around 0.914. The spatiotemporal feature tensor matches the action fingerprint database, and the output action confidence score is 0.962.

[0055] The control group simulated background disturbance behavior at a train station. A pedestrian walked parallel to the train body at a distance of 0.5m from the door, generating a 2D pixel displacement. Since the target did not spatially overlap with the physical edge image of the door, the morphological exposure rate variable R measured by the processor fluctuated between 0.024 and 0.052. Its first-order difference sequence ΔR showed alternating positive and negative states, with a cumulative difference value of 0.032, which was below the action trigger threshold. The system did not output a positive crossing state indicator. This control group, lacking the calculation of the morphological exposure rate variable R, was affected by the pedestrian approaching the camera when judging solely based on the 2D bounding box area change rate. Geometric expansion interference generated a false alarm signal. Performance statistics show that under a superimposed 20dB Gaussian white noise environment, the accuracy of the sample group recognition of this invention is 98.2%, while the accuracy of the control group without static threshold mask array constraints is 76.5%. When the noise intensity increases from 10dB to 30dB, the fluctuation range of the accuracy of the sample group recognition of this invention is less than 2.1%. As the interference frequency increases, the monotonicity discrimination logic of the morphological exposure rate variable R remains stable. This method anchors the overall action semantics to the spatial distribution features of the surface pixel set, thus determining the physical evidence chain for the recognition of illegal actions.

[0056] Example 4: This example combines Figures 1 to 2 The method for passenger transport inspection based on target detection is explained, such as... Figure 1 As shown, step 101 obtains a continuous video frame sequence representing the internal environment of the operating vehicle. Step 102 extracts the target detection results of each video frame in the continuous video frame sequence to obtain the two-dimensional bounding box corresponding to the target human body. In step 103, a static threshold mask array that coincides with the physical edge image of the vehicle door in the image coordinate system is loaded in the image region where the two-dimensional bounding box is located. Then, step 104 identifies the moving pixels within the two-dimensional bounding box to construct a feature pixel set. Based on the distribution of their landing points in the static threshold mask array, the morphological exposure rate variable is calculated to generate a temporal evolution sequence representing the truncated form of the target human body. The process involves calculating the first-order difference sequence of the temporal evolution sequence in step 105, maintaining positive difference in the first-order difference sequence in step 106, and marking the target human body as a positive crossing state when the cumulative difference value is greater than the preset action trigger threshold. Then, step 107 is executed to perform feature splicing between the determination result of the positive crossing state and the displacement vector of the two-dimensional bounding box in the time domain to generate a spatiotemporal feature tensor representing the evolution process of the boarding action. Finally, step 108 is executed to calculate the feature similarity distance between the spatiotemporal feature tensor and the pre-stored action fingerprint, and outputting a warning signal based on the action confidence generated by the feature similarity distance and the real-time operating parameters of the operating vehicle.

[0057] like Figure 2As shown, the input layer includes a positive cross-state binary identifier, a rate change envelope, and a motion direction component. These three parameters converge in the feature dimension splicing module to perform feature fusion. The fused data is then input to the one-dimensional convolutional layer dimension mapping module for feature extraction. The resulting spatiotemporal feature tensor, along with the action fingerprint data pre-stored in the action fingerprint database, enters the feature similarity distance calculation unit. The action confidence calculated by this unit, along with the real-time running parameters, serves as the input to the warning signal output module, which ultimately outputs the warning signal.

[0058] Example 5: In the initial installation stage of the terminal in the operating vehicle, the processor retrieves the intrinsic parameter matrix K and the extrinsic parameter matrix M relative to the vehicle chassis coordinate system of the image acquisition unit installed on the top of the vehicle. It also retrieves the 3D vertex coordinate sequence of the passenger vehicle door in the vehicle body structure design drawing. Through coordinate projection transformation, the 3D vertex coordinate sequence is mapped to the image coordinate system to determine the 2D pixel boundary. In the image plane area enclosed by the 2D pixel boundary, a binarization algorithm is used to fill the grayscale mapping value of each pixel point, and a static threshold mask array that coincides with the physical edge image of the door in the image coordinate system is determined.

[0059] The processor acquires the first 100 frames from a continuous video frame sequence as background modeling samples. It uses a Gaussian mixture model to establish a background probability distribution containing three Gaussian components for each pixel in the image coordinate system. It calculates the discrete deviation between the current frame's pixel grayscale value and the corresponding background probability distribution. Pixels with a grayscale deviation greater than 2.5 times the standard deviation of the distribution are identified as pixels with motion attributes and included in the feature pixel set. The processor retrieves 50 pre-stored standard boarding video samples from the storage unit, calculates the cumulative difference value corresponding to each standard sample, determines the average cumulative difference value μ and the standard deviation σ, and then applies the formula... The action trigger threshold is determined. In this implementation scenario, the action trigger threshold is set to 0.25, where T is the action trigger threshold, μ is the average of the cumulative difference values, and σ is the standard deviation of the cumulative difference values.

[0060] Based on the uniformity approximation principle of rigid body kinematics, the optical flow velocity vectors generated by pixels on the surface of the same physical entity within a very short time window exhibit similarity in both height direction and amplitude. A density-space clustering algorithm is used to segment the feature pixel set, and the optical flow vectors of each pixel between adjacent video frames are extracted. Calculate each optical flow vector Angle θ between the door and the perpendicular line to the door normal and the vector magnitude Determine the threshold for the included angle distance metric. Distance metric threshold The relative difference of the included angle θ is less than And amplitude The relative difference is less than Spatially neighboring pixels are grouped into the same effective pixel subset, and the optical flow vector... The physical mapping of instantaneous velocity and orientation of a pixel in a two-dimensional image plane, the included angle θ and its corresponding angular distance metric threshold. The motion trajectory is limited to crossing the vector angle boundary condition towards the interior of the carriage, and the vector amplitude is... Distance metric threshold Limit the discrete tolerance range of motion rate at the same location and set a uniform feature alignment time step. The cubic spline interpolation algorithm is used to uniformly resample the envelope of rate change with time dimension differences and the motion direction component to a length of [length missing]. The sequence format synchronously copies and fills the binary identifier representing the positive crossing state into a constant time series of the same length. When the one-dimensional convolutional layer generates the spatiotemporal feature tensor, the convolution kernel stride is set to 2 pixels. The binary identifier of the positive crossing state, the motion direction component, and the envelope channel of the rate change are spliced ​​to determine the mapping relationship between the physical motion signal and the semantic feature space dimension. The processor retrieves the feature tensor set extracted from 1000 groups of labeled boarding action sequences, calculates the arithmetic mean of each dimension of features on the time axis, and constructs the benchmark fingerprint vector in the action fingerprint database. This method uses the geometric projection procedure to determine the physical boundary benchmark of image processing, and determines the causal relationship between motion pixel recognition and threshold determination through background modeling and statistical calibration. This makes the determination results of the morphological exposure rate variable R and its first-order difference sequence ΔR have a traceable physical basis, improving the stability of the generation of inspection and warning signals.

[0061] Example 6: In this example, during the operation of a commercial vehicle, the processor monitors the overlap between the static threshold mask array and the physical edge image of the vehicle door to address visual axis shift caused by vehicle vibration. The calibration program extracts the pixel gradient magnitudes within the reference boundary of the static threshold mask array and its 20-pixel outward neighborhood within a preset detection period. Using the reference mask boundary determined during initial installation as a reference template, a two-dimensional normalized cross-correlation matching coefficient matrix is ​​calculated within the outward pixel neighborhood. The pixel coordinate displacement corresponding to the maximum point in this matrix is ​​identified, and a two-dimensional affine translation vector is determined to characterize the relative displacement between the image coordinate system and the physical door reference system. Based on the vector, translation compensation calculation is performed on the original vertex coordinates of the static threshold mask array. The calibrated mask area is used as the physical reference for extracting feature pixel set to calculate the morphological exposure rate variable R. For the three-dimensional rotation and deflection of the camera optical axis caused by mechanical vibration fatigue under long-term operating conditions of passenger vehicles, a baseline adaptive update procedure is set. The cumulative mileage parameter of the vehicle is monitored. When the cumulative mileage reaches the preset calibration trigger threshold, the processor extracts the two-dimensional projected pixel coordinates of three preset physical feature reference points in the door pedal area within the target video frame. According to the perspective projection imaging principle, the geometric mapping from the physical three-dimensional spatial feature points to the two-dimensional imaging plane is rigidly constrained by the camera extrinsic parameter matrix. The homography matrix H between the currently extracted pixel coordinates and the reference coordinate sequence under the initial calibration state is calculated using the direct linear transformation algorithm. Singular value decomposition is performed on the homography matrix H to extract a three-dimensional rotation compensation matrix containing the current pitch angle offset component and yaw angle offset component of the optical axis. , the three-dimensional rotation compensation matrix The updated extrinsic parameter matrix is ​​output by multiplying the initial extrinsic parameter matrix M. The updated extrinsic parameter matrix is ​​then used to re-execute the mapping and projection procedure from the 3D vertex coordinate sequence to the image coordinate system. The homography matrix H represents the algebraic relationship of a specific perspective mapping between two non-parallel projective planes. The 3D rotation compensation matrix... It represents the physical correction amount of angular displacement generated by a three-dimensional spatial coordinate system around a fixed geometric axis.

[0062] When the system performs data filling of the action fingerprint database, the processor retrieves 1000 sets of labeled video sequences of boarding actions covering different lighting environments and different human body shape characteristics as benchmark training samples. The processor obtains the original spatiotemporal feature vectors of each sample in the positive traversal state through a feature extraction algorithm. Global average pooling is used to map the original vectors with temporal dimension differences into fixed-length 128-dimensional feature vectors. The arithmetic mean of the feature vectors of the same type of action in the feature space is calculated as the benchmark center vector of the action fingerprint. The Euclidean distance distribution of each sample vector relative to this benchmark center vector is statistically analyzed, and the distance value corresponding to the 95% confidence interval is selected as the similarity. The decision boundary for degree determination enables the real-time generated spatiotemporal feature tensor to output action confidence values ​​based on this decision boundary. To ensure the absolute self-consistency of this feature similarity determination in mathematical and physical logic, all benchmark training samples undergo pre-processing identical to the real-time operating conditions before performing dimensionality reduction mapping. They are truncated by a static threshold mask array to obtain the rate change envelope and motion direction, and these underlying mechanical parameters are input into a homogeneous one-dimensional convolutional layer for processing. This ensures that the final generated benchmark center vector and the real-time spatiotemporal feature tensor maintain strict homology in data space distribution and abstract semantics, eliminating physical scale misalignment during the comparison process.

[0063] Example 7: In this example, during inspections of vehicles with high passenger flow, the processor monitors the overlap between the static threshold mask array and the physical edge image of the vehicle door to address visual axis shift. The system calibration program retrieves the time smoothing coefficient α within a preset detection period, setting the value range of α to 0.6 to 0.8 to balance feature extraction sensitivity and noise reduction intensity. The determined smoothed variable... Satisfying the relation ,in The smoothed shape exposure rate variable. The shape exposure rate variable measured in the current frame. The variable representing the smoothed morphological exposure rate of the previous frame is α, where α is the time smoothing coefficient. When the processor detects that the first-order difference sequence ΔR undergoes more than three polarity switches within a 15-frame sampling window, it determines that the target human body is in a lingering state and automatically retrieves the duration variable of the car door opening signal. Numerical incremental compensation is applied to the action trigger threshold T, with the compensation increment based on the proportional coefficient k and the duration variable. The product of these factors determines the triggering benchmark for action detection, which shifts upward with the duration of the stop, offsetting false positive crossing state flags caused by a person lingering at the door. Here, k is a proportionality coefficient. For the duration of the door opening signal, in this dynamic incremental compensation mechanism, the proportional coefficient k is not a randomly set numerical constant, but a statistical empirical quantity derived through linear regression analysis based on tens of thousands of passenger flow monitoring data collected during the research and development period from typical passenger stations. The specific value of this proportional coefficient is established to ensure that the area under the system's operating characteristic curve is optimized. Thus, through objective real-world data feedback, the upward fluctuation of the system threshold has a quantitative support basis to resist fluctuations in the real environment.

[0064] The processor extracts the smoothed temporal evolution sequence and calculates the slope of the change in morphological exposure rate. It uses a shift register containing 5 delay units to temporarily store the feature vectors corresponding to consecutive video frames. It compares the motion direction components stored in each unit of the register. When more than 4 units of the register point to the interior space, it is confirmed as a steady-state motion trend. By comparing the positive and negative polarities of this slope, discrete human posture disturbance signals are filtered out, and the dimensional mapping path from the original pixel distribution features to the semantic feature space is determined. This method establishes calibration criteria for complex behavior patterns by introducing a time smoothing coefficient α and polarity monitoring logic. When the inspection system faces situations such as human retreat or pause, it compensates for data dispersion by adjusting the action trigger threshold T. The evolution characteristics of the morphological exposure rate variable R still maintain consistency with the physical motion trend in non-ideal environments containing interference, improving the confidence score when matching the spatiotemporal feature tensor with the action fingerprint database and solving the state recognition bias in complex passenger flow backgrounds.

[0065] To verify the basis for generating the violation inspection warning signal, after receiving the action confidence score, the business logic unit simultaneously extracts the real-time geographic coordinates output by the vehicle-mounted satellite positioning module and the transient rate signal fed back by the vehicle speed sensor, calculates the Euclidean distance between the current geographic coordinates and the preset station reference point, and determines the geographic distance offset. In terms of geographical distance offset Under conditions where the distance is greater than 200m and the vehicle's transient speed signal is less than 5km / h, the vehicle is determined to be in an abnormal state of soliciting passengers outside the designated station. The processor compares the obtained action confidence score with the confidence benchmark fed back by the action fingerprint database. If the action confidence score is greater than 0.90 for 30 consecutive frames, the processor outputs an inspection warning signal that confirms the violation, based on the determination of the abnormal state of soliciting passengers outside the designated station.

[0066] The embodiments of this application have been described above with reference to the accompanying drawings. Unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other. This application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit of this application and the scope of protection of this invention, and all of these forms are within the protection scope of this application.

Claims

1. A passenger transport operation inspection method based on target detection, characterized in that, Includes the following steps: Step 101: Obtain a continuous video frame sequence representing the internal environment of the operating vehicle; Step 102: Extract the target detection results of each video frame in the continuous video frame sequence. The target detection results include the two-dimensional bounding box corresponding to the target human body. Step 103: Load a static threshold mask array within the image region where the two-dimensional bounding box is located. The static threshold mask array coincides with the physical edge image of the vehicle door in the image coordinate system. Step 104: Identify pixels with motion attributes within the two-dimensional bounding box to construct a feature pixel set. Based on the distribution of the feature pixel set within the static threshold mask array, calculate the morphological exposure rate variable corresponding to each video frame and generate a temporal evolution sequence characterizing the physical boundary truncation of the target human body morphology. Step 105: Calculate the first-order difference sequence of the time-series evolution sequence; Step 106: If the first-order difference sequence maintains positive difference within a preset number of consecutive frames and the cumulative difference value is greater than a preset action trigger threshold, the target human body is marked as a positive crossing state. Step 107: The determination result representing the positive crossing state is concatenated with the displacement vector of the two-dimensional bounding box in the time domain to generate a spatiotemporal feature tensor representing the evolution process of the target human boarding action. Step 108: Calculate the feature similarity distance between the spatiotemporal feature tensor and each pre-stored action fingerprint in the action fingerprint database, and output an early warning signal based on the action confidence generated by the feature similarity distance and the real-time operating parameters of the operating vehicle.

2. The passenger transport operation inspection method based on target detection according to claim 1, characterized in that, Step 107 involves concatenating the determination result representing the positive crossing state with the displacement vector of the two-dimensional bounding box in the time domain to generate a spatiotemporal feature tensor representing the evolution of the target human's boarding action. This includes: Step 1071, extracting the coordinate evolution trajectory of the geometric center of the two-dimensional bounding box in the image coordinate system; Step 1072, calculating the motion direction component and velocity change envelope of the coordinate evolution trajectory within the corresponding time period of the temporal evolution sequence to construct the displacement vector; Step 1073, using a one-dimensional convolutional layer to perform dimensional mapping between the binary identifier representing the positive crossing state and the motion direction component and velocity change envelope to generate the spatiotemporal feature tensor.

3. The passenger transport operation inspection method based on target detection according to claim 1, characterized in that, Before loading the static threshold mask array in the image region where the two-dimensional bounding box is located in step 103, the method further includes: step 1031, extracting the reference boundary of the static threshold mask array and the pixel gradient magnitude in its extended pixel neighborhood; step 1032, using the pre-stored reference mask boundary as a template, determining the two-dimensional affine translation vector corresponding to the extreme value of gradient magnitude overlap in the extended pixel neighborhood using normalized cross-correlation matching operation; and step 1033, performing coordinate axis translation calibration on the static threshold mask array based on the two-dimensional affine translation vector.

4. The passenger transport operation inspection method based on target detection according to claim 1, characterized in that, The morphological exposure rate variable is calculated through the following steps: Step 1041, the number of pixels that fall into the static threshold mask array in the statistical feature pixel set is determined as the first feature statistical value; Step 1042: Count the total number of pixels in the feature pixel set and determine it as the second feature statistical value; Step 1043: The ratio of the first feature statistic to the second feature statistic is determined as the morphological exposure rate variable.

5. The passenger transport operation inspection method based on target detection according to claim 1, characterized in that, Step 104, based on the distribution of the feature pixel set's landing points within the static threshold mask array, calculates the morphological exposure rate variable corresponding to each video frame, further including: Step 1044, extracting the pixel-level optical flow vector of each pixel in the feature pixel set between adjacent video frames; Step 1045, using the directional consistency constraint of the pixel-level optical flow vector to cluster and segment the feature pixel set, separating the effective pixel subset that meets the feature of crossing into the vehicle from the overlapping human pixel clusters; Step 1046, mapping the effective pixel subset to the static threshold mask array, and calculating the morphological exposure rate variable corresponding to the effective pixel subset.

6. The passenger transport operation inspection method based on target detection according to claim 1, characterized in that, Before acquiring the continuous video frame sequence representing the internal environment of the operating vehicle in step 101, the method further includes: step 1011, monitoring the physical state signal of the vehicle door; step 1012, triggering the acquisition of the continuous video frame sequence when the physical state signal of the vehicle door indicates that the door is in an open state.

7. The passenger transport operation inspection method based on target detection according to claim 1, characterized in that, Step 108: Calculate the feature similarity distance between the spatiotemporal feature tensor and each pre-stored action fingerprint in the action fingerprint database, and output a warning signal based on the action confidence generated by the feature similarity distance and the real-time operating parameters of the operating vehicle. This includes: Step 1081: Obtain the real-time operating parameters of the operating vehicle, including real-time geographic location parameters, operating speed signal, and preset geofence reference coordinates; Step 1082: Calculate the spatial distance offset between the real-time geographic location parameters and the preset geofence reference coordinates; Step 1083: Output a warning signal when the action confidence is greater than a preset probability threshold, the spatial distance offset is greater than 50m, and the operating speed signal is less than 5km / h.

8. The passenger transport operation inspection method based on target detection according to claim 1, characterized in that, Step 102, extract the target detection results of each video frame in the continuous video frame sequence, including: Step 1021, use a convolutional neural network model to perform feature mapping on each video frame to generate a multi-channel feature map; Step 1022, perform target candidate region regression based on the multi-channel feature map and output the vertex coordinates of the two-dimensional bounding box.

9. A passenger transport operation inspection method based on target detection according to claim 7, characterized in that, After outputting the warning signal in step 1083, the method further includes: step 1084, which associates and stores the feature evidence chain characterizing the evolution of the target human's boarding action with the warning signal.

Images