Platform door gap intruder detection method, apparatus, device, and storage medium

Abstract

The application provides a platform door gap intruder detection method, device, equipment and storage medium. The method comprises the following steps: acquiring an RGB image and a depth image of a platform door gap area, and establishing a spatial mapping relationship between the RGB image and the depth image; performing target detection based on the RGB image to obtain at least one candidate target area and an image detection result corresponding to each candidate target area; determining a candidate depth area corresponding to each candidate target area in the depth image based on the spatial mapping relationship; determining a depth anomaly area in the candidate depth area, and extracting a depth area feature of the candidate depth area; performing spatial matching on the depth anomaly area and the corresponding candidate target area to obtain a spatial consistency feature; and detecting whether there is an intruder in the platform door gap area based on the image detection result, the depth area feature and the spatial consistency feature. The application can improve the accuracy of intruder detection of the platform door gap.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method, apparatus, equipment and storage medium for detecting intrusions through gaps in platform screen doors. Background Technology

[0002] After a subway train stops, a gap is formed between the platform screen doors and the train doors, requiring safety checks. Objects such as passenger limbs, clothing, backpack straps, umbrellas, and plastic bags can potentially enter this gap. The location, width, and visibility of this gap in the image may vary depending on the train model, stopping location, and door structure.

[0003] When the train body and door frame obstruct the lighting, the ambient brightness is low, glass reflects or transmits light, or the target is only partially visible, the appearance of the target in the visible light image may not match the actual space occupied within the platform door gap. Furthermore, when different imaging units are used to obtain image and depth information, the two types of images may differ in terms of coordinate system, spatial resolution, and field of view, making it difficult to directly confirm whether the candidate target in the image actually occupies space at the corresponding physical location.

[0004] Therefore, how to reduce false or missed detections of intruders under conditions of train obstruction and low light is a technical problem that needs to be solved. Summary of the Invention

[0005] In view of this, the present application provides a method, apparatus, device, and storage medium for detecting intrusions through platform screen door gaps to solve at least one problem existing in the background art.

[0006] In a first aspect, a method for detecting intrusions through platform screen door gaps is provided, the method comprising: Acquire RGB and depth images of the platform door gap area, and establish a spatial mapping relationship between the RGB images and the depth images; Target detection is performed based on the RGB image to obtain at least one candidate target region and the image detection result corresponding to each candidate target region; Based on the spatial mapping relationship, a candidate depth region corresponding to each candidate target region is determined in the depth image; Depth anomaly regions are identified within the candidate depth regions, and depth region features of the candidate depth regions are extracted. Spatial matching is performed between the depth anomaly region and the corresponding candidate target region to obtain spatial consistency features; Based on the image detection results, the depth region features, and the spatial consistency features, the presence of intruders within the gap area of ​​the platform door is detected.

[0007] In conjunction with the first aspect, in an alternative implementation, before performing target detection based on the RGB image, the method further includes: Based on the RGB image and the depth image, the platform door boundary, train door boundary, structural region, and depth abrupt change region are determined; the structural region includes at least one of the door frame structural region and the car body structural region. The initial gap area is determined based on the relative position between the platform door boundary and the train door boundary; The initial gap region is morphologically processed to obtain the boundary constraint region; The depth constraint region is determined based on the overlap between the boundary constraint region and the depth abrupt change region; The structural region is excluded from the depth-constrained region, and the region after excluding the structural region is filtered for connected components to obtain the region of interest; wherein the region of interest is used to perform the target detection.

[0008] In conjunction with the first aspect, in an alternative implementation, the method further includes: Based on the region of interest, a region mask is constructed, and the region mask is used to crop or filter the non-detection regions of the RGB image and the depth image to obtain the RGB detection image and the depth detection image. The target detection based on the RGB image includes: The target detection is performed based on the RGB detection image; The step of determining the candidate depth region corresponding to each candidate target region in the depth image based on the spatial mapping relationship includes: Based on the spatial mapping relationship, a candidate depth region corresponding to each candidate target region is determined in the depth detection image.

[0009] In conjunction with the first aspect, in an optional implementation, determining the depth anomaly region within the candidate depth region includes: Pixels that meet the preset abnormality conditions are selected from the candidate depth regions to obtain an initial set of abnormal pixels; The initial abnormal pixel set is subjected to connectivity analysis to extract connected pixel groups that meet preset connectivity conditions, forming the depth abnormal region; The preset abnormal conditions include at least one of the following: A depth mutation condition is used to filter pixels in the candidate depth region whose depth value is less than a preset proximity threshold when the difference between the minimum effective depth of the candidate depth region and the neighborhood background depth of the candidate depth region is greater than a preset depth mutation threshold. Distance anomaly condition is used to filter pixels in the candidate depth region whose depth gradient is greater than a preset gradient threshold; wherein, the depth gradient is the rate of change of depth of the boundary pixels of the candidate depth region along the edge normal direction; Spatial protrusion condition is used to calculate the protrusion height of the depth of each pixel in the candidate depth region relative to the depth of the neighboring background, and to filter pixels whose protrusion height is greater than a preset protrusion threshold. The protrusion height corresponding to any pixel is the difference between the depth value of the neighboring background and the depth value of the pixel. The neighborhood background depth is determined based on the depth statistics of the effective depth points within the annular band region, which is the region formed by extending a preset pixel distance outward from the boundary of the candidate depth region.

[0010] In conjunction with the first aspect, in an optional implementation, extracting the depth region features of the candidate depth region includes: A set of effective depth points is determined within the candidate depth region, and the effective depth ratio is determined based on the number of effective depth points in the set of effective depth points and the total number of pixels within the candidate depth region. Based on the depth values ​​in the set of effective depth points, determine the depth variance and minimum effective depth of the candidate depth region; determine the neighborhood background depth based on the depth values ​​surrounding the candidate depth region, and determine the depth difference between the neighborhood background depth and the minimum effective depth; A depth verification score is determined based on at least one of the depth difference, the effective depth ratio, and the depth variance, and the depth verification score is used as the depth region feature. The depth verification score is positively correlated with the depth difference and the effective depth ratio, and negatively correlated with the depth variance.

[0011] In conjunction with the first aspect, in an optional implementation, the step of spatially matching the depth anomaly region with the corresponding candidate target region to obtain spatial consistency features includes: Based on the spatial mapping relationship, the depth anomaly region is mapped to the coordinate system of the RGB image; Determine the intersection and union regions between the mapped depth anomaly region and the corresponding candidate target region; The spatial consistency feature is determined based on the ratio of the area of ​​the intersection region to the area of ​​the union region.

[0012] In conjunction with the first aspect, in an optional implementation, the image detection result includes RGB detection confidence; the step of detecting whether an intruder exists within the platform door gap area based on the image detection result, the depth region features, and the spatial consistency features includes: The RGB detection confidence, the depth region features, and the spatial consistency features are weighted and fused according to their respective fusion weights to obtain the intrusion risk score. If the intrusion risk score reaches the risk threshold, it is determined that an intruder exists in the candidate target area.

[0013] In conjunction with the first aspect, in an optional implementation, the fusion weights are determined in the following manner: Based on the brightness characteristics in the RGB image, determine the RGB quality index; Determine the depth quality index based on the effective depth ratio and depth variance of the candidate depth region; Based on the threshold conditions satisfied by the RGB quality index and the depth quality index, the baseline weights corresponding to at least one of the RGB detection confidence and the depth region features are selectively adjusted. The adjusted weights and the baseline weights corresponding to the spatial consistency features are normalized to obtain the fusion weights corresponding to the RGB detection confidence, the depth region features, and the spatial consistency features, respectively.

[0014] In conjunction with the first aspect, in an alternative implementation, the method further includes: Temporal stability verification is performed on the intrusion risk scores corresponding to the same candidate target in multiple consecutive image frames; The proportion of target image frames whose intrusion risk score reaches the risk threshold in a predetermined number of consecutive image frames is counted. When the ratio reaches the stability threshold, a detection result indicating the presence of an intruder in the candidate target is output.

[0015] Secondly, a platform door gap intrusion detection device is provided, comprising: The image acquisition and mapping module is configured to acquire RGB images and depth images of the platform door gap area, and establish a spatial mapping relationship between the RGB images and the depth images; The image detection module is configured to perform target detection based on the RGB image to obtain at least one candidate target region and an image detection result corresponding to each candidate target region; The depth region determination module is configured to determine a candidate depth region corresponding to each candidate target region in the depth image based on the spatial mapping relationship. The depth information extraction module is configured to determine depth anomaly regions within the candidate depth region and extract depth region features from the candidate depth region. The spatial matching module is configured to perform spatial matching between the depth anomaly region and the corresponding candidate target region to obtain spatial consistency features; The fusion detection module is configured to detect whether there is an intruder in the gap area of ​​the platform door based on the image detection results, the depth region features and the spatial consistency features.

[0016] Thirdly, an electronic device is provided, including a processor and a memory, wherein the memory stores a computer program or instructions, and when the computer program or instructions are executed by the processor, the electronic device performs the steps of the platform door gap intrusion detection method provided in any optional embodiment of the first aspect.

[0017] Fourthly, a computer-readable storage medium is provided, on which a computer program or instructions are stored, which, when executed by a processor, implement the steps of the platform door gap intrusion detection method provided in any optional embodiment of the first aspect.

[0018] The platform screen door gap intrusion detection method, apparatus, device, and storage medium provided in this application first establish a spatial mapping relationship between an RGB image and a depth image. Then, based on the spatial mapping relationship and each candidate target region in the RGB image, a candidate depth region corresponding to the candidate target region is determined in the depth image. Furthermore, depth anomaly regions representing abnormal locations and depth region features representing the depth state of the regions are extracted from the candidate depth regions. Thus, the depth information used for the candidate target is limited to the physical region corresponding to that candidate target.

[0019] Based on this, spatial consistency features are obtained by spatially matching depth anomaly regions with candidate target regions to represent the degree of spatial correspondence between the two. These features are then combined with image detection results, depth region features, and spatial consistency features for intrusion detection. Therefore, compared to the risks of false positives or false negatives caused by fixed rectangular detection areas or single visual detection, this application can improve the accuracy of intrusion detection in platform door gaps under complex conditions such as train obstruction, low light, reflection, and partial target visibility. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating a method for detecting intrusions through gaps in platform screen doors, according to one embodiment. Figure 2 This is a flowchart illustrating a method for determining a region of interest according to an embodiment; Figure 3 This is a flowchart illustrating a method for region masking and candidate depth region determination according to an embodiment; Figure 4 yes Figure 1 One of the flowcharts for step S140 shown; Figure 5 yes Figure 1 The second flowchart of step S140 is shown. Figure 6 yes Figure 1 The flowchart of step S150 is shown below; Figure 7 yes Figure 1 The flowchart of step S160 is shown below; Figure 8 This is a schematic diagram illustrating the process of determining fusion weights according to one embodiment; Figure 9 This is a schematic diagram of a platform door gap intrusion detection device according to an embodiment. Detailed Implementation

[0021] To make the technical solution and beneficial effects of this application more apparent and understandable, a detailed description is provided below by listing specific embodiments. The accompanying drawings are not necessarily drawn to scale, and local features may be enlarged or reduced to more clearly show the details of the local features; unless otherwise defined, the technical and scientific terms used herein have the same meanings as those in the technical field to which this application pertains.

[0022] The embodiments in this application are not exhaustive, but merely illustrative of some embodiments, and are not intended to limit the scope of protection of this disclosure. Unless otherwise specified, each step in a particular embodiment can be implemented as an independent embodiment, and the steps can be arbitrarily combined. For example, a solution after removing some steps in a particular embodiment can also be implemented as an independent embodiment, and the order of the steps in a particular embodiment can be arbitrarily interchanged. Furthermore, the optional implementation methods in a particular embodiment can be arbitrarily combined; moreover, the embodiments can be arbitrarily combined, for example, some or all steps of different embodiments can be arbitrarily combined, and a particular embodiment can be arbitrarily combined with the optional implementation methods of other embodiments.

[0023] In each embodiment of this application, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions of the embodiments are consistent and can be referenced by each other. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships.

[0024] In the description of the embodiments of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, features defined as "first" and "second" may explicitly or implicitly include one or more features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0025] After a subway train stops at a station, a narrow gap is formed between the platform screen doors and the train doors. Due to the influence of train type, stopping location, relative position of the platform screen doors and train doors, and door structure, the image area corresponding to this gap is not constant; when objects such as passenger limbs, clothing, backpack straps, umbrellas, and plastic bags enter this gap, only a partial outline may be shown.

[0026] In this scenario, the appearance of a target in the image does not always correspond one-to-one with its actual spatial occupancy. Obstruction of lighting by the train body and door frame, changes in ambient brightness, and partial visibility of the target may cause objects entering the gap to lack complete texture and boundaries. Furthermore, reflections or transmissions from the platform screen door glass, train door glass, and the train body surface may create a target appearance in the image that does not correspond to its actual spatial occupancy. Therefore, it is difficult to determine whether a candidate target actually occupies the platform screen door gap space based solely on the appearance information presented in the image.

[0027] Depth information describes the distance relationship between the measured location and the depth sensing device, but it differs from the representation of image appearance. On one hand, depth data can be affected by invalid depth points, measurement noise, and discontinuous target edge responses; on the other hand, depth information itself typically cannot directly indicate the target category. Furthermore, when the RGB imaging unit and the depth sensing device use different coordinate systems, spatial resolutions, or field of view, the location of a candidate target in the RGB image cannot be directly used as the value region in the depth image, and local changes in the depth image cannot be automatically considered as the spatial occupancy of that candidate target.

[0028] Based on this, embodiments of this application provide a method for detecting intruders through gaps in platform screen doors. This method determines candidate target regions in an RGB image, determines corresponding candidate depth regions based on the spatial mapping relationship between the RGB image and the depth image, extracts depth anomaly regions and depth region features within the candidate depth regions, obtains spatial consistency features through spatial matching between the depth anomaly regions and the candidate target regions, and detects the presence of intruders based on the image detection results, depth region features, and spatial consistency features.

[0029] like Figure 1 As shown in the embodiment of this application, the method for detecting intruders in platform screen doors includes the following steps: S110: Acquire RGB and depth images of the platform door gap area and establish a spatial mapping relationship between the RGB and depth images.

[0030] S120: Perform target detection based on RGB images to obtain at least one candidate target region and the image detection result corresponding to each candidate target region.

[0031] S130: Based on spatial mapping relationships, determine the candidate depth region corresponding to each candidate target region in the depth image.

[0032] S140: Identify depth anomaly regions within the candidate depth regions and extract depth region features from the candidate depth regions.

[0033] S150: Spatial matching is performed between deep anomaly regions and corresponding candidate target regions to obtain spatial consistency features.

[0034] S160: Based on image detection results, depth region features, and spatial consistency features, detect whether there are intruders in the gap area of ​​the platform door.

[0035] The platform screen door gap intrusion detection method provided in this application can be executed by an electronic device with image processing capabilities. This electronic device can be connected to an RGB imaging unit and a depth sensing device to receive RGB images and depth images; it can also receive already acquired RGB images and depth images from other data acquisition devices or storage devices. This application does not limit the specific form of the electronic device executing this method. The depth sensing device can be at least one of a Time-of-Flight (ToF) depth camera, a structured light depth camera, a binocular or multi-view depth camera, or a laser depth sensor. The RGB imaging unit and the depth sensing device can be integrated or separate; in the case of separate installation, they can be jointly calibrated to obtain spatial mapping parameters between the two imaging coordinate systems. The RGB imaging unit and the depth sensing device can be installed above, to the side above, or in other locations that can cover the gap area between the platform screen door and the train door; when they are separate installations, their respective installation positions can be different, but their fields of view must at least cover a common platform screen door gap detection area.

[0036] In step S110 above, the platform screen door gap area refers to the spatial area between the platform screen door and the stopped train that needs to be detected for intrusions. This area may include the gap between the platform screen door and the train door, or it may include an area adjacent to the gap that may be occupied by an intruder. The platform screen door gap area usually has the same shape and position in different images, but it may also change with the train's stopping position and the state of the door. An RGB image refers to a visible light image that can characterize the platform screen door gap area. An RGB image can be an image frame directly output by an RGB imaging unit, or it can be an image obtained after correcting, cropping, or other preprocessing of an image frame. An RGB image can provide visible light image information for target detection. A depth image refers to an image that can characterize the depth or distance values ​​of each sampling position within the platform screen door gap area. The pixel positions in the depth image correspond to the spatial sampling positions, and the pixel values ​​are used to characterize the depth information of the corresponding spatial sampling positions. Due to factors such as reflectivity, occlusion, or the measurement range of the depth sensing device, there may be pixels in the depth image that do not have effective depth values.

[0037] In this step, the RGB image and depth image can be image frames from the RGB image stream and depth data stream, respectively. When detecting dynamic scenes, RGB and depth images corresponding to the acquisition time can be obtained from the RGB image stream and depth data stream, so that the two images represent the state of the platform door gap area at the same or similar time. After obtaining the RGB and depth images, lens distortion correction can be performed on the corresponding images. Alternatively, corresponding image frames can be determined from continuously acquired images based on the acquisition time of the RGB and depth images. Lens distortion correction and acquisition time correspondence processing are used to ensure that the image positions on which subsequent spatial mapping is based have a corresponding spatial correspondence basis.

[0038] Spatial mapping refers to the coordinate correspondence between positions in an RGB image and positions in a depth image, used to determine the image coordinates of the same physical location or region in the two images. In this step, the spatial mapping relationship can be determined based on the intrinsic parameters of the RGB imaging unit, the intrinsic parameters of the depth sensing device, and the extrinsic parameters between the two imaging units. Alternatively, a coordinate mapping table can be pre-generated during the calibration process and called during intrusion detection. The spatial mapping relationship can represent a mapping from the RGB image coordinate system to the depth image coordinate system, or a mapping from the depth image coordinate system to the RGB image coordinate system, or it can include both mapping directions simultaneously.

[0039] In this design, the RGB image and the depth image do not need to have the same spatial resolution. For a pixel, set of pixels, or image region in the RGB image, the corresponding pixel, set of pixels, or image region in the depth image can be determined through a spatial mapping relationship. Correspondingly, for depth anomaly regions in the depth image, their corresponding regions in the RGB image coordinate system can also be determined based on this spatial mapping relationship, thus providing a coordinate basis for subsequent spatial matching.

[0040] In step S120 above, an RGB image can be input into a trained object detection model, which then detects potential intruders within the RGB image. The object detection model can be trained based on pre-acquired images of the platform door gap scene, enabling it to output the location of candidate targets and corresponding image detection results. Here, a candidate target region refers to an image region obtained through RGB image object detection that may contain the object to be detected; the candidate target region indicates the location to be depth-checked and does not necessarily indicate the actual presence of an intruder in that region; the candidate target region can be represented by a candidate target bounding box or other data used to represent the target's location and extent. The image detection result refers to the detection information corresponding to the candidate target region obtained from object detection based on the RGB image. The image detection result may include one or more of the following: the location, scale, candidate category, and detection confidence of the candidate target region.

[0041] In this step, for each candidate target output by the target detection model, the candidate target region can be determined based on the position and range of the candidate target. The candidate target region is represented by the candidate target bounding box. The image detection result includes the candidate category and detection confidence corresponding to the candidate target region. The candidate category is used to characterize the object type to which the candidate target may belong, and the detection confidence is used to characterize the degree of support of the image information in the RGB image for the detection result.

[0042] Candidate targets identified by object detection can include passenger limbs, clothing, backpack straps, umbrellas, plastic bags, or other objects that may enter the platform screen door gap area. Due to factors such as reflections, shadows, images of people inside the carriage, and partial visibility of the target, the candidate targets output by the object detection model may not necessarily correspond to the actual space occupied within the platform screen door gap area. Therefore, the candidate target areas and image detection results output in this step are used as input for subsequent depth verification and fusion detection, and not directly as the final result of intrusion detection.

[0043] In step S130 above, for each candidate target region obtained in step S120, its corresponding position in the depth image is determined according to the spatial mapping relationship established in step S110. The area covered by the pixels corresponding to the candidate target region in the depth image is determined as the candidate depth region corresponding to that candidate target region. The candidate depth region is used to define the spatial range for depth information processing for the corresponding candidate target. The candidate depth region and the candidate target region correspond to the same physical region in the platform door gap space, but their coordinate positions, sizes, and shapes in their respective images do not need to be the same.

[0044] In this step, the candidate target region is located in the coordinate system of the RGB image, and the candidate depth region is located in the coordinate system of the depth image. When the spatial mapping relationship represents a mapping from the RGB image coordinate system to the depth image coordinate system, the boundary position of the candidate target region or the pixels covered by the candidate target region can be mapped to the depth image, and the candidate depth region can be determined based on the mapping result. When using reverse mapping or a calibration mapping table, the depth image pixels corresponding to the candidate target region can also be selected based on the corresponding positions of the pixels in the depth image in the RGB image. By determining the candidate depth region for each candidate target region separately, subsequent depth information processing can be performed around the candidate targets already detected in the RGB image.

[0045] In step S140 above, within each candidate depth region, a depth anomaly region is determined based on the depth values ​​within the candidate depth region. Specifically, depth pixels exhibiting abrupt depth changes, distance anomalies, or spatial protrusions can be identified based on the changes in depth values ​​within the candidate depth region relative to adjacent space or the surrounding background. Depth anomaly regions are then formed based on the positions of these depth pixels within the candidate depth region. These depth anomaly regions represent locations within the physical area corresponding to the candidate target that may be occupied by space. For example, when a candidate target in an RGB image corresponds to an object actually entering the gap area of ​​a platform door, the object may appear in the depth image as a local area closer to the depth sensing device than the surrounding background, or as a depth variation different from the surrounding spatial structure. Conversely, reflected or transmitted images in the RGB image may not form corresponding depth anomaly regions at the corresponding locations.

[0046] In addition to identifying depth anomaly regions, depth region features are extracted based on depth information within candidate depth regions. These features can include characteristics describing the distance distribution, effective depth point distribution, depth value dispersion, depth variation, or local spatial convexity of the candidate depth region. For example, corresponding depth region features can be obtained based on one or more of the following: minimum depth, average depth, effective depth ratio, depth variance, depth abrupt change features, and spatial convexity features within the candidate depth region.

[0047] In this step, the depth anomalous region is the location and range of depth anomalous pixels in the image, used for spatial matching with candidate target regions; the depth region features are used to describe the depth state in the candidate depth regions and serve as input for subsequent intrusion detection.

[0048] In step S150 above, since the depth anomaly region and the candidate target region may be located in the depth image coordinate system and the RGB image coordinate system respectively, before performing spatial matching, the depth anomaly region can be mapped to the RGB image coordinate system, or the candidate target region can be mapped to the depth image coordinate system, based on the spatial mapping relationship established in step S110, so that the two are in the same image coordinate system. Subsequently, spatial matching is performed on the depth anomaly region and the candidate target region that are in the same image coordinate system. Spatial matching can be determined based on the overlap relationship between the two regions, and spatial consistency features can be obtained accordingly.

[0049] Spatial consistency features are used to describe whether candidate targets detected in an RGB image and spatial anomalies detected in a depth image are located at corresponding physical locations, and whether their regional ranges have a corresponding spatial correspondence.

[0050] Spatial consistency features are used to quantify the degree of correspondence between RGB position evidence and depth anomaly position evidence for the same candidate target. A high degree of overlap between the candidate target region and the mapped depth anomaly region indicates that they point to the same or similar physical location in the platform door gap; a low degree of overlap occurs when the candidate target region is mainly formed by reflected or transmitted images and no depth anomaly appears at the corresponding location, or when the depth anomaly is only local depth noise outside the candidate target region.

[0051] In step S160 above, for each candidate target region, a fusion detection is performed based on the image detection result, depth region features, and spatial consistency features corresponding to that candidate target region. The image detection result provides the image semantics of the candidate target and the reliability of the image detection; the depth region features provide the distance distribution and spatial occupancy information of the candidate depth region; and the spatial consistency features provide the spatial correspondence information between the image detection result and the depth anomaly region.

[0052] During fusion detection, image detection results, depth region features, and spatial consistency features can be converted into representational data that can be used in the same detection process. Then, according to preset fusion rules, it is determined whether intruders exist in the candidate target region. The preset fusion rules make judgments based on these three types of information, requiring the image detection results to be verified by combining the depth information at the corresponding location with cross-image spatial correspondences.

[0053] For example, it can be compared whether the RGB detection confidence, depth region features, and spatial consistency features in the image detection results meet their respective threshold conditions; when all three meet their respective threshold conditions, it is determined that there is an intruder in the platform door gap area.

[0054] For example, the RGB detection confidence, depth region features and spatial consistency features can be weighted and fused to obtain a fusion result, which can be used to determine whether there are intruders in the gap area of ​​the platform door.

[0055] When the fusion detection result corresponding to a candidate target meets the intrusion detection condition, it is determined that the object represented by the candidate target has intruded into the gap area of ​​the platform door, and the corresponding candidate target area is taken as the position of the intrusion in the RGB image.

[0056] The platform door gap intrusion detection method provided in this application first obtains candidate target regions with image semantic information based on RGB images, and then determines candidate depth regions based on the candidate target regions, so that depth information processing is performed on the local space corresponding to the candidate targets. By obtaining depth anomaly regions and depth region features respectively, on the one hand, the depth anomaly position of the corresponding depth anomaly region used for spatial matching is retained, and on the other hand, the depth state information of the corresponding depth region features used for fusion detection is obtained.

[0057] Based on this, by spatially matching depth anomaly regions with candidate target regions, it can be determined whether candidate targets in RGB images have depth spatial information support at their corresponding locations. Subsequently, the presence of intruders is detected by combining image detection results, depth region features, and spatial consistency features, ensuring that the detection results do not solely depend on texture and color in the RGB image, nor solely on local depth changes in the depth image. In low light, reflective, occluded, or partially visible target conditions, depth region features and spatial consistency features can provide verification information related to actual spatial occupancy for target detection in RGB images; when depth changes in the depth image do not correspond to the candidate target location, the spatial matching results can characterize the degree of correspondence between the depth change and the candidate target.

[0058] In some embodiments, when performing spatial correspondence processing on RGB images and depth images, lens distortion correction can be performed separately first. Taking normalized image coordinates (x, y) as an example, let the square of r equal to the sum of the squares of x and y. , , The radial distortion coefficient is... , The tangential distortion coefficient represents the distorted coordinates. Equation 1 can be satisfied:

[0059]

[0060] The RGB image frames and depth frames can be reverse-corrected according to the distortion model shown in Equation 1, or a lookup correction can be performed using a pre-generated correction mapping table. For the corrected RGB image frames and depth frames, their image frame timestamps are recorded respectively. and depth frame timestamp When both satisfy Equation 2, the RGB image frame and the depth frame are determined as a set of synchronized frames:

[0061] In Equation 2, Δt represents the maximum allowable time deviation, which can range from 5ms to 100ms, for example, 33ms. The above synchronization can be achieved using hardware triggering, a unified clock, or nearest neighbor matching based on timestamps. For image frames exceeding this time deviation, the image frame can be discarded or a temporally adjacent image frame can be waited for, ensuring that the information used for subsequent spatial matching corresponds to a similar acquisition time.

[0062] Spatial mapping relationships can be determined by the intrinsic and extrinsic parameters of the RGB imaging unit and the depth sensing device, or by a calibration mapping table. Taking the projection of pixels in the depth map onto the RGB image coordinate system as an example, let a point in the depth map be... The depth value of this point is The intrinsic parameter matrices of the depth sensing device and the RGB imaging unit are respectively and The rotation matrix and translation vector from the depth coordinate system to the RGB coordinate system are respectively and Then the corresponding RGB image coordinates It can be determined according to formula 3:

[0063] In Equation 3, This represents homogeneous coordinate normalization and pixel projection operations.

[0064] Spatial mapping relationships can be used to map pixels or regions in a depth map to the RGB image coordinate system. Alternatively, they can be used through inverse mapping, mapping table indexing, or boundary point projection to map candidate target regions in an RGB image to the depth image coordinate system. Since RGB and depth images can have different spatial resolutions, during mapping, rounding, interpolation, and out-of-bounds cropping can be performed in the target coordinate system based on the projected boundary coordinates.

[0065] In some embodiments, such as Figure 2 As shown, before performing step S120, the method further includes: S111: Based on RGB and depth images, determine the platform door boundary, train door boundary, structural region, and depth abrupt change region; the structural region includes at least one of the door frame structural region and the car body structural region. S112: Determine the initial gap area based on the relative position between the platform door boundary and the train door boundary; S113: Perform morphological processing on the initial gap region to obtain the boundary constraint region; S114: Determine the depth constraint region based on the overlap between the boundary constraint region and the depth abrupt change region; S115: Exclude structural regions from the depth-constrained regions, and perform connected component filtering on the excluded regions to obtain the region of interest; wherein, the region of interest is used to perform target detection.

[0066] Specifically, before performing target detection based on RGB images, the region of interest (ROI) can be determined based on the occlusion state after the train stops. After the train stops, the car body, train doors, platform doors, and door frame structures occupy part of the image. The position and outline of the observable gap between the platform doors and the train doors also change with the stopping position and the type of train. Therefore, the ROI can be determined based on the platform door boundaries, train door boundaries, structural regions, and depth abrupt change regions in the current frame or multiple adjacent image frames, rather than being pre-fixed as a rectangular region in the entire image.

[0067] The platform door boundary set can be determined from RGB and depth images. Train door boundary assembly Door frame structural area Vehicle body structure area and deep mutation regions Platform door boundaries and train door boundaries can be obtained through one or more of the following methods: edge detection, straight line or curve fitting, segmentation model, and registration of a preset door template with the current image; depth abrupt change regions can be obtained based on the depth difference between adjacent pixels in the depth map or local depth gradients. Door frame structure regions and vehicle body structure regions are used to characterize areas occupied by fixed or quasi-fixed door structures that are not considered intrusion detection targets. In an optional implementation, glass reflection regions can also be determined. Image area of ​​people inside the carriage And in subsequent processing, it is treated as a non-detection area.

[0068] Based on the platform door boundary set Set off with the boundary of the train door The relative positions between them determine the initial gap area. In one implementation, This represents the x-coordinate of the inner boundary of the platform screen door at the y-axis. This represents the x-coordinate of the near-side boundary of the train door at the y-axis. and Let the effective longitudinal ranges of the platform door boundary and the train door boundary be represented respectively, then the initial gap area... It can be represented by Equation 4:

[0069] The initial gap region shown in Equation 4 is jointly defined by the boundaries of the two doors at the same vertical coordinate and the intersection of their effective longitudinal ranges. When the boundaries of the two doors are curves or segmented boundaries, the pixel set between them can be determined row by row along the longitudinal direction; when the image coordinate direction or installation angle is different, the region between the two boundaries can also be determined along the horizontal direction or the door normal after perspective correction.

[0070] For the initial gap area Morphological processing is performed to obtain the boundary-constrained region. Morphological processing can include at least one of dilation, erosion, closing, and opening operations, used to connect local fracture areas near boundaries, fill small holes, or correct gap boundaries according to a preset boundary allowance. In one implementation, For the scale and the preset boundary margin The corresponding structuring elements, dilation and erosion operations are respectively represented by the symbols... and This indicates that the boundary constraint region... It can be determined according to formula 5:

[0071] Preset boundary margin The value can range from 5 pixels to 80 pixels, for example, 20 pixels. The specific value can be set based on the resolution of the RGB image, the gate boundary recognition error, and the fluctuation range of the parking position. This margin is used to reserve a processing range for boundary recognition errors and does not mean that the boundary constraint area is fixed as a regular rectangle.

[0072] Furthermore, the boundary constraint area is determined. With deep mutation regions The overlapping portion between them is used as the depth constraint region. In one implementation, the depth-constrained region Determine according to formula 6:

[0073] Deep mutation region This can include pixel regions with valid depth values ​​and local depth changes meeting preset conditions. Therefore, Equation 6 uses the planar position defined by the RGB image gate boundary and the spatial boundary reflected by the depth image to define the subsequent detection range. If there are local invalid values ​​in the depth image, the invalid points can be marked first, and only the portion with valid depth data can be used to form depth constraints.

[0074] From the deep constraint region After excluding structural regions, the remaining regions are further filtered by connected component selection to obtain the regions of interest. The structural area includes at least the door frame structural area. and vehicle body structure area One of them; in the identified glass reflective area Image area of ​​people inside the carriage In that case, they can also be excluded. Let's assume... This represents the j-th connected component formed after excluding the corresponding region. If we denote the area threshold of the connected region, then one method for determining the region of interest is shown in Equation 7:

[0075] In Equation 7, Area represents the pixel area of ​​the connected region. It can range from 20 pixels to 2000 pixels, for example, 100 pixels. The area is smaller than... Connected components can be used as boundary segmentation noise or discrete depth noise and excluded from subsequent processing. This is composed of multiple preserved connected components. The region of interest can be non-rectangular or include multiple separate local regions, and can be updated according to the train's stopping position, the relative position of the doors, and the current depth distribution. In this embodiment, the region of interest is used to define the effective processing range for candidate target detection and candidate depth region determination.

[0076] In some embodiments, such as Figure 3 As shown, the method also includes: S131: Based on the region of interest, construct a region mask, and use the region mask to crop or filter the non-detection regions of the RGB image and the depth image to obtain the RGB detection image and the depth detection image. In step S120 above, target detection based on the RGB image includes: S132: Perform target detection based on the RGB detection image; In step S130 above, based on the spatial mapping relationship, the candidate depth region corresponding to each candidate target region is determined in the depth image, including: S133: Based on spatial mapping relationships, determine the candidate depth region corresponding to each candidate target region in the depth detection image.

[0077] Specifically, obtain the region of interest Then, a region mask can be generated based on the region of interest. A binary region mask is shown in Equation 8:

[0078] In Equation 8, pixels with a mask value of 1 belong to the region of interest. Pixels with a mask value of 0 do not belong to the region of interest. Region masks can be processed pixel-by-pixel with both RGB and depth images, or the two types of images can be cropped and filtered based on the outer boundary of the region of interest and the mask value to obtain RGB detection images and depth detection images, respectively. Therefore, the RGB detection image is used to perform object detection, and the depth detection image is used to determine the candidate depth regions corresponding to the candidate object regions. Regions with a mask value of 0, such as door frame structures, vehicle body structures, identified glass reflections, and images of people inside the vehicle, are not considered valid detection regions in either data processing path.

[0079] When the coordinate systems or resolutions of the RGB image and the depth image are different, an initial region mask can be obtained first in the coordinate system on which the generated mask is based. Then, based on the established spatial mapping relationship, the initial region mask is mapped to another coordinate system, thereby forming masks that match the RGB image and the depth image respectively. Alternatively, masks with corresponding region boundaries can be generated in the two coordinate systems separately. The above methods ensure that the physical detection range represented by the RGB detection image and the depth detection image remains corresponding, without requiring them to have the same pixel size.

[0080] When performing object detection in RGB detection images, a trained object detection model can be used. The object detection model can be a convolutional neural network object detection model based on single-stage or two-stage feature extraction, or it can be other image detection models that can output the location, category, and confidence score of candidate objects. Let the RGB detection image at time t be... The candidate target set is The relationship between the two can be expressed by Equation 9:

[0081] The i-th candidate target in the candidate target set It can be represented by Equation 10:

[0082] In Equation 10, This refers to the bounding box of the candidate target or the candidate target region output by the detection model. Candidate target categories, RGB detection confidence scores are applied. Candidate object categories can include passenger limbs, clothing, backpack straps, umbrellas, plastic bags, or other objects that may enter the platform screen door gap area. Confidence scores can be used to filter candidate objects and suppress duplicate bounding boxes, retaining only those located within the region of interest. Furthermore, the RGB detection confidence level reaches the candidate target confidence threshold. Candidate targets. The threshold can be from 0.10 to 0.80, for example, 0.30; the cross-union ratio threshold used for duplicate box suppression. The value can be between 0.30 and 0.80, for example, 0.50. The object detection model can be configured to output a set of candidate objects in a frame rate range of 10 fps to 60 fps, for example, running at 25 fps.

[0083] For each candidate target region Based on spatial mapping relationships, the boundary points or pixels within the region are mapped to the depth detection image, forming corresponding candidate depth regions. When the candidate target region is located in the RGB image coordinate system, this mapping step is a region mapping from the RGB coordinate system to the depth coordinate system; when it is necessary to spatially match the depth anomaly region with the candidate target region later, the depth anomaly region is mapped back to the RGB image coordinate system. Therefore, the candidate depth region It is the range of values ​​in the depth image corresponding to the candidate target region; the two do not necessarily have the same pixel coordinates or area.

[0084] In some embodiments, such as Figure 4 As shown, determining the depth anomaly region in step S140 above includes: S141: Select pixels that meet the preset abnormality conditions from the candidate depth region to obtain the initial abnormal pixel set; S142: Perform connectivity analysis on the initial abnormal pixel set, extract connected pixel groups that meet the preset connectivity conditions, and form a deep abnormal region.

[0085] The preset abnormal conditions include at least one of the following: The depth mutation condition is used to filter pixels in the candidate depth region whose depth value is less than a preset proximity threshold when the difference between the minimum effective depth of the candidate depth region and the neighborhood background depth of the candidate depth region is greater than a preset depth mutation threshold. Distance anomaly condition is used to filter pixels in the candidate depth region whose depth gradient is greater than a preset gradient threshold; wherein, the depth gradient is the rate of change of depth of the boundary pixels of the candidate depth region along the edge normal direction; The spatial protrusion condition is used to calculate the protrusion height of the depth of each pixel in the candidate depth region relative to the depth of the neighboring background, and to filter pixels whose protrusion height is greater than a preset protrusion threshold. The protrusion height corresponding to any pixel is the difference between the depth value of the neighboring background and the depth value of the pixel. The neighborhood background depth is determined based on the depth statistics of the effective depth points within the annular band region, which is the region formed by extending a preset pixel distance outward from the boundary of the candidate depth region.

[0086] Specifically, the depth value in the depth image represents the distance from the measured location to the depth sensing device; under the same depth coordinate convention, the closer the measured location is to the depth sensing device, the smaller its corresponding depth value. For candidate depth regions... First, we can determine the set of valid depth points with valid depth values. The minimum effective depth is determined by this set. .

[0087] Valid depth values ​​can be those that are within the measurement range of the depth sensing device and meet the device's validity flag or preset data validity conditions. Zero values, null values, values ​​outside the measurement range, and values ​​marked as low confidence by the device can be considered invalid depth values. Alternatively, depth values ​​can be selected from candidate depth regions. Extend the boundary outward by a preset pixel distance, and take the extended region and the candidate depth region. The difference between the two points is used as the annular zone region, and the average or median depth of the effective depth points within this annular zone region is determined as the neighborhood background depth. .

[0088] Candidate depth regions can be selected based on at least one of the following conditions: depth mutation condition, distance anomaly condition, and spatial convexity condition. Pixels with valid depth values ​​are selected. For depth abrupt changes, the depth of the neighboring background is considered. With minimum effective depth When the difference between the two values ​​exceeds the preset depth mutation threshold, the depth value is compared with the minimum effective depth. Pixels whose difference between the two depth samples is less than a preset threshold are selected. For distance anomaly conditions, depth samples at preset pixel intervals can be obtained along the edge normal direction of the candidate depth region boundary for pixels. The depth change rate is determined by the ratio of the absolute value of the difference between the two depth samples to the preset pixel distance. This depth change rate is used as the depth gradient, and pixels with a depth gradient greater than a preset gradient threshold are selected. For spatial protrusion conditions, the protrusion height of any pixel is equal to the neighborhood background depth. The difference between the depth value and the depth value of the pixel; since a smaller depth value corresponds to a position closer to the depth sensing device, pixels with a positive protrusion height and greater than a preset protrusion threshold indicate that they protrude towards the depth sensing device relative to the neighboring background. Pixels that satisfy at least one of the above preset abnormal conditions are grouped into an initial abnormal pixel set.

[0089] When performing connectivity analysis on the initial set of anomalous pixels, interconnected anomalous pixels can be divided into corresponding connected pixel groups according to the four-neighbor connectivity rule or the eight-neighbor connectivity rule. Preset connectivity conditions may include the number of pixels in a connected pixel group reaching a preset threshold or the area of ​​a connected pixel group reaching a preset area threshold. Connected pixel groups that satisfy the preset connectivity conditions are extracted, and the region covered by one connected pixel group or the union of the regions covered by multiple connected pixel groups is determined as the depth anomalous region. Discrete pixels that do not meet the preset connectivity conditions or small-area connected pixel groups are not used to form depth anomalous regions.

[0090] In this embodiment, an initial set of anomalous pixels is obtained by filtering pixels that meet preset anomaly conditions from candidate depth regions. Connectivity analysis is then performed on this initial set of anomalous pixels to extract connected pixel groups that meet the preset connectivity conditions, thus forming the final depth anomaly region. This filtering process enables precise localization of anomalies within the candidate depth region. Simultaneously, the preset anomaly conditions include at least one of depth abrupt change conditions, edge gradient conditions, and spatial protrusion conditions. This allows for the detection of anomalous depth from multiple physical dimensions, such as the difference between depth values ​​and the background, the rate of depth change at edges, and the degree of pixel protrusion relative to the background. This ensures the effective extraction of depth anomaly regions, thereby improving detection accuracy.

[0091] In some embodiments, such as Figure 5 As shown, the extraction of depth region features in step S140 above includes: S145: Determine the set of effective depth points within the candidate depth region, and determine the effective depth ratio based on the number of effective depth points in the set of effective depth points and the total number of pixels within the candidate depth region; S146: Based on the depth values ​​in the set of effective depth points, determine the depth variance and minimum effective depth of the candidate depth region; determine the neighborhood background depth based on the depth values ​​around the candidate depth region, and determine the depth difference between the neighborhood background depth and the minimum effective depth; S147: Determine the depth verification score based on at least one of depth difference, effective depth ratio and depth variance, and use the depth verification score as a depth region feature. The depth verification score is positively correlated with the depth difference and effective depth ratio, and negatively correlated with the depth variance.

[0092] Specifically, set Set of effective depth points The number of effective depth points in the data. If the number of pixels in the candidate depth region is 1, then the effective depth ratio is 1. It can be determined according to formula 11:

[0093] when When it equals 0, it can be Set to 0, and mark the corresponding candidate target as lacking effective depth support. Average depth of the candidate depth region. It can be determined according to Equation 12:

[0094] The depth variance of the candidate depth region can be determined according to Equation 13:

[0095] In Equations 12 and 13, Represents coordinates in a depth image The depth value at that location. When When the value is 0, the average depth and depth variance are not calculated; instead, the corresponding candidate target is treated as a candidate target lacking depth support. When the value is greater than 0, the depth variance is used to characterize the dispersion of the effective depth values ​​in the candidate depth region. A smaller depth variance indicates a lower dispersion of the effective depth values, corresponding to higher depth stability within the region.

[0096] Neighborhood background depth With minimum effective depth Depth difference between It can be determined according to formula 14:

[0097] Depth region features may include at least one of the following: minimum effective depth, average depth, effective depth ratio, depth variance, depth difference, depth abrupt change feature, and spatial convexity feature. In one implementation, the depth difference, effective depth ratio, and depth variance are each normalized and combined to form a depth verification score. Specifically, it can be determined according to Formula 15:

[0098] In Equation 15, This is the threshold for deep mutation. The effective depth ratio threshold, ε is the depth variance threshold, and ε is a positive number to prevent the denominator from being zero. It can be from 0.02 m to 0.30 m, for example, 0.08 m; It can be between 0.10 and 0.90, for example, 0.30; The value can be from 0.001 m² to 0.20 m², for example, 0.03 m²; ε can be from 1e-8 to 1e-3, for example, 1e-6. According to Equation 15, the depth verification score is... Difference with depth and effective depth ratio Positively correlated with, and negatively correlated with, depth variance; the upper limit of each normalized component is 1, allowing depth evidence of different dimensions to be combined within a unified numerical range. Equation 15 is only one combination form; different weights can also be set for each component according to the noise characteristics of the depth sensing device used, or at least one of the above characteristics can be selected to determine the depth verification score.

[0099] In one optional implementation, when the depth difference reaches a depth mutation threshold, the effective depth ratio reaches an effective depth ratio threshold, and the depth variance is not greater than a depth variance threshold, a spatial occupancy marker corresponding to the candidate depth region can be generated. This spatial occupancy marker is used as part of the depth verification result and can participate in depth verification score determination, candidate target pre-screening, or subsequent fusion discrimination, rather than as a separate output independent of subsequent processing. For example, when the spatial occupancy marker indicates that stable occupancy has not been detected, the corresponding depth verification score can be reduced, or the candidate target can be submitted for temporal stability verification.

[0100] In some embodiments, such as Figure 6 As shown, step S150 above further includes: S151: Based on spatial mapping relationships, map depth anomaly regions to the coordinate system of the RGB image; S152: Determine the intersection and union regions between the mapped depth anomaly regions and the corresponding candidate target regions; S153: Determine the spatial consistency characteristics based on the ratio of the area of ​​the intersection region to the area of ​​the union region.

[0101] Specifically, for deep anomaly regions With the corresponding candidate target region Perform spatial matching; if there are depth anomaly regions... If the current location is in the depth image coordinate system, then based on the established spatial mapping relationship, the depth anomaly region will be... The pixels within the boundary or region are mapped to the RGB image coordinate system. In this embodiment, the mapped depth anomaly region is still denoted as... .Sure and The intersection and union regions between them are defined, and the ratio of the area of ​​the intersection region to the area of ​​the union region is used as a spatial consistency feature. One calculation method is shown in Equation 16:

[0102] In Equation 16, The intersection region represents the overlapping area indicated by the two types of data, while the union region represents the overall area covered by the two regions. When... and When the union of the sets is empty, Set to 0. Spatial consistency feature. It is the matching result after the coordinate correspondence between the depth anomaly region and the RGB candidate target region, which is different from the spatial mapping relationship used for coordinate transformation, and also different from the effective depth ratio of the candidate depth region itself. In addition to the region intersection-union ratio, spatial consistency features can also be formed by combining the centroid distance, boundary distance or inclusion ratio of the two regions; Equation 16 gives one of the directly implementable methods.

[0103] In some embodiments, such as Figure 7 As shown, step S160 above further includes: S161: The RGB detection confidence, depth region features and spatial consistency features are weighted and fused according to their respective fusion weights to obtain the intrusion risk score; S162: If the intrusion risk score reaches the risk threshold, determine that there is an intruder in the candidate target area.

[0104] Specifically, the image detection results include RGB detection confidence. Deep region features can be achieved using depth verification scores. Spatial consistency features can be represented by spatial consistency scores. The intrusion risk score of the candidate target is obtained by weighting and fusing the data according to their respective fusion weights. One type of fusion is shown in Equation 17:

[0105] In Equation 17, , , These are the non-negative fusion weights corresponding to the RGB detection confidence score, depth verification score, and spatial consistency score, respectively, with a sum of 1. The baseline fusion weights can be set to, for example, 0.40, 0.40, and 0.20 respectively. The intrusion risk score simultaneously reflects the semantic detection evidence of the candidate target in the RGB image, the geometric occupancy evidence in the candidate depth region, and the degree of spatial consistency between the two types of evidence. When the intrusion risk score... Reaching the preset risk threshold When an intruder is identified as being present in the candidate target region, the intrusion risk score is determined to be below the risk threshold. If the candidate target is found to contain an intruder, the system will either not output a determination result indicating that the candidate target contains an intruder, or the candidate target will be retained for verification in subsequent image frames. It can be between 0.50 and 0.95, for example, 0.70.

[0106] In some embodiments, such as Figure 8 As shown, the determination of fusion weights used in step S161 above includes: S1611: Determine RGB quality indicators based on the brightness characteristics in RGB images; S1612: Determine the depth quality index based on the effective depth ratio and depth variance of the candidate depth region; S1613: Based on the threshold conditions satisfied by the RGB quality index and the depth quality index, selectively adjust the benchmark weight corresponding to at least one of the RGB detection confidence and the depth region features. S1614: Normalize the adjusted weights and the baseline weights corresponding to the spatial consistency features to obtain the fusion weights corresponding to the RGB detection confidence, depth region features and spatial consistency features respectively.

[0107] Specifically, the fusion weights can be adjusted based on the quality of the RGB image and the data quality of the candidate depth regions. The brightness features of the RGB image can be the region of interest. Luminance statistics for an internal RGB image. Let... This represents the number of pixels in the RGB image at time t within the region of interest Rt. The brightness channel value at the location, the average brightness of the area. It can be determined according to formula 18:

[0108] When the region of interest is empty, the RGB quality index of the previous valid image frame can be used, or the current RGB quality index can be set to a preset low quality value. Let the dark light brightness normalization threshold be... Then the RGB quality index It can be determined according to Equation 19:

[0109] in, It can support 30 to 180 gray levels, for example, 80 gray levels; The value ranges from 0 to 1. The closer the average brightness of the area is to or exceeds... , The closer to 1, the better. The brightness statistics are not limited to the average brightness. They can also be the brightness median, preset quantiles, the proportion of low-brightness pixels, or a value obtained by combining multiple brightness statistics. However, they should be able to characterize the usability of the RGB image in the current region of interest.

[0110] Depth Quality Indicators Based on candidate depth regions Effective depth ratio and depth variance The value is determined to be between 0 and 1, and the depth variance is used to characterize depth stability. In some examples, the effective depth ratio can be used. Compared with the effective depth ratio threshold The ratio is limited to the range of 0 to 1 to obtain the effective depth proportion mass component; the depth variance threshold is set. With depth variance The ratio of the sum of ² and positive numbers ε is limited to the range of 0 to 1 to obtain the depth variance quality component; then, the average of the two quality components or the weighted sum according to preset coefficients is calculated, and the resulting value is limited to the range of 0 to 1 to obtain the depth quality index. When there are no valid depth points in the candidate depth region, the depth quality index can be... Set it to 0. Therefore, the depth quality metric... It is positively correlated with the effective depth ratio and negatively correlated with the depth variance, and is used to evaluate the effective sampling ratio and dispersion of the raw depth data that forms the depth verification score.

[0111] You can set the low-light detection threshold separately. and depth of trust threshold Based on the threshold conditions satisfied by the RGB quality index and the depth quality index, the baseline weight corresponding to at least one of the RGB detection confidence score and the depth verification score is selectively adjusted. Less than and Greater than When the current RGB image is in a dark state and the depth data meets the confidence condition, the adjusted weights corresponding to the RGB detection confidence can be adjusted accordingly. Positively correlated, and the adjusted weights corresponding to the depth verification scores are related to... Positive correlation; one method for determining unnormalized weights is shown in Equation 20:

[0112] In Equation 20, , , These are the baseline weights for RGB detection confidence, depth verification score, and spatial consistency score, respectively. , and These are the corresponding unnormalized weights. In low-light conditions where the depth data meets the reliability criteria... The reduction of The corresponding decrease, factor and Joint regulation , Keep as Subsequently, the normalization factor is determined according to Equation 21. :

[0113] The three unnormalized weights are normalized using a normalization factor to obtain the fusion weights used in Equation 17, as shown in Equation 22:

[0114] After processing by Equation 22, , and All three are non-negative and their sum is 1. It can be between 0.20 and 0.60, for example, a value of 0.40; It can be between 0.30 and 0.90, for example, a value of 0.60.

[0115] Besides the combination of low light and depth data meeting the reliability criteria, other combinations of threshold conditions can also be processed accordingly. When Not less than and Greater than At this time, three benchmark weights can be used, or the RGB detection confidence weight and depth verification score weight can be adjusted only within a preset range according to the corresponding quality indicators; when Not less than and Not greater than At that time, the unnormalized weight corresponding to the deep verification score can be reduced, while maintaining or following the principle of... Correct the unnormalized weights corresponding to the RGB detection confidence; when Less than and Not greater than At this time, the unnormalized weights corresponding to RGB detection confidence and depth verification score can be reduced simultaneously, while retaining the baseline weight of spatial consistency score, or the current frame result can be treated as a low-quality result and submitted to consecutive frames for verification. After selectively adjusting either combination, the adjusted weights are normalized together with the baseline weight corresponding to the spatial consistency score. The above threshold combination processing follows the following relationship: the relative weight of RGB detection confidence is positively correlated with the RGB quality index, the relative weight of depth verification score is positively correlated with the depth quality index, and the normalization process keeps the three weights at the same fusion scale.

[0116] In some embodiments, the method further includes: performing temporal stability verification on the intrusion risk scores corresponding to the same candidate target in multiple consecutive image frames; counting the proportion of target image frames whose intrusion risk scores reach a risk threshold in a preset number of consecutive image frames; and outputting a detection result indicating the presence of an intruder in the candidate target when the proportion reaches a stability threshold.

[0117] Specifically, before outputting the intrusion detection result based on the intrusion risk score, the temporal stability of the intrusion risk scores corresponding to the same candidate target in multiple consecutive image frames can be verified. Candidate targets in consecutive frames can be associated as the same candidate target based on at least one of the following: candidate target category, spatial overlap of candidate target regions in adjacent frames, and positional changes. In cases where short-term occlusion causes temporary loss of candidate bounding boxes, the target identifier can be retained for a preset number of frames, and statistics can continue to be calculated when the target reappears and meets the association conditions.

[0118] Suppose that the same candidate target has risk scores in L consecutive frames up to time t. Statistics show that some of them reached the risk threshold. The number of target image frames is calculated, and the proportion of this number to L frames is determined. When the proportion reaches the stability threshold η, the detection result indicating the presence of an intruder in the candidate target is output; one method of temporal stability determination is shown in Equation 23:

[0119] In Equation 23, The indicator function is η; the number of consecutive frames L can be 3 to 30, for example 5; the stability threshold η can be 0.50 to 1.00, for example 0.60. When Equation 23 is true, a stable risk marker can be generated, and the intruder presence status can be output accordingly; when Equation 23 is false, no definite intruder presence status is output, or the statistical proportion within the sliding window is updated by receiving subsequent image frames. Thus, the output of the temporal stability verification is related to the final detection result, rather than simply generating intermediate markers that are not subsequently used.

[0120] Intruder detection results can include the intruder's presence status, location, scale, category, and risk level. Location and scale can be determined jointly by the candidate target area and the spatially mapped depth anomaly area. The category can be taken from the RGB target detection results, and the risk level can be determined based on the preset score range of the intrusion risk score. When a candidate target is identified as an intruder, an alarm message containing an alarm signal, risk level data, and target location data can be generated and sent to the platform screen door control system or the station's integrated monitoring system. Whether to directly trigger door control operations can be determined by the external system according to its security control rules. The detection method in this embodiment provides the intruder detection results and corresponding data to the external system.

[0121] Next, the detection method provided in the embodiments of this application will be further explained with reference to specific examples.

[0122] In one specific embodiment, the RGB imaging unit and the depth sensing device jointly cover the gap area between one platform screen door and the train door. The RGB imaging unit outputs a 1920×1080 resolution RGB image, and the depth sensing device outputs a 640×480 resolution depth image. Both are acquired at a frame rate of 25fps. The depth sensing device uses a ToF depth camera and is calibrated with the RGB imaging unit using intrinsic and extrinsic parameters. The processing device performs lens distortion correction on the two images and performs synchronization frame matching with a maximum time deviation of 33ms. For each set of synchronization frames, a spatial mapping relationship between the RGB image coordinates and the depth image coordinates is established based on the calibration parameters. The depth sensing device can also be replaced with a structured light, binocular or multi-view, or laser depth sensor, or it can be integrated with the RGB imaging unit.

[0123] After the train stops, the processing equipment obtains the platform door boundary, train door boundary, door frame structure region, car body structure region, and depth abrupt change region from the synchronized RGB and depth images. It can further identify glass reflection areas and passenger image regions inside the carriages. The processing equipment uses the area between the two door boundaries as the initial gap region, performs morphological processing with a 20-pixel boundary margin, overlaps the resulting boundary-constrained region with the effective portion of the depth abrupt change region, and excludes the aforementioned structures and non-detection areas. Then, a 100-pixel connected component area threshold is used to filter the remaining connected components, obtaining the region of interest determined according to the current stopping state. Based on this region of interest, region masks adapted to the coordinate systems of the RGB and depth images are generated, thereby obtaining the RGB detection image and the depth detection image.

[0124] The processing device runs a target detection model on the RGB detection image, using a candidate target confidence threshold of 0.30 and a duplicate box suppression threshold of 0.50 to output candidate target regions, candidate target categories, and RGB detection confidence scores. For each candidate target region, the processing device determines candidate depth regions in the depth detection image based on spatial mapping relationships, obtains a set of effective depth points, calculates the minimum effective depth, average depth, effective depth ratio, and depth variance, and determines the neighborhood background depth based on the average or median depth of the effective depth points within the annular zone formed by the outward expansion of the candidate depth region boundary. The depth abrupt change threshold is set to 0.08 m, the effective depth ratio threshold to 0.30, and the depth variance threshold to 0.03 m. 2 In Equation 15 above, the positive number ε is taken as 1 × 10. -6 The processing equipment determines the depth verification score based on the depth difference, effective depth ratio, and depth variance. It then selects an initial set of anomalous pixels according to at least one of the following conditions: depth mutation, distance anomaly, and spatial protrusion. The initial set of anomalous pixels is then subjected to connectivity analysis to form a depth anomaly region by connecting pixel groups that meet the preset connectivity conditions.

[0125] The processing device maps depth anomaly regions to the RGB image coordinate system, calculates their intersection-over-union (IoU) with the corresponding candidate target regions, and obtains the spatial consistency score. The baseline weights for RGB detection confidence, depth verification score, and spatial consistency score are 0.40, 0.40, and 0.20, respectively, with a risk threshold of 0.70. The dark light brightness normalization threshold (TB) is set to 80 gray levels, and the dark light judgment threshold... Set to 0.40, the depth confidence threshold. The value is set to 0.60. The processing device determines the RGB quality index based on the average brightness of the RGB image within the region of interest, and determines the depth quality index based on the effective depth ratio and depth variance of the candidate depth region. Then, it adjusts and normalizes the fusion weights according to the threshold conditions met. For the same candidate target, the processing device counts the proportion of image frames with an intrusion risk score of 0.70 in 5 consecutive frames. When this proportion reaches 0.60, it outputs a detection result indicating the presence of an intruder and sends the location, size, type, risk level, and alarm signal of the intruder to the platform screen door control system or the station integrated monitoring system.

[0126] In multi-gate concurrent deployment, a concurrent scheduling module can be set up to cache, sort, and distribute synchronous frames output by multiple RGB imaging units and depth sensing devices. The concurrent scheduling module can include a low-level data acquisition unit, a cross-environment data exchange unit, and an upper-level inference unit. The low-level data acquisition unit receives multiple image frames output by the camera software development kit or video acquisition driver in the first operating environment; the cross-environment data exchange unit transmits synchronous frames to the second operating environment through shared memory, a circular buffer, or a message queue; the upper-level inference unit executes the target detection model in the second operating environment and transmits the candidate target detection results to the cross-domain fusion detection processing. The first operating environment is, for example, a C language environment, and the second operating environment is, for example, a Python runtime environment. Alternatively, C++, Rust, Go, ONNX runtimes, or embedded inference environments can be used to complete the corresponding functions. Let Ng represent the number of gates processed concurrently, for example, 12; let Nc represent the number of imaging devices corresponding to each gate, which can be set according to the number of RGB imaging units and depth sensing devices configured for that gate. For a scenario with 12 platform gates on one side, the RGB-depth synchronization frame corresponding to each platform gate can be treated as an independent detection task. This task can be implemented on a multi-core CPU platform or a domestic CPU and operating system platform using thread binding, batch inference, or pipeline scheduling. Each gate detection task uses its own spatial mapping parameters, region of interest state, and continuous frame statistical state, ensuring that the data from different platform gates remain consistent during concurrent processing.

[0127] The various embodiments or implementation methods described in this specification are presented in a progressive manner. Each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other.

[0128] It is worth noting that, without contradiction, different embodiments can be arbitrarily combined. For example, some or all of the steps of different embodiments can be arbitrarily combined, and one embodiment can be arbitrarily combined with the optional implementations of other embodiments.

[0129] Figure 9 This is a schematic diagram of a platform screen door gap intrusion detection device according to one embodiment. Figure 9 As shown, the platform screen door gap intrusion detection device 100 includes: The image acquisition and mapping module 101 is configured to acquire RGB and depth images of the platform door gap area and establish a spatial mapping relationship between the RGB and depth images.

[0130] The image detection module 102 is configured to perform target detection based on RGB images, and obtain at least one candidate target region and the image detection result corresponding to each candidate target region.

[0131] The depth region determination module 103 is configured to determine the candidate depth region corresponding to each candidate target region in the depth image based on spatial mapping relationships.

[0132] The depth information extraction module 104 is configured to identify depth anomaly regions within the candidate depth region and extract depth region features from the candidate depth region.

[0133] The spatial matching module 105 is configured to perform spatial matching between depth anomaly regions and corresponding candidate target regions to obtain spatial consistency features.

[0134] The fusion detection module 106 is configured to detect whether there are intruders in the gap area of ​​the platform door based on image detection results, depth region features and spatial consistency features.

[0135] In some embodiments, the image acquisition and mapping module 101 is further configured to: Based on RGB and depth images, determine the platform door boundary, train door boundary, structural region, and depth abrupt change region; the structural region includes at least one of the door frame structural region and the car body structural region. The initial gap area is determined based on the relative position between the platform door boundary and the train door boundary; Morphological processing is performed on the initial gap region to obtain the boundary constraint region; The depth constraint region is determined based on the overlap between the boundary constraint region and the depth abrupt change region; Structural regions are excluded from the depth-constrained regions, and connected component filtering is performed on the excluded regions to obtain regions of interest; these regions of interest are used to perform object detection.

[0136] In some embodiments, the image acquisition and mapping module 101 is further configured to: Based on the region of interest, a region mask is constructed, and the region mask is used to crop or filter the non-detection regions of the RGB image and the depth image to obtain the RGB detection image and the depth detection image. Object detection based on RGB images includes: Perform target detection based on RGB detected images; Based on spatial mapping relationships, candidate depth regions corresponding to each candidate target region are determined in the depth image, including: Based on spatial mapping relationships, candidate depth regions corresponding to each candidate target region are determined in the depth detection image.

[0137] In some embodiments, the depth information extraction module 104 is further configured to: Determine the set of valid depth points with valid depth values ​​from the candidate depth region; The minimum effective depth of the candidate depth region is determined based on the set of effective depth points, and the neighborhood background depth is determined based on the effective depth points located around the candidate depth region. Depth anomalous pixels are determined based on the depth difference between the minimum effective depth and the neighborhood background depth, and at least one of distance anomalies and spatial protrusions in the candidate depth region. Depth anomalous regions are formed based on depth anomalous pixels.

[0138] In some embodiments, the depth information extraction module 104 is further configured to: Pixels that meet the preset abnormality conditions are selected from the candidate depth regions to obtain an initial set of abnormal pixels; The initial abnormal pixel set is subjected to connectivity analysis to extract connected pixel groups that meet preset connectivity conditions, forming the depth abnormal region; The preset abnormal conditions include at least one of the following: The depth mutation condition is used to filter pixels in the candidate depth region whose depth value is less than a preset proximity threshold when the difference between the minimum effective depth of the candidate depth region and the neighborhood background depth of the candidate depth region is greater than a preset depth mutation threshold. Distance anomaly condition is used to filter pixels in the candidate depth region whose depth gradient is greater than a preset gradient threshold; wherein, the depth gradient is the rate of change of depth of the boundary pixels of the candidate depth region along the edge normal direction; The spatial protrusion condition is used to calculate the protrusion height of the depth of each pixel in the candidate depth region relative to the depth of the neighboring background, and to filter pixels whose protrusion height is greater than a preset protrusion threshold. The protrusion height corresponding to any pixel is the difference between the depth value of the neighboring background and the depth value of the pixel. The neighborhood background depth is determined based on the depth statistics of the effective depth points within the annular band region, which is the region formed by extending a preset pixel distance outward from the boundary of the candidate depth region.

[0139] In some embodiments, the space matching module 105 is further configured to: Based on spatial mapping relationships, depth anomaly regions are mapped to the coordinate system of the RGB image; Determine the intersection and union regions between the mapped depth anomaly regions and the corresponding candidate target regions; Spatial consistency characteristics are determined by the ratio of the area of ​​the intersection region to the area of ​​the union region.

[0140] In some embodiments, the fusion detection module 106 is further configured to: The RGB detection confidence, depth region features, and spatial consistency features are weighted and fused according to their respective fusion weights to obtain the intrusion risk score. If the intrusion risk score reaches the risk threshold, an intruder is identified in the candidate target area.

[0141] In some embodiments, the fusion detection module 106 is further configured to: Determine RGB quality indicators based on the brightness characteristics in RGB images; Determine the depth quality index based on the effective depth ratio and depth variance of the candidate depth region; Based on the threshold conditions satisfied by the RGB quality index and the depth quality index, the baseline weights corresponding to at least one of the RGB detection confidence and the depth region features are selectively adjusted. The adjusted weights and the baseline weights corresponding to the spatial consistency features are normalized to obtain the fusion weights corresponding to the RGB detection confidence, depth region features and spatial consistency features respectively.

[0142] In some embodiments, the apparatus further includes: The stability verification module is configured to perform temporal stability verification on the intrusion risk score corresponding to the same candidate target in multiple consecutive image frames; to count the proportion of target image frames whose intrusion risk score reaches the risk threshold in a preset number of consecutive image frames; and to output the detection result indicating the presence of an intruder in the candidate target when the proportion reaches the stability threshold.

[0143] The platform screen door gap intrusion detection device provided in this application belongs to the same concept as the platform screen door gap intrusion detection method provided in the above-mentioned embodiments of this application. It can execute the platform screen door gap intrusion detection method provided in any of the above-mentioned embodiments of this application, and has the corresponding functional modules and beneficial effects of executing the platform screen door gap intrusion detection method. Technical details not described in detail in this embodiment can be found in the specific processing content of the platform screen door gap intrusion detection method provided in the above-mentioned embodiments of this application, and will not be repeated here.

[0144] It should be understood that the various modules in the platform screen door gap intrusion detection device can be implemented in the form of a processor calling software, or in the form of a hardware circuit. The functions of some or all modules can be realized through the design of the hardware circuit, which can be understood as one or more processors.

[0145] This application also provides an electronic device, including a processor, a memory, and an executable program stored in the memory and capable of being run by the processor. When the processor runs the executable program, it performs the steps of the platform door gap intrusion detection method provided in any of the foregoing embodiments.

[0146] This application also provides a computer-readable storage medium storing an executable program thereon, which, when executed by a processor, implements the steps of the platform door gap intrusion detection method provided in any of the foregoing embodiments.

[0147] For ease of understanding, the following focuses on explaining the terminology used in this embodiment: In this application embodiment, the processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction read and execute capabilities, such as a Central Processing Unit (CPU), a microprocessor, a Graphics Processing Unit (GPU) (which can be understood as a type of microprocessor), or a Digital Signal Processor (DSP). In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. The logical relationships of the aforementioned hardware circuits are fixed or reconfigurable. For example, the processor is a hardware circuit implemented using an Application-Specific Integrated Circuit (ASIC) or a Programmable Logic Device (PLD), such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units or modules. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a Neural Network Processing Unit (NPU), a Tensor Processing Unit (TPU), a Deep Learning Processing Unit (DPU), etc.

[0148] The computer-readable storage medium provided in this embodiment can execute the platform door gap intrusion detection method of the above embodiment. Its implementation principle and technical effect are similar to those of the above embodiment, and will not be repeated here.

[0149] The aforementioned computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0150] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in application-specific integrated circuits (ASICs). Alternatively, the processor and the readable storage medium can exist as discrete components in an electronic device or a host device.

[0151] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0152] The various embodiments or implementation methods described in this specification are presented in a progressive manner. Each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other.

[0153] In the description of this specification, references to "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0154] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for detecting intruders through platform screen door gaps, characterized in that, The method includes: Acquire RGB and depth images of the platform door gap area, and establish a spatial mapping relationship between the RGB images and the depth images; Target detection is performed based on the RGB image to obtain at least one candidate target region and the image detection result corresponding to each candidate target region; Based on the spatial mapping relationship, a candidate depth region corresponding to each candidate target region is determined in the depth image; Depth anomaly regions are identified within the candidate depth regions, and depth region features of the candidate depth regions are extracted. Spatial matching is performed between the depth anomaly region and the corresponding candidate target region to obtain spatial consistency features; Based on the image detection results, the depth region features, and the spatial consistency features, the presence of intruders within the gap area of ​​the platform door is detected.

2. The method according to claim 1, characterized in that, Before performing target detection based on the RGB image, the method further includes: Based on the RGB image and the depth image, the platform door boundary, train door boundary, structural region, and depth abrupt change region are determined; the structural region includes at least one of the door frame structural region and the car body structural region. The initial gap area is determined based on the relative position between the platform door boundary and the train door boundary; The initial gap region is morphologically processed to obtain the boundary constraint region; The depth constraint region is determined based on the overlap between the boundary constraint region and the depth abrupt change region; The structural region is excluded from the depth-constrained region, and the region after excluding the structural region is filtered for connected components to obtain the region of interest; wherein the region of interest is used to perform the target detection.

3. The method according to claim 2, characterized in that, The method further includes: Based on the region of interest, a region mask is constructed, and the region mask is used to crop or filter the non-detection regions of the RGB image and the depth image to obtain the RGB detection image and the depth detection image. The target detection based on the RGB image includes: The target detection is performed based on the RGB detection image; The step of determining the candidate depth region corresponding to each candidate target region in the depth image based on the spatial mapping relationship includes: Based on the spatial mapping relationship, a candidate depth region corresponding to each candidate target region is determined in the depth detection image.

4. The method according to any one of claims 1 to 3, characterized in that, The step of determining the depth anomaly region within the candidate depth region includes: Pixels that meet the preset abnormality conditions are selected from the candidate depth regions to obtain an initial set of abnormal pixels; The initial abnormal pixel set is subjected to connectivity analysis to extract connected pixel groups that meet preset connectivity conditions, forming the depth abnormal region; The preset abnormal conditions include at least one of the following: The depth mutation condition is used to filter pixels in the candidate depth region whose depth value is less than a preset proximity threshold when the difference between the minimum effective depth of the candidate depth region and the neighborhood background depth of the candidate depth region is greater than a preset depth mutation threshold. Distance anomaly condition is used to filter pixels in the candidate depth region whose depth gradient is greater than a preset gradient threshold; wherein, the depth gradient is the rate of change of depth of the boundary pixels of the candidate depth region along the edge normal direction; The spatial protrusion condition is used to calculate the protrusion height of the depth of each pixel in the candidate depth region relative to the depth of the neighboring background, and to filter pixels whose protrusion height is greater than a preset protrusion threshold. The protrusion height corresponding to any pixel is the difference between the depth value of the neighboring background and the depth value of the pixel. The neighborhood background depth is determined based on the depth statistics of the effective depth points within the annular band region, which is the region formed by extending a preset pixel distance outward from the boundary of the candidate depth region.

5. The method according to claim 1, characterized in that, The extraction of depth region features from the candidate depth regions includes: A set of effective depth points is determined within the candidate depth region, and the effective depth ratio is determined based on the number of effective depth points in the set of effective depth points and the total number of pixels within the candidate depth region. Based on the depth values ​​in the set of effective depth points, determine the depth variance and minimum effective depth of the candidate depth region; determine the neighborhood background depth based on the depth values ​​surrounding the candidate depth region, and determine the depth difference between the neighborhood background depth and the minimum effective depth; A depth verification score is determined based on at least one of the depth difference, the effective depth ratio, and the depth variance, and the depth verification score is used as the depth region feature. The depth verification score is positively correlated with the depth difference and the effective depth ratio, and negatively correlated with the depth variance.

6. The method according to claim 1, characterized in that, The step of spatially matching the depth anomaly region with the corresponding candidate target region to obtain spatial consistency features includes: Based on the spatial mapping relationship, the depth anomaly region is mapped to the coordinate system of the RGB image; Determine the intersection and union regions between the mapped depth anomaly region and the corresponding candidate target region; The spatial consistency feature is determined based on the ratio of the area of ​​the intersection region to the area of ​​the union region.

7. The method according to claim 1, characterized in that, The image detection result includes RGB detection confidence; the step of detecting whether there is an intruder in the platform door gap area based on the image detection result, the depth region features, and the spatial consistency features includes: The RGB detection confidence, the depth region features, and the spatial consistency features are weighted and fused according to their respective fusion weights to obtain the intrusion risk score. If the intrusion risk score reaches the risk threshold, it is determined that an intruder exists in the candidate target area.

8. The method according to claim 7, characterized in that, The fusion weights are determined in the following way: Based on the brightness characteristics in the RGB image, determine the RGB quality index; Determine the depth quality index based on the effective depth ratio and depth variance of the candidate depth region; Based on the threshold conditions satisfied by the RGB quality index and the depth quality index, the baseline weights corresponding to at least one of the RGB detection confidence and the depth region features are selectively adjusted. The adjusted weights and the baseline weights corresponding to the spatial consistency features are normalized to obtain the fusion weights corresponding to the RGB detection confidence, the depth region features, and the spatial consistency features, respectively.

9. The method according to claim 7 or 8, characterized in that, The method further includes: Temporal stability verification is performed on the intrusion risk scores corresponding to the same candidate target in multiple consecutive image frames; The proportion of target image frames whose intrusion risk score reaches the risk threshold in a predetermined number of consecutive image frames is counted. When the ratio reaches the stability threshold, a detection result indicating the presence of an intruder in the candidate target is output.

10. A device for detecting intrusions through platform screen door gaps, characterized in that, include: The image acquisition and mapping module is configured to acquire RGB images and depth images of the platform door gap area, and establish a spatial mapping relationship between the RGB images and the depth images; The image detection module is configured to perform target detection based on the RGB image to obtain at least one candidate target region and an image detection result corresponding to each candidate target region; The depth region determination module is configured to determine a candidate depth region corresponding to each candidate target region in the depth image based on the spatial mapping relationship. The depth information extraction module is configured to determine depth anomaly regions within the candidate depth region and extract depth region features from the candidate depth region. The spatial matching module is configured to perform spatial matching between the depth anomaly region and the corresponding candidate target region to obtain spatial consistency features; The fusion detection module is configured to detect whether there is an intruder in the gap area of ​​the platform door based on the image detection results, the depth region features and the spatial consistency features.

11. An electronic device, characterized in that, It includes a processor and a memory, wherein the memory stores a computer program or instructions, which, when executed by the processor, cause the electronic device to perform the method of any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the method of any one of claims 1 to 9.

Images