Target detection method and device in screen door scene

Abstract

The invention relates to the technical field of target detection, and discloses a target detection method and device in a screen door scene, and the method comprises the steps: obtaining a preset detection region and a template point cloud; the method comprises the following steps: acquiring current point cloud data of a preset detection area in a rail transit operation process after a shielding door is closed in real time, taking the current point cloud data as real-time point cloud, carrying out difference on the real-time point cloud and the template point cloud to obtain a difference result, and judging whether an abnormal area exists or not through a clustering algorithm; if yes, obtaining an abnormal region in the real-time point cloud as a region of interest, obtaining a center coordinate of an abnormal object, and outwards expanding a preset number of pixel points to form a sparse depth prompt point set; projecting the point cloud data of the region of interest to a two-dimensional plane to generate a dense depth prompt mask; and inputting the two pieces of prompt information into an SAM model, performing instance segmentation on the region of interest, obtaining a two-dimensional bounding box of the abnormal object and the category of the abnormal object, and completing identification of the abnormal object.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to a target detection method and device in a shielding door scenario. Background Art

[0002] At present, when obstacles appear between the screen door and the train door, they are mainly judged manually by the locomotive driver and the platform attendant. The platform monitoring equipment can only provide certain assistance for manual identification, which makes the identification results easily affected by factors such as platform conditions, personnel's physiological and psychological state, and obstacle identification is time-consuming and laborious. Therefore, under the current background of intelligent and information-based development of rail, rail obstacle detection technology will become a key technology to realize unmanned train driving and unmanned station operations. In order to ensure the safety of passengers and the safe operation of trains, it is very valuable to find a solution for automatic and intelligent detection and identification of obstacle intrusion.

[0003] In recent years, a variety of intelligent obstacle detection technologies have been proposed at home and abroad. The core of these technologies focuses on the research and application of sensor selection and data processing algorithms. The current mainstream obstacle detection methods are roughly divided into image-based target detection and point cloud-based target detection. Although the detection technology is constantly improving, its application in the field of rail transit double door anti-pinch detection is still relatively scarce; insufficient obstacle recognition between shielding doors may pose a hidden danger to the safe operation of trains. The current mainstream obstacle detection technologies include two-dimensional target-based detection and three-dimensional target-based detection.

[0004] Detection based on two-dimensional targets is usually based on image processing, but it also has obvious shortcomings in the shielding door detection scenario. The two-dimensional vision system can only provide plane information, but cannot directly measure the depth of the target object. This is easy to cause detection errors and limited detection accuracy when dealing with shielding doors and other objects that are far away or the detection targets are small. In addition, two-dimensional detection is greatly affected by changes in ambient light, especially in the shielding door scenario. Due to metal reflection and occlusion, the quality of the collected image is reduced, affecting the accuracy of feature extraction, and thus affecting the detection accuracy. During the opening and closing process of the shielding door, the shape and position of the target object will change, and the two-dimensional image is difficult to adapt to this dynamic change, which may lead to response delays, false detections, and missed detections.

[0005] Based on the detection of three-dimensional targets, point clouds and depth sensors are used to obtain the spatial information of the target object. Although three-dimensional detection can provide depth information, its accuracy is easily affected by ambient light, sensor calibration errors and parallax errors. Especially when the processing distance is far or the target object is small, the accuracy drops significantly, resulting in unstable detection results. Point cloud data often contains noise, especially in complex shielding door scenes. Noise increases the difficulty of target detection, reduces detection accuracy and stability, and affects the actual application effect. At the same time, the cost of three-dimensional sensors is high, and the manufacturing and maintenance costs of the system are also relatively expensive, which makes it difficult to promote three-dimensional detection systems in cost-sensitive markets, limiting the possibility of large-scale applications.

[0006] Therefore, in the special scenario of the shielding door, directly using the collected data for detection will result in low detection accuracy and stability. Furthermore, at this stage, deep learning models are used for obstacle recognition. Although some deep learning models perform well in image segmentation, in the scenario of the shielding door, due to the special acquisition angle, the metal material of the shielding door will produce strong reflections, which will interfere with the normal processing of the image, resulting in difficulty in feature extraction and increased errors. In addition, the structure of the shielding door and changes in the status of the shielding door during rail transit operation will also increase the difficulty and time of detection.

[0007] In summary, the existing technology is not suitable for the shielding door scene. Due to the special material and structure of the shielding door, and the dynamic changes of the shielding door, the existing technology has low recognition accuracy and poor detection efficiency for obstacles in the shielding door scene. Summary of the invention

[0008] Therefore, the technical problem to be solved by the present invention is to overcome the problem that the prior art cannot take both cost and recognition accuracy into consideration when performing obstacle recognition on the shielding door scene.

[0009] In order to solve the above technical problems, the present invention provides a target detection method in a shielding door scenario, comprising: In the three-dimensional coordinate system, the axis constraints are performed on the detection area where the shielding door is located to obtain the preset detection area; Obtaining point cloud data of a preset detection area of ​​the shielding door in an initial closed state as a template point cloud; Real-time collection of current point cloud data of the preset detection area after the platform shield door is closed during the operation of rail transit as real-time point cloud; The real-time point cloud of the preset detection area is differentiated from the template point cloud to obtain the differential result; the differential result is detected using the point cloud clustering method to obtain the differential clustered point cloud to determine whether there is an abnormal area: If the number of point clouds in the differential clustering point cloud exceeds the preset number of point clouds, it is considered that there is an abnormal area in the preset detection area, and the abnormal area in the real-time point cloud is obtained as the area of ​​interest; Based on the point cloud data of the area of ​​interest, the center coordinates of the abnormal object are obtained, and the center coordinates of the abnormal object are expanded outward by a preset number of pixels to form a sparse depth prompt point set; Project the point cloud data of the region of interest onto a two-dimensional plane to generate a dense depth cue mask; Taking the sparse depth cue point set and the dense depth cue mask as cue information, the region of interest and the cue information are input into the SAM model, and the region of interest is segmented to obtain the two-dimensional bounding box and abnormal object category of the abnormal object.

[0010] Preferably, the differential result is detected using a point cloud clustering method to obtain a differential clustering point cloud and determine whether there is an abnormal area. It also includes: if the number of point clouds in the differential clustering point cloud does not exceed the preset number of point clouds, it is considered that there is no abnormal area in the preset detection area and the detection is terminated.

[0011] Preferably, after obtaining the two-dimensional bounding box of the abnormal object, the method further includes: mapping the two-dimensional bounding box of the abnormal object to a three-dimensional space to obtain the three-dimensional space coordinates of the abnormal object.

[0012] Preferably, the acquisition of point cloud data includes: Use binocular cameras to collect left-eye images and right-eye images of the shielding door area; The left image and the right image are stereo matched to obtain the left and right eye disparity; according to the triangular similarity principle and the left and right eye disparity, the depth coordinate of each pixel point is calculated to generate a depth image; Based on the horizontal and vertical coordinates of each pixel in the left image and the right image and their corresponding depth coordinates, the left image and the right image are converted into point cloud data.

[0013] Preferably, the depth coordinate of each pixel is calculated based on the triangular similarity principle and the left and right eye parallax, which is expressed as: ; in, Represents the depth coordinate of the pixel point, represents the focal length of the binocular camera, Indicates the distance between the left camera and the right camera in a binocular camera. and Respectively represent the horizontal coordinates of the pixel points in the left camera and the right camera.

[0014] Preferably, obtaining the center coordinates of the abnormal object based on the point cloud data of the region of interest includes: Get the maximum horizontal coordinate, vertical coordinate and depth coordinate in the point cloud data of the area of ​​interest, expressed as: ; Get the minimum horizontal coordinate, vertical coordinate and depth coordinate in the point cloud data of the area of ​​interest, expressed as: ; Abnormal object center coordinates , respectively expressed as: Abnormal object horizontal coordinate: ; Abnormal object vertical coordinate: ; Depth coordinates of abnormal objects: .

[0015] Preferably, the center coordinates of the abnormal object are expanded outward by a preset number of pixels to form a sparse depth prompt point set, which is expressed as: ; in, Represents the coordinates of the i-th pixel in the sparse depth cue point set; Indicates the preset number of pixels that expand outward.

[0016] Preferably, projecting the point cloud data of the region of interest onto a two-dimensional plane to generate a dense depth hint mask comprises: Abnormal Object Area Marked as 1, background area marked as 0, dense depth hint mask It is expressed as: .

[0017] Preferably, the point cloud clustering algorithm includes a k-means clustering algorithm, a DBSCAN clustering algorithm and a Euclidean clustering algorithm.

[0018] This embodiment provides an object detection device in a shielding door scenario, including: Binocular camera, used to collect left-eye and right-eye images when the platform shield door is initially closed and after the platform shield door is closed during rail transit operation; The target detection device in the shielding door scene is connected to the binocular camera for communication, and applies the target detection method in the shielding door scene as described above, obtains the corresponding point cloud data for identification based on the left eye image and the right eye image, and obtains the center point, size and outline of the obstacle point cloud data; A display device is communicatively connected to the target detection device in the shielding door scene, and is used to display the two-dimensional boundary box and abnormal object category of the identified abnormal object.

[0019] The above technical solution of the present invention has the following beneficial effects compared with the prior art: The target detection method in the platform shielding door scenario described in the present invention obtains the region of interest by differentiating the template point cloud in the initial closed state of the platform shielding door in the preset detection area with the real-time point cloud after the platform shielding door is closed during rail transit operation; and constructs a sparse depth prompt point set and a dense depth prompt mask based on the region of interest, and inputs them into the SAM model together with the region of interest to perform instance segmentation and identify obstacles. The present invention combines three-dimensional point cloud clustering with two-dimensional image segmentation to accurately obtain the size and position of obstacles, further evaluate the driving safety of rail transit, and ensure the reliability of detection results; at the same time, after detecting suspected anomalies, the present invention constructs a sparse depth prompt point set and a dense depth prompt mask to guide the SAM model; the sparse depth prompt point set is used to guide the SAM model to focus on the abnormal area and more accurately define the obstacle boundary, and the dense depth prompt mask defines the approximate range of the obstacle, which can guide the SAM model to perform segmentation in a narrower area; the sparse depth prompt point set and the dense depth prompt mask are used to eliminate point cloud noise, provide strong prior guidance for anomaly detection, so that the reasoning accuracy of the SAM model in complex scenes of shielding doors is significantly improved, the segmentation effect is optimized, and interference factors such as occlusion and reflection can be effectively dealt with, and the detection robustness of the model in narrow or dim environments is enhanced, thereby ensuring that high-precision detection is still achieved under complex conditions.

[0020] And because the SAM model is difficult to achieve real-time performance under long-term operation, the present invention performs preliminary detection through point cloud differential clustering. When no abnormality is detected, there is no need to call the SAM model, thereby reducing the system load. After a suspected abnormality is found, the SAM model is called to further improve the detection accuracy, thereby ensuring the efficiency and accuracy of the detection.

[0021] The present invention generates point cloud data based on the left-eye image and the right-eye image collected by the binocular camera, and differentiates the point cloud data in different states to perform subsequent obstacle recognition. There is no need to add high-cost equipment such as radar, and the installation is convenient and low-cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein: Figure 1 It is a flowchart of the steps of the target detection method in the shielding door scenario provided by the present invention; Figure 2 It is a schematic diagram of the preset detection area provided by the present invention; Figure 3 It is the point cloud clustering result diagram provided by the present invention; Figure 4It is the SAM segmentation result diagram provided by the present invention; Figure 5 This is a graphic interface display diagram of the target detection device in the shielding door scenario provided by the present invention; Figure 6 This is a point cloud interface display diagram of the target detection device in the shielding door scenario provided by the present invention. DETAILED DESCRIPTION

[0023] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0024] Reference Figure 1 As shown, the step flow chart of the target detection method in the shielding door scenario provided by the present invention includes the following steps: S101: In a three-dimensional coordinate system, an axis constraint is performed on a detection area where the shielding door is located to obtain a preset detection area; S102: Acquire point cloud data of a preset detection area of ​​the shielding door in an initial closed state as a template point cloud; S103: collecting in real time the current point cloud data of the preset detection area after the platform shield door is closed during the operation of the rail transit as the real-time point cloud; S104: Differentiate the real-time point cloud and the template point cloud of the preset detection area to obtain a differential result; use the point cloud clustering method to detect the differential result, obtain a differential clustered point cloud, and determine whether there is an abnormal area: S104-1: If the number of point clouds in the differential clustering point cloud does not exceed the preset number of point clouds, it is considered that there is no abnormal area in the preset detection area, and the detection is terminated; S104-2: If the number of point clouds in the differential clustering point cloud exceeds the preset number of point clouds, it is considered that there is an abnormal area in the preset detection area, and the abnormal area in the real-time point cloud is obtained as the area of ​​interest; S105: based on the point cloud data of the region of interest, obtaining the center coordinates of the abnormal object, and expanding the center coordinates of the abnormal object outward by a preset number of pixels to form a sparse depth prompt point set; S106: Projecting the point cloud data of the region of interest onto a two-dimensional plane to generate a dense depth cue mask; S107: Using the sparse depth cue point set and the dense depth cue mask as cue information, inputting the region of interest and the cue information into the SAM model, performing instance segmentation on the region of interest, and obtaining a two-dimensional bounding box of the abnormal object and an abnormal object category.

[0025] In step S101, this embodiment divides a preset detection area by performing axis constraints on the three-dimensional coordinate axes respectively, and only detects the area that may affect the normal closing of the shielding door, so as to reduce the detection range and improve the detection efficiency.

[0026] In an embodiment of the present invention, obtaining point cloud data includes: Use binocular cameras to collect left-eye images and right-eye images of the shielding door area; The left image and the right image are stereo matched to obtain the left and right eye disparity; according to the triangular similarity principle and the left and right eye disparity, the depth coordinate of each pixel point is calculated to generate a depth image; Based on the horizontal and vertical coordinates of each pixel in the left image and the right image and their corresponding depth coordinates, the left image and the right image are converted into point cloud data.

[0027] Among them, the depth coordinate of each pixel , expressed as: ; represents the focal length of the binocular camera, Indicates the distance between the left camera and the right camera in a binocular camera. and Respectively represent the horizontal coordinates of the pixel points in the left camera and the right camera.

[0028] Specifically, in step S105, the construction of the sparse depth cue point set includes: S105-1: Obtain the maximum horizontal coordinate, vertical coordinate and depth coordinate in the point cloud data of the area of ​​interest, expressed as: ; S105-2: Obtain the minimum horizontal coordinate, vertical coordinate and depth coordinate in the point cloud data of the area of ​​interest, expressed as: ; S105-3: Center coordinates of the anomaly , respectively expressed as: Abnormal object horizontal coordinate: ; Abnormal object vertical coordinate: ; Depth coordinates of abnormal objects: .

[0029] S105-4: Expand the center coordinates of the abnormal object outward by a preset number of pixels to form a sparse depth prompt point set, which is expressed as: ; in, Represents the coordinates of the i-th pixel in the sparse depth cue point set; Indicates the preset number of pixels that expand outward.

[0030] Specifically, in step S106, the point cloud data of the area of ​​interest is projected onto a two-dimensional plane, and the projected abnormal object area is marked as 1, and the background area is marked as 0, so the dense depth hint mask It is expressed as: .

[0031] In an embodiment of the present invention, after obtaining the two-dimensional bounding box of the abnormal object, it also includes: mapping the two-dimensional bounding box of the abnormal object to three-dimensional space, obtaining the three-dimensional space coordinates of the abnormal object, so that the staff can directly perform abnormality inspection according to the three-dimensional coordinates of the abnormal object.

[0032] In an embodiment of the present invention, the point cloud clustering algorithm includes a k-means clustering algorithm, a DBSCAN clustering algorithm and a Euclidean clustering algorithm.

[0033] The target detection system based on the SAM model and point cloud in the embodiment of the present invention aims to use the high precision of the SAM model to perform instance segmentation on the image of the detection area. The system preliminarily detects the suspected abnormal area through the three-dimensional point cloud clustering algorithm, and projects the area into the image coordinate system. The center point and boundary contour of the clustering result are input into the SAM model as sparse and dense prompts, thereby realizing accurate instance segmentation in the two-dimensional image. Finally, the SAM model outputs a two-dimensional detection box of the small target anomaly to improve the detection accuracy.

[0034] In order to obtain a valid two-dimensional frame, a feature extractor of the SAM model is introduced. This embodiment introduces the SAM model to perform instance segmentation on the ROI area, forms the edge and contour of the mask image based on the different segmented areas, converts the point cloud obtained by the binocular camera into a two-dimensional image, obtains the point cloud in the mask through the mask, a matrix (K), a rotation matrix (R) and a translation matrix (t). The external parameter matrix can be expressed as: ; Among them, rx, ry and rz represent the rotation error angle of each axis in the world coordinate system, R (rx, ry, rz) is their corresponding rotation 3×3 matrix, and t is the translation error between cameras. Then, the point cloud of the world coordinate system is converted to the pixel coordinate system of the camera perspective, which is expressed as: ; Where [x, y, z] is the three-dimensional coordinate of the radar, and [u, v] is the pixel coordinate system for projecting the point cloud to the camera image. The camera intrinsic parameter matrix K is: ; Among them, (fu,fv) and (cu,cv) are the focal length and optical center distance of the camera respectively, and bs is the distance between the optical centers of the cameras.

[0035] SAM can generate multiple accurate and reasonable masks for any image based on the prompts, demonstrating substantial impact and potential progress in segmentation tasks. In order to improve the segmentation quality of the original SAM model, the prompt adapter is used to prompt the model based on a dedicated environment (between two doors). Depth prompts are divided into sparse depth prompts and dense depth prompts. After obtaining the coordinates of the center point of the obstacle, its center point coordinates are expanded outward by 10 pixels as a sparse depth prompt point set; the overall outline of the obstacle is projected onto the two-dimensional image to generate a mask as a dense depth prompt mask; input into the SAM model to optimize the inference results; the SAM model accepts the input sparse point prompts to guide the model to focus on a specific area. These point prompts can be marked as "positive examples" or "negative examples" to help the model define the target boundary more accurately. The SAM model accepts masks as dense prompts, which define the approximate range of the target and guide the model to segment in a narrower area; by constructing a sparse depth prompt point set (providing target center point information) and a dense depth prompt mask (providing target overall outline information), the two are combined and input into the SAM model as prompt information. These prompts help the SAM model to more accurately segment target objects in complex scenes such as shielding doors, optimize its reasoning results, and improve detection accuracy.

[0036] Based on the above description, specifically, when the target detection method in the screen door scenario provided by the present invention is used to perform target detection, the target detection is performed by designing a prompt adapter to guide the optimization reasoning of the model, which specifically includes: The design of the hint adapter is based on a method that combines sparse depth hints with dense depth hints; sparse hints are used to provide the center point coordinates of obstacles, and dense hints are used to provide the overall contour information of obstacles.

[0037] ① Generation of sparse depth hints: Obtain the center coordinates of the obstacle through the detection algorithm Then, the center point coordinates are expanded outward by 10 pixels to form a sparse depth cue point set { }, expressed as: ; The acquisition of the center point coordinates of the obstacle includes: Get the maximum horizontal coordinate, vertical coordinate and depth coordinate in the point cloud cluster, expressed as: ; Get the smallest horizontal coordinate, vertical coordinate and depth coordinate in the point cloud cluster, expressed as: ; Obstacle center coordinates , respectively expressed as: Obstacle horizontal coordinate: ; Obstacle ordinate: ; Obstacle depth coordinates: .

[0038] ② Generation of dense depth cues: The SAM model can accept masks as dense cues. The masks define the approximate range of the target and guide the model to segment in a narrower area. The overall outline of the obstacle is projected onto the two-dimensional image to generate a dense depth cue mask M, which is expressed as: ; The mask M is a binary image where the obstacle area is marked as 1 and the background area is marked as 0.

[0039] ③Optimization reasoning of SAM model: The SAM model has an interface that accepts sparse and dense hints, converting a sparse depth hint point set { } and dense depth hint mask M are input into the SAM model to optimize the inference results. The SAM model has strong generalization ability and can distinguish most objects with high detection accuracy. By combining these two kinds of depth hint information, the SAM model can generate more accurate and reasonable masks; ; Where I represents the input image, { } represents a sparse depth cue point set, and M represents a dense depth cue mask.

[0040] The present invention significantly improves the segmentation accuracy of the SAM model by introducing sparse and dense prompts. In particular, sparse prompts with depth information are introduced into the detection process to guide the model to more accurately identify abnormal areas. After the suspected anomaly is detected by three-dimensional clustering, it is further confirmed by the SAM model, which improves the accuracy and stability of the detection. At the same time, the combination of sparse and dense prompts enables the SAM model to effectively deal with complex factors such as reflection and occlusion in the shielding door scene, and enhances the detection robustness of the model in narrow or dim environments, thereby ensuring that high-precision detection is still achieved under complex conditions. In addition, since the SAM model is difficult to achieve real-time performance under long-term operation, the present invention performs preliminary detection through point cloud differential clustering. When no anomaly is detected, there is no need to call the SAM model, thereby reducing the system load; and after the suspected anomaly is found, the SAM model is called to further improve the detection accuracy and ensure the efficiency and accuracy of the detection.

[0041] Based on the above embodiment, the target detection method in the platform screen door scenario provided by the present invention is applied to the actual rail transit operation, and the specific steps include: S201: In a preset three-dimensional detection area, the point cloud data of the previous and next frames are differentiated to preliminarily detect possible anomalies; the differential results are processed using a point cloud clustering algorithm to identify clusters of suspected anomalies; Use a clustering algorithm to classify valid targets: Treat each point as a class and calculate the minimum distance between every two classes ; Merge the two classes with the smallest distance into a new class; Recalculate the distance between the two classes; Repeat the above two steps until there is no distance between any two categories that satisfies , the clustering ends and multiple clusters are obtained; when the number of point clouds in a cluster exceeds the preset number of point clouds, it is judged to be abnormal, otherwise it is considered to be noise; Reference Figure 2 The figure shows the schematic diagram of the preset detection area; Figure 3 As shown, it is the point cloud clustering result diagram; S202: If there is no difference between the previous and next frames after the point cloud data of the previous and next frames of the preset three-dimensional detection area are differentiated, it is considered that there is no abnormality and the detection process ends; S203: If there is a difference between the front and rear frames after the difference of the point cloud data of the front and rear frames of the preset three-dimensional detection area, the center of the cluster and the points around it are projected onto the two-dimensional image as sparse prompts; dense prompts are generated by combining the overall contour information of the obstacle; the sparse prompts and dense prompts are input into the SAM model; the SAM model is used to perform instance segmentation on the target area, the segmentation effect is optimized through sparse prompts and dense prompts, and the segmentation results are output.

[0042] Reference Figure 4 As shown, it is the SAM segmentation result diagram; S204: If the segmented area meets the preset abnormal label (such as a person, a bag, an umbrella, etc.), the detection result and the abnormal position are output; This embodiment combines 3D point cloud clustering detection with 2D image segmentation to accurately obtain the size and location of anomalies, thereby evaluating their impact on driving safety and ensuring the reliability of detection results. After detecting suspected anomalies, the SAM model is guided by sparse and dense prompts, which significantly improves its reasoning accuracy in complex scenes of shielding doors, optimizes the segmentation effect, effectively copes with interference factors such as occlusion and reflection, and improves the accuracy of detection.

[0043] The embodiment of the present invention further provides a target detection device in a shielding door scenario, including: Binocular camera, used to collect left-eye and right-eye images when the platform shield door is initially closed and after the platform shield door is closed during rail transit operation; The target detection device in the shielding door scene is connected to the binocular camera for communication, and applies the target detection method in the shielding door scene as described above, obtains the corresponding point cloud data for identification based on the left eye image and the right eye image, and obtains the center point, size and outline of the obstacle point cloud data; A display device is communicatively connected to the target detection device in the shielding door scene, and is used to display the two-dimensional boundary box and abnormal object category of the identified abnormal object.

[0044] Specifically, refer to Figure 5 and Figure 6 The figure shows an image interface display diagram and a point cloud interface display diagram of a target detection device in a shielding door scene.

[0045] Based on the above embodiments, the embodiments of the present invention are combined with the data collected on the vehicle depot test platform to illustrate the performance of the present invention, that is, the performance of the algorithm is verified through simulation experiments, as follows: In the test experiment, different types of test objects were set up, and the obstacle detection effect was tested at three distances (60cm, 120cm, and 180cm). The details are as follows: Test the object of type T-ODS-FA-BL-4 (Class A test object black 4 cm³ foam); Design type is T-test report, machine / component type is ODS-obstacle detection system, report extension type is F-function, test type: foam block WH, test size XX is 4cm*4cm*4cm, 6cm*6cm*6cm, 8cm*8cm*8cm; The obstacle detection results are shown in Table 1: Table 1 Obstacle detection results

[0046] It can be seen from the table that the binocular-based foreign body detection method can detect foreign bodies and accurately locate the positions of foreign bodies.

[0047] The target detection method in the platform shielding door scenario described in the present invention obtains the region of interest by differentiating the template point cloud in the initial closed state of the platform shielding door in the preset detection area with the real-time point cloud after the platform shielding door is closed during rail transit operation; and constructs a sparse depth prompt point set and a dense depth prompt mask based on the region of interest, and inputs them into the SAM model together with the region of interest to perform instance segmentation and identify obstacles. The present invention combines three-dimensional point cloud clustering with two-dimensional image segmentation to accurately obtain the size and position of obstacles, further evaluate the driving safety of rail transit, and ensure the reliability of detection results; at the same time, after detecting suspected anomalies, the present invention constructs a sparse depth prompt point set and a dense depth prompt mask to guide the SAM model; the sparse depth prompt point set is used to guide the SAM model to focus on the abnormal area and more accurately define the obstacle boundary, and the dense depth prompt mask defines the approximate range of the obstacle, which can guide the SAM model to perform segmentation in a narrower area; the sparse depth prompt point set and the dense depth prompt mask are used to eliminate point cloud noise, provide strong prior guidance for anomaly detection, so that the reasoning accuracy of the SAM model in complex scenes of shielding doors is significantly improved, the segmentation effect is optimized, and interference factors such as occlusion and reflection can be effectively dealt with, and the detection robustness of the model in narrow or dim environments is enhanced, thereby ensuring that high-precision detection is still achieved under complex conditions. And because the SAM model is difficult to achieve real-time performance under long-term operation, the present invention performs preliminary detection through point cloud differential clustering. When no abnormality is detected, there is no need to call the SAM model, thereby reducing the system load; and after a suspected abnormality is found, the SAM model is called to further improve the detection accuracy, ensuring the efficiency and accuracy of the detection. In addition, the present invention generates point cloud data based on the left and right images collected by the binocular camera, and differentiates the point cloud data in different states, so that subsequent obstacle identification can be performed, without adding high-cost equipment such as radar, and is easy to install and low-cost.

[0048] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0049] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0050] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0051] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0052] Obviously, the above embodiments are merely examples for the purpose of clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.

Claims

1. A target detection method in a screen door scenario, characterized in that: include: In the three-dimensional coordinate system, the axis constraints are performed on the detection area where the shielding door is located to obtain the preset detection area; Obtaining point cloud data of a preset detection area of ​​the shielding door in an initial closed state as a template point cloud; Real-time collection of current point cloud data of the preset detection area after the platform shield door is closed during the operation of rail transit as real-time point cloud; The real-time point cloud of the preset detection area is differentiated from the template point cloud to obtain the differential result; the differential result is detected using the point cloud clustering method to obtain the differential clustered point cloud to determine whether there is an abnormal area: If the number of point clouds in the differential clustering point cloud exceeds the preset number of point clouds, it is considered that there is an abnormal area in the preset detection area, and the abnormal area in the real-time point cloud is obtained as the area of ​​interest; Based on the point cloud data of the area of ​​interest, the center coordinates of the abnormal object are obtained, and the center coordinates of the abnormal object are expanded outward by a preset number of pixels to form a sparse depth prompt point set; Project the point cloud data of the region of interest onto a two-dimensional plane to generate a dense depth cue mask; Taking the sparse depth cue point set and the dense depth cue mask as cue information, the region of interest and the cue information are input into the SAM model, and the region of interest is segmented to obtain the two-dimensional bounding box and abnormal object category of the abnormal object.

2. The target detection method in the screen door scene according to claim 1 is characterized in that: The differential result is detected by using a point cloud clustering method to obtain a differential clustering point cloud and determine whether there is an abnormal area. The method also includes: if the number of point clouds in the differential clustering point cloud does not exceed the preset number of point clouds, it is considered that there is no abnormal area in the preset detection area and the detection is terminated.

3. The target detection method in the platform shielding door scene according to claim 1, characterized in that: After obtaining the two-dimensional bounding box of the abnormal object, the method further includes: mapping the two-dimensional bounding box of the abnormal object to a three-dimensional space to obtain the three-dimensional space coordinates of the abnormal object.

4. The target detection method in the screen door scene according to claim 1 is characterized in that: Acquisition of point cloud data, including: Use binocular cameras to collect left-eye images and right-eye images of the shielding door area; The left image and the right image are stereo matched to obtain the left and right eye disparity; according to the triangular similarity principle and the left and right eye disparity, the depth coordinate of each pixel point is calculated to generate a depth image; Based on the horizontal and vertical coordinates of each pixel in the left image and the right image and their corresponding depth coordinates, the left image and the right image are converted into point cloud data.

5. The target detection method in the scene of the platform shielding door according to claim 4 is characterized in that: According to the triangular similarity principle and the left and right eye disparity, the depth coordinate of each pixel is calculated, which is expressed as: ； in, Represents the depth coordinate of the pixel point, represents the focal length of the binocular camera, Indicates the distance between the left camera and the right camera in a binocular camera. and Respectively represent the horizontal coordinates of the pixel points in the left camera and the right camera.

6. The target detection method in the scene of the platform shielding door according to claim 1, characterized in that: The step of obtaining the center coordinates of the abnormal object based on the point cloud data of the region of interest includes: Get the maximum horizontal coordinate, vertical coordinate and depth coordinate in the point cloud data of the area of ​​interest, expressed as: ; Get the minimum horizontal coordinate, vertical coordinate and depth coordinate in the point cloud data of the area of ​​interest, expressed as: ; Abnormal object center coordinates , respectively expressed as: Abnormal object horizontal coordinate: ; Abnormal object vertical coordinate: ; Depth coordinates of abnormal objects: .

7. The target detection method in the scene of the platform shielding door according to claim 6, characterized in that: The center coordinates of the abnormal object are expanded outward by a preset number of pixels to form a sparse depth prompt point set, which is expressed as: ； in, Represents the coordinates of the i-th pixel in the sparse depth cue point set; Indicates the preset number of pixels that expand outward.

8. The target detection method in the scene of the platform shielding door according to claim 7, characterized in that: The step of projecting the point cloud data of the region of interest onto a two-dimensional plane to generate a dense depth hint mask includes: Abnormal Object Area Marked as 1, background area marked as 0, dense depth hint mask It is expressed as: 。 9. The target detection method in the scene of the platform shielding door according to claim 1, characterized in that: The point cloud clustering algorithms include k-means clustering algorithm, DBSCAN clustering algorithm and Euclidean clustering algorithm.

10. A target detection device in a shielding door scenario, characterized in that: include: Binocular camera, used to collect left-eye and right-eye images when the platform shield door is initially closed and after the platform shield door is closed during rail transit operation; The target detection device in the shielding door scene is connected to the binocular camera for communication, and applies the target detection method in the shielding door scene as described in any one of claims 1 to 9, obtains the corresponding point cloud data for identification based on the left eye image and the right eye image, and obtains the center point, size and outline of the obstacle point cloud data; A display device is communicatively connected to the target detection device in the shielding door scene, and is used to display the two-dimensional boundary box and abnormal object category of the identified abnormal object.

Images