A target detection method and device in a shielded door scene

Abstract

The application relates to the technical field of target detection and discloses a target detection method and equipment in a shield door scene, which comprises the following steps: acquiring a preset detection area and a template point cloud; acquiring current point cloud data of the preset detection area after a shield door is closed in a rail transit operation process as real-time point cloud, and performing difference between the real-time point cloud and the template point cloud to obtain a difference result; judging whether an abnormal area exists or not through a clustering algorithm; if the abnormal area exists, acquiring the abnormal area in the real-time point cloud as a region of interest, acquiring a center coordinate of an abnormal object, and extending a preset number of pixel points outward to form a sparse depth prompt point set; projecting point cloud data of the region of interest onto a two-dimensional plane to generate a dense depth prompt mask; inputting two prompt information into a SAM model to perform instance segmentation on the region of interest, acquiring a two-dimensional boundary box of the abnormal object and an abnormal object category, and completing identification of the abnormal object.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, in particular to a target detection method and device in a shielded door scene. BACKGROUND

[0002] Currently, when an obstacle appears between the shielded door and the train door, the train driver and the platform train attendant mainly rely on human judgment. The platform monitoring equipment can only provide certain assistance for manual identification, which leads to the fact that the identification result is easily affected by factors such as the platform situation, personnel physiology, and psychological state, and the obstacle identification is time-consuming and laborious. Therefore, under the current background of track intelligentization and informatization development, track obstacle detection technology will become a key technology for realizing train unmanned driving and station yard operation unmanned. In order to ensure the safety of passengers and the safe driving of trains, it is very valuable to find a solution for automatic and intelligent detection and identification of obstacle intrusion.

[0003] In recent years, a variety of intelligent obstacle detection technologies have been proposed at home and abroad, and the core of these technologies is concentrated on the research and application of sensor selection and data processing algorithms. The current mainstream obstacle detection methods are roughly divided into image-based target detection and point cloud-based target detection. Although the detection technology is constantly improving, its application in the field of track transportation double-door anti-pinch detection is still relatively scarce; the insufficient obstacle identification between shielded doors may pose a hidden danger to the safe operation of trains. The current mainstream obstacle detection technology includes two-dimensional target-based detection and three-dimensional target-based detection.

[0004] Two-dimensional target-based detection is usually based on image processing, but it also has obvious shortcomings in the shielded door detection scene. The two-dimensional vision system intelligently provides planar information and cannot directly measure the depth of the target object, which can easily lead to detection errors and limited detection accuracy when dealing with shielded doors with a long distance or small detection targets. Moreover, two-dimensional detection is greatly affected by changes in environmental light, especially in the shielded door scene, as the quality of the collected images decreases due to metal reflection and shielding, affecting the accuracy of feature extraction and thus the detection accuracy. During the opening and closing process of the shielded door, the shape and position of the target object change, and two-dimensional images are difficult to adapt to such dynamic changes, which may lead to response delays, false positives, and missed detections.

[0005] Based on three-dimensional target detection, point cloud and depth sensor are used to obtain the spatial information of the target object; three-dimensional detection can provide depth information, but its accuracy is easily affected by environmental light, sensor calibration error and parallax error, especially when dealing with a long distance or a small target object, the accuracy decreases significantly, resulting in unstable detection results; point cloud data often contains noise, especially in complex shield door scenes, noise increases the difficulty of target detection, reduces the detection accuracy and stability, and affects the actual application effect. At the same time, the cost of three-dimensional sensor is high, and the manufacturing and maintenance cost of the system is also relatively expensive, which makes the three-dimensional detection system difficult to promote in the cost-sensitive market, and limits the possibility of large-scale application.

[0006] Therefore, in the special shield door scene, directly using the collected data for detection has low detection accuracy and stability; further, at present, a deep learning model is used for obstacle recognition, although some deep learning models perform well in image segmentation, but in the shield door scene, due to the special collection angle, the metal material of the shield door will produce strong light interference, which will interfere with the normal processing of the image, resulting in difficulty in feature extraction and increased error; and the structure of the shield door and the change of the shield door state during the operation of the rail transit will also increase the detection difficulty and detection time.

[0007] In summary, the prior art is not suitable for the shield door scene, due to the special material and structure of the shield door, and the dynamic change of the shield door, resulting in low recognition accuracy and poor detection efficiency of the prior art for obstacle recognition in the shield door scene. SUMMARY

[0008] Therefore, the technical problem to be solved by the present application is to overcome the problem that the prior art cannot balance the cost and recognition accuracy when recognizing obstacles in the shield door scene.

[0009] To solve the above technical problems, the present application provides a target detection method in a shield door scene, comprising:

[0010] In the three-dimensional coordinate system, the detection area where the shield door is located is subjected to axis constraint to obtain a preset detection area;

[0011] Obtain the point cloud data of the preset detection area of the shield door in the initial closed state as a template point cloud;

[0012] Real-time acquisition of the current point cloud data of the preset detection area after the shield door is closed during the operation of the rail transit as real-time point cloud;

[0013] Difference between the real-time point cloud and the template point cloud of the preset detection area is obtained, and the difference result is detected by using a point cloud clustering method to obtain a difference clustering point cloud, and it is judged whether there is an abnormal area:

[0014] If the number of point clouds in the differential clustering point cloud exceeds the preset number of point clouds, it is considered that there is an abnormal region in the preset detection region, and the abnormal region in the real-time point cloud is obtained as the region of interest;

[0015] Based on the point cloud data of the region of interest, the center coordinates of the abnormal object are obtained, and the center coordinates of the abnormal object are extended by a preset number of pixels to form a sparse depth hint point set;

[0016] The point cloud data of the region of interest is projected onto a two-dimensional plane to generate a dense depth hint mask;

[0017] The sparse depth hint point set and the dense depth hint mask are used as hint information, and the region of interest and the hint information are input into the SAM model to perform instance segmentation on the region of interest, and the two-dimensional bounding box of the abnormal object and the category of the abnormal object are obtained.

[0018] Preferably, the differential result is detected by using a point cloud clustering method to obtain a differential clustering point cloud, and it is judged whether there is an abnormal region. It also includes: if the number of point clouds in the differential clustering point cloud does not exceed the preset number of point clouds, it is considered that there is no abnormal region in the preset detection region, and the detection is ended.

[0019] Preferably, after obtaining the two-dimensional bounding box of the abnormal object, it further includes: mapping the two-dimensional bounding box of the abnormal object to a three-dimensional space to obtain the three-dimensional space coordinates of the abnormal object.

[0020] Preferably, the point cloud data is obtained, including:

[0021] The left eye image and the right eye image of the shielding door region are collected by using a binocular camera;

[0022] The left eye image and the right eye image are stereoscopically matched to obtain left and right eye disparities; according to the principle of triangular similarity and the left and right eye disparities, the depth coordinates of each pixel point are calculated to generate a depth image;

[0023] Based on the horizontal and vertical coordinates of each pixel point in the left eye image and the right eye image and the corresponding depth coordinates, the left eye image and the right eye image are converted into point cloud data.

[0024] Preferably, the depth coordinates of each pixel point are calculated according to the principle of triangular similarity and the left and right eye disparities, and are expressed as:

[0025] ;

[0026] Wherein, the depth coordinates of the pixel point are represented by z, the focal length of the binocular camera is represented by f, the distance between the left eye camera and the right eye camera in the binocular camera is represented by b, with respectively represent the horizontal coordinates of the pixel points in the left eye camera and the right eye camera.

[0027] Preferably, the point cloud data based on the region of interest is used to obtain the center coordinates of the abnormal object, including:

[0028] The maximum horizontal coordinate, vertical coordinate and depth coordinate in the point cloud data of the region of interest are obtained, denoted as: ;

[0029] The minimum horizontal coordinate, vertical coordinate and depth coordinate in the point cloud data of the region of interest are obtained, denoted as: ;

[0030] The center coordinates of the abnormal object , are respectively denoted as:

[0031] The horizontal coordinate of the abnormal object is: ;

[0032] The vertical coordinate of the abnormal object is: ;

[0033] The depth coordinate of the abnormal object is: .

[0034] Preferably, the center coordinates of the abnormal object are extended outward by a preset number of pixel points to form a sparse depth prompt point set, denoted as:

[0035] ;

[0036] wherein, represents the coordinates of the i-th pixel point in the sparse depth prompt point set; represents the preset number of pixel points extended outward.

[0037] Preferably, the point cloud data of the region of interest is projected onto a two-dimensional plane to generate a dense depth prompt mask, including:

[0038] The abnormal object region is marked as 1, and the background region is marked as 0, and the dense depth prompt mask is denoted as:

[0039] .

[0040] Preferably, the point cloud clustering algorithm includes a k-means clustering algorithm, a DBSCAN clustering algorithm and a Euclidean clustering algorithm.

[0041] The embodiment provides a target detection device in a shielding door scene, including:

[0042] A binocular camera is used to collect left eye images and right eye images of a shield door in an initial closing state and after the shield door is closed during rail transit operation;

[0043] A target detection device in a shield door scene is in communication connection with the binocular camera, and applies the target detection method in the shield door scene as described above, obtains corresponding point cloud data based on the left eye images and the right eye images, recognizes the center point, size and contour of the obstacle point cloud data;

[0044] A display device is in communication connection with the target detection device in the shield door scene, and is used to display a two-dimensional bounding box of the recognized abnormal object and an abnormal object category.

[0045] The above technical solutions of the present application have the following beneficial effects compared with the prior art:

[0046] The target detection method in the shield door scene of the present application differentiates the template point cloud in the preset detection area in the initial closing state of the shield door and the real-time point cloud after the shield door is closed during rail transit operation, obtains a region of interest, and constructs a sparse depth prompt point set and a dense depth prompt mask based on the region of interest, which are input into a SAM model together with the region of interest for instance segmentation to identify the obstacle. The present application accurately obtains the size and position of the obstacle by combining three-dimensional point cloud clustering and two-dimensional image segmentation, further evaluates the driving safety of rail transit, and ensures the reliability of the detection result. At the same time, the present application constructs a sparse depth prompt point set and a dense depth prompt mask after detecting a suspected abnormality to guide the SAM model. The sparse depth prompt point set is used to guide the SAM model to focus on the abnormal area to more accurately define the obstacle boundary, and the dense depth prompt mask defines the approximate range of the obstacle to guide the SAM model to segment in a narrower area. The use of the sparse depth prompt point set and the dense depth prompt mask eliminates point cloud noise and provides strong prior guidance for abnormal detection, so that the inference accuracy of the SAM model in the complex scene of the shield door is significantly improved, the segmentation effect is optimized, interference factors such as occlusion and reflection are effectively dealt with, and the detection robustness of the model in a narrow or dim environment is enhanced, thereby ensuring high-precision detection under complex conditions.

[0047] And since the SAM model is difficult to achieve real-time performance under long-time operation, the present application performs preliminary detection through point cloud differential clustering, does not need to call the SAM model when no abnormality is detected, and reduces the system load. When a suspected abnormality is found, the SAM model is called to further improve the detection accuracy and ensure the efficiency and accuracy of the detection.

[0048] And the left eye image and the right eye image collected by the binocular camera are used to generate point cloud data, and the point cloud data in different states is differentiated, so that subsequent obstacle identification can be performed, without increasing high-cost equipment such as radar, and the installation is convenient and low in cost. BRIEF DESCRIPTION OF DRAWINGS

[0049] In order to make the content of the present application more easily understood, the present application will be further described in detail below according to specific embodiments of the present application and in conjunction with the drawings, in which:

[0050] Figure 1 is a step flow chart of the target detection method in the shielding door scene provided by the present application;

[0051] Figure 2 is a preset detection area schematic diagram provided by the present application;

[0052] Figure 3 is a point cloud clustering result diagram provided by the present application;

[0053] Figure 4 is a SAM segmentation result diagram provided by the present application;

[0054] Figure 5 is an image interface display diagram of the target detection device in the shielding door scene provided by the present application;

[0055] Figure 6 is a point cloud interface display diagram of the target detection device in the shielding door scene provided by the present application. DETAILED DESCRIPTION

[0056] The present application will be further described below in conjunction with the drawings and specific embodiments, so that those skilled in the art can better understand the present application and implement it, but the embodiments are not limiting to the present application.

[0057] Referring to Figure 1 the step flow chart of the target detection method in the shielding door scene provided by the present application, the specific steps include:

[0058] S101: In a three-dimensional coordinate system, the detection area where the shielding door is located is subjected to axis constraint to obtain a preset detection area;

[0059] S102: Obtain point cloud data of the preset detection area of the shielding door in an initial closed state as a template point cloud;

[0060] S103: Real-time collect current point cloud data of the preset detection area after the shielding door is closed in the process of rail transit operation as real-time point cloud;

[0061] S104: difference between the real-time point cloud of the preset detection area and the template point cloud is obtained, and the difference result is detected by using a point cloud clustering method to obtain a difference clustering point cloud, and it is judged whether an abnormal area exists or not:

[0062] S104-1: if the number of point clouds in the difference clustering point cloud does not exceed the preset number of point clouds, it is considered that there is no abnormal area in the preset detection area, and the detection is ended;

[0063] S104-2: if the number of point clouds in the difference clustering point cloud exceeds the preset number of point clouds, it is considered that there is an abnormal area in the preset detection area, and the abnormal area in the real-time point cloud is obtained as a region of interest;

[0064] S105: based on the point cloud data of the region of interest, the center coordinates of the abnormal object are obtained, the center coordinates of the abnormal object are extended by a preset number of pixels to form a sparse depth prompt point set;

[0065] S106: the point cloud data of the region of interest is projected onto a two-dimensional plane to generate a dense depth prompt mask;

[0066] S107: taking the sparse depth prompt point set and the dense depth prompt mask as prompt information, the region of interest and the prompt information are input into a SAM model, the region of interest is instance segmented, and a two-dimensional bounding box of the abnormal object and a class of the abnormal object are obtained.

[0067] In step S101, the embodiment divides a preset detection area by respectively performing axis constraint on three-dimensional coordinate axes, and only detects the area that may affect the normal closing of the shielding door, so as to reduce the detection range and improve the detection efficiency.

[0068] In the embodiment of the application, the point cloud data is obtained, including:

[0069] The left eye image and the right eye image of the shielding door area are collected by using a binocular camera;

[0070] The left eye image and the right eye image are stereoscopically matched to obtain left and right eye disparities; the depth coordinates of each pixel point are calculated according to the principle of triangular similarity and the left and right eye disparities, and a depth image is generated;

[0071] Based on the horizontal and vertical coordinates of each pixel point in the left eye image and the right eye image and the corresponding depth coordinates, the left eye image and the right eye image are converted into point cloud data.

[0072] wherein the depth coordinates of each pixel point are expressed as: focal length of the binocular camera, distance between the left eye camera and the right eye camera in the binocular camera,​ respectively represent the horizontal coordinates of the pixel points in the left eye camera and the right eye camera. respectively represent the horizontal coordinates of the pixel points in the left eye camera and the right eye camera.

[0073] Specifically, in step S105, the construction of the sparse depth hint point set includes:

[0074] S105-1: Obtain the maximum horizontal coordinate, vertical coordinate and depth coordinate in the point cloud data of the region of interest, denoted as: ;

[0075] S105-2: Obtain the minimum horizontal coordinate, vertical coordinate and depth coordinate in the point cloud data of the region of interest, denoted as: ;

[0076] S105-3: Abnormal object center coordinates , respectively denoted as:

[0077] Abnormal object horizontal coordinate: ;

[0078] Abnormal object vertical coordinate: ;

[0079] Abnormal object depth coordinate: .

[0080] S105-4: Extend the abnormal object center coordinates outward by a preset number of pixel points to form a sparse depth hint point set, denoted as:

[0081] ;

[0082] wherein, represents the coordinates of the i-th pixel point in the sparse depth hint point set; represents the preset number of pixel points to be extended outward.

[0083] Specifically, in step S106, the point cloud data of the region of interest is projected onto a two-dimensional plane, and the projected abnormal object region is marked as 1 and the background region is marked as 0, so the dense depth hint mask is denoted as:

[0084] .

[0085] In the embodiment of the present application, after obtaining the two-dimensional bounding box of the abnormal object, it further includes: mapping the two-dimensional bounding box of the abnormal object to a three-dimensional space to obtain the three-dimensional space coordinates of the abnormal object, so that the staff can directly perform abnormal investigation according to the three-dimensional coordinates of the abnormal object.

[0086] In the embodiment of the present application, the point cloud clustering algorithm includes a k-means clustering algorithm, a DBSCAN clustering algorithm and a Euclidean clustering algorithm.

[0087] The target detection system based on the SAM model and the point cloud in the embodiment of the present application aims to use the high precision of the SAM model to perform instance segmentation on the image of the detection region. The system preliminarily detects a suspected abnormal region through a three-dimensional point cloud clustering algorithm, projects the region to an image coordinate system, and inputs the center point and the boundary contour of the clustering result as sparse and dense hints into the SAM model, so as to realize accurate instance segmentation in a two-dimensional image. Finally, the SAM model outputs a two-dimensional detection frame of a small target anomaly, improving the detection precision.

[0088] In order to obtain an effective two-dimensional frame, a feature extractor of the SAM model is introduced. In the embodiment, the SAM model is introduced to perform instance segmentation on the ROI region, different regions based on segmentation form a mask picture edge, contour, convert the point cloud obtained by the binocular camera into a two-dimensional image, and obtain the point cloud in the mask through the mask. An outer parameter matrix can be expressed as:

[0089] ;

[0090] Wherein, rx, ry and rz represent the rotation error angles of each axis in the world coordinate system, R (rx, ry, rz) is the corresponding rotation 3x3 matrix, and t is the translation error between cameras. Subsequently, the point cloud in the world coordinate system is converted to the pixel coordinate system in the camera view, which is expressed as:

[0091] ;

[0092] Wherein, [x, y, z] is the three-dimensional coordinate of the radar, and [u, v] is the pixel coordinate system of the point cloud projected to the camera image. Wherein the camera intrinsic matrix K is:

[0093] ;

[0094] Wherein, (fu, fv) and (cu, cv) are the focal length and the distance of the optical center of the camera, respectively, and bs is the distance between the optical centers of the camera.

[0095] The SAM can generate multiple accurate and reasonable masks for any image according to the prompt, which shows substantial impact and potential progress in the segmentation task. In order to improve the segmentation quality of the original SAM model, the prompt adapter is used to prompt the model based on the special environment (between two doors). The depth prompt is divided into sparse depth prompt and dense depth prompt. After obtaining the center point coordinates of the obstacle, the center point coordinates are expanded outward by 10 pixel points as a sparse depth prompt point set; the whole outline of the obstacle is projected onto the two-dimensional image to generate a mask as a dense depth prompt mask; the input SAM model optimizes the inference result; the SAM model accepts the input sparse point prompt to guide the model to focus on a specific area, and these point prompts can be marked as "positive examples" or "negative examples" to help the model define the target boundary more accurately. The SAM model accepts the mask as a dense prompt, which defines the approximate range of the target and guides the model to segment in a narrower area; by constructing a sparse depth prompt point set (providing target center point information) and a dense depth prompt mask (providing target overall outline information), the two are combined and input into the SAM model as prompt information. These prompt information helps the SAM model to more accurately segment the target object in the complex scene of the shielding door, optimizes its inference result, and improves the detection accuracy.

[0096] Based on the above description, specifically, the target detection method provided by the present application is used for target detection, and the optimization inference of the model is guided for target detection by designing a prompt adapter, specifically including:

[0097] The design of the prompt adapter is based on the combination of sparse depth prompt and dense depth prompt; the sparse prompt is used to provide the center point coordinates of the obstacle, and the dense prompt is used to provide the overall outline information of the obstacle.

[0098] ① Generation of sparse depth prompt:

[0099] Obtain the center point coordinates of the obstacle by detection algorithm . Then, the center point coordinates are expanded outward by 10 pixel points to form a sparse depth prompt point set }, which is represented as:

[0100] ;

[0101] Wherein, the acquisition of the center point coordinates of the obstacle includes:

[0102] Obtain the maximum horizontal coordinate, vertical coordinate and depth coordinate in the point cloud cluster, which is represented as: ;

[0103] Obtain the minimum horizontal coordinate, vertical coordinate and depth coordinate in the point cloud cluster, which is represented as: ;

[0104] Obstacle center coordinates , respectively, as:

[0105] Obstacle horizontal coordinates: ;

[0106] Obstacle vertical coordinates: ;

[0107] Obstacle depth coordinates: .

[0108] ② Generation of dense depth cues:

[0109] The SAM model can accept a mask as a dense cue, which defines the approximate range of the target and guides the model to segment in a narrower area; project the overall outline of the obstacle onto the two-dimensional image to generate a dense depth cue mask M, denoted as:

[0110] ;

[0111] The mask M is a binary image, where the obstacle area is marked as 1 and the background area is marked as 0.

[0112] ③ Optimization inference of SAM model:

[0113] The SAM model has an interface that accepts sparse and dense cues. The sparse depth cue point set { } and the dense depth cue mask M are input into the SAM model to optimize the inference result. The SAM model has strong generalization ability and can distinguish most objects with high detection accuracy; by combining these two depth cue information, the SAM model can generate more accurate and reasonable masks;

[0114] ;

[0115] where I represents the input image, { } represents the sparse depth cue point set, and M represents the dense depth cue mask.

[0116] The application significantly improves the segmentation accuracy of the SAM model by introducing sparse and dense hints. In particular, sparse hints with depth information are introduced into the detection process to guide the model to more accurately identify abnormal areas. After suspected abnormalities are detected by three-dimensional clustering, the SAM model is further confirmed to improve the accuracy and stability of the detection. At the same time, the combination of sparse and dense hints enables the SAM model to effectively deal with complex factors such as reflection and occlusion in the shield door scene, enhancing the detection robustness of the model in narrow or dim environments, thereby ensuring high-precision detection under complex conditions. In addition, since the SAM model is difficult to achieve real-time performance under long-time operation, the application performs preliminary detection by point cloud difference clustering, and does not need to call the SAM model when no abnormality is detected, reducing system load; when suspected abnormalities are found, the SAM model is called to further improve detection accuracy, ensuring the efficiency and accuracy of detection.

[0117] Based on the above embodiments, the target detection method provided by the application in the shield door scene is applied to actual rail transit operation, and the specific steps include:

[0118] S201: In a preset three-dimensional detection area, difference the front and rear frame point cloud data to preliminarily detect possible abnormalities; use a point cloud clustering algorithm to process the difference result to identify suspected abnormal clustering clusters;

[0119] Classify the effective target using a clustering algorithm:

[0120] Regarding each point as a class, calculate the minimum distance between each two classes ;

[0121] Merge the two classes with the minimum distance into a new class;

[0122] Recalculate the distance between the two classes;

[0123] Repeat the above two steps until there is no distance between any two classes that satisfies , then the clustering ends and multiple clustering clusters are obtained; when the number of point clouds in the clustering cluster exceeds the preset number of point clouds, it is judged that there is an abnormality, otherwise it is considered to be noise;

[0124] Referring to FIG. 1, it is a schematic diagram of a preset detection area; referring to FIG. 2, it is a point cloud clustering result diagram; Figure 2 Figure 3

[0125] S202: If there is no difference between the front and rear frames after the difference of the front and rear frame point cloud data of the preset three-dimensional detection area, it is considered that there is no abnormality, and the detection process ends;

[0126] ​​S203: If there is a difference between the front and rear frames after the difference between the front and rear frame point cloud data of the preset three-dimensional detection area is obtained, the center of the clustering cluster and the points around it are projected onto a two-dimensional image as sparse prompts; the overall contour information of the obstacle is combined to generate dense prompts; the sparse prompts and the dense prompts are input into the SAM model; the target area is segmented by using the SAM model, the segmentation effect is optimized by the sparse prompts and the dense prompts, and the segmentation result is output.

[0127] Referring to Figure 4 As shown in the SAM segmentation result diagram;

[0128] S204: If the segmentation area meets the preset abnormal label (for example, people, bags, umbrellas, etc.), the detection result and the abnormal position are output.

[0129] The embodiment accurately obtains the size and position of the anomaly by combining three-dimensional point cloud clustering detection and two-dimensional image segmentation, thereby evaluating the influence of the anomaly on driving safety and ensuring the reliability of the detection result. After detecting the suspected anomaly, the SAM model is guided by sparse and dense prompts, so that the reasoning accuracy of the SAM model in the complex scene of the shielding door is significantly improved, the segmentation effect is optimized, interference factors such as shielding and reflection are effectively dealt with, and the accuracy of the detection is improved.

[0130] The embodiment of the application also provides a target detection device in a shielding door scene, comprising:

[0131] A binocular camera is configured to collect left-eye images and right-eye images of the shielding door in an initial closing state and after the shielding door is closed during track transportation operation.

[0132] A target detection device in a shielding door scene is in communication connection with the binocular camera, and applies the target detection method in the shielding door scene as described above to obtain corresponding point cloud data based on the left-eye images and the right-eye images, to identify the center point, size and contour of the obstacle point cloud data.

[0133] A display device is in communication connection with the target detection device in the shielding door scene, and is configured to display a two-dimensional bounding box of an identified abnormal object and a category of the abnormal object.

[0134] Specifically, referring to Figure 5 and Figure 6 As shown in the image interface display diagram and the point cloud interface display diagram of the target detection device in the shielding door scene.

[0135] Based on the above embodiment, the performance of the data collected on the vehicle depot test platform based on the performance of the application is described, that is, the performance of the algorithm is verified by simulation experiments, and the specific embodiments are as follows:

[0136] In the test experiment, different types of test objects were set up, and the detection effect of obstacles was tested at 3 distances (60 cm, 120 cm, 180 cm). The specific contents are as follows:

[0137] The object of type T-ODS-F-A-BL-4 (class A test object black 4cm³ foam) was detected;

[0138] The design type is T-test report, the whole machine / component type is ODS-obstacle detection system, the report extension type is F-function, the test type is foam block WH, and the test size XX is 4cm*4cm*4cm, 6cm*6cm*6cm, 8cm*8cm*8cm;

[0139] The obstacle detection results are shown in Table 1:

[0140] Table 1 Obstacle detection results

[0141]

[0142] As can be seen from the table, using the foreign matter detection method based on double purposes can detect foreign matters while accurately positioning the positions of the foreign matters.

[0143] The target detection method in the shielding door scene provided by the application obtains the region of interest by differentiating the template point cloud in the preset detection region in the initial closing state of the shielding door from the real-time point cloud after the shielding door is closed in the operation process of rail transit; and constructs a sparse depth prompt point set and a dense depth prompt mask based on the region of interest, which are input into a SAM model together with the region of interest to perform instance segmentation and identify the obstacles. The application accurately obtains the size and position of the obstacles by combining three-dimensional point cloud clustering with two-dimensional image segmentation, further evaluates the driving safety of rail transit, and ensures the reliability of the detection result. At the same time, the application constructs a sparse depth prompt point set and a dense depth prompt mask after detecting a suspected anomaly to guide the SAM model; the sparse depth prompt point set is used to guide the SAM model to focus on the abnormal area and more accurately define the boundary of the obstacle, and the dense depth prompt mask defines the approximate range of the obstacle and can guide the SAM model to perform segmentation in a narrower area. The sparse depth prompt point set and the dense depth prompt mask eliminate point cloud noise and provide strong prior guidance for anomaly detection, so that the inference accuracy of the SAM model in the complex shielding door scene is significantly improved, the segmentation effect is optimized, interference factors such as occlusion and reflection are effectively addressed, the detection robustness of the model in narrow or dim environments is enhanced, and high-precision detection is still achieved under complex conditions. Moreover, the SAM model is difficult to achieve real-time performance under long-time operation, so the application performs preliminary detection by point cloud differential clustering, does not need to call the SAM model when no anomaly is detected, and reduces the system load; and calls the SAM model to further improve the detection accuracy when a suspected anomaly is found, ensuring the efficiency and accuracy of detection. Moreover, the application generates point cloud data based on the left-eye image and the right-eye image collected by the binocular camera, differentiates the point cloud data in different states, and can perform subsequent obstacle identification without increasing high-cost devices such as radars, is convenient to install and low in cost.

[0144] Those skilled in the art will understand that embodiments of the application can be provided as methods, systems, or computer program products. Therefore, the application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.

[0145] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 one or more flow or blocks

[0146] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 one or more flow or blocks

[0147] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 one or more flow or blocks

[0148] Obviously, the above-described embodiments are only examples and are not intended to limit the present application. Based on the above description, one of ordinary skill in the art can further make other variations and changes to the present application. Here, it is not necessary or possible to enumerate all the embodiments. The obvious variations and changes derived therefrom are still within the scope of the present application.

Claims

1. A method for target detection in a shielded door scene, characterized in that, The method comprises the following steps: In a three-dimensional coordinate system, the detection area where the shielding door is located is subjected to axis constraint to obtain a preset detection area; Point cloud data of the preset detection area of the shielding door in an initial closed state is obtained as a template point cloud; Current point cloud data of the preset detection area after the shielding door is closed during rail transit operation is collected in real time as real-time point cloud; The real-time point cloud and the template point cloud of the preset detection area are subjected to difference to obtain a difference result; the difference result is detected by using a point cloud clustering method to obtain a difference clustering point cloud, and it is determined whether an abnormal area exists; If the number of point clouds in the difference clustering point cloud exceeds a preset number of point clouds, it is considered that the preset detection area has an abnormal area, and an abnormal area in the real-time point cloud is obtained as a region of interest; Based on the point cloud data of the region of interest, an abnormal object center coordinate is acquired, a preset number of pixel points are extended outward from the abnormal object center coordinate, and a sparse depth hint point set is constituted, which is represented as: ; wherein, represents the coordinate of the i-th pixel point in the sparse depth hint point set, represents the preset number of pixel points extended outward, represents the abnormal object center coordinate; Point cloud data of the region of interest is projected onto a two-dimensional plane to generate a dense depth prompt mask; The region of interest and prompt information are input into a SAM model by taking the sparse depth prompt point set and the dense depth prompt mask as the prompt information, instance segmentation is performed on the region of interest, and a two-dimensional bounding box of an abnormal object and a class of the abnormal object are obtained.

2. The method of claim 1, wherein, The point cloud clustering method is used to detect the difference result to obtain a difference clustering point cloud, and it is determined whether an abnormal area exists. If the number of point clouds in the difference clustering point cloud does not exceed the preset number of point clouds, it is considered that the preset detection area does not have an abnormal area, and the detection is ended.

3. The method of claim 1, wherein, After the two-dimensional bounding box of the abnormal object is obtained, the two-dimensional bounding box of the abnormal object is mapped to a three-dimensional space to obtain a three-dimensional space coordinate of the abnormal object.

4. The method of claim 1, wherein, The point cloud data is obtained by using a binocular camera to collect left-eye images and right-eye images of the shielding door area; Stereo matching is performed on the left-eye images and the right-eye images to obtain left-eye and right-eye disparities; the depth coordinates of each pixel point are calculated according to the triangular similarity principle and the left-eye and right-eye disparities, and a depth image is generated; The left-eye images and the right-eye images are converted into point cloud data based on the horizontal and vertical coordinates of each pixel point in the left-eye images and the right-eye images and the corresponding depth coordinates. The depth coordinates of each pixel point are calculated according to the triangular similarity principle and the left-eye and right-eye disparities, and are expressed as:

5. The method of claim 4, wherein, The point cloud data of the region of interest is projected onto a two-dimensional plane to generate a dense depth prompt mask, which comprises: ； wherein, denotes a depth coordinate of a pixel point, denotes a focal length of the binocular camera, denotes a distance between the left camera and the right camera in the binocular camera, and denote a horizontal coordinate of a pixel point in the left camera and the right camera, respectively.

6. The method of claim 1, wherein, The point cloud clustering algorithm comprises a k-means clustering algorithm, a DBSCAN clustering algorithm, and a Euclidean clustering algorithm. The maximum horizontal coordinate, vertical coordinate and depth coordinate in the point cloud data of the region of interest are obtained, denoted as: ; The minimum horizontal coordinate, vertical coordinate and depth coordinate in the point cloud data of the region of interest are obtained, denoted as: ; Abnormal object center coordinates , respectively, as Abnormal object abscissa: ; Abnormal object ordinate: ; Abnormal object depth coordinate: .

7. The method of claim 1, wherein, The method comprises the following steps: abnormal object region marked as 1 and background region is marked as 0, dense depth hint mask is represented as: 。 8. The method of claim 1, wherein, A binocular camera is used to collect left-eye images and right-eye images of the shielding door in an initial closed state and after the shielding door is closed during rail transit operation; 9. A target detection device in a barrier gate scenario, characterized by A target detection device in a shielding door scene is in communication connection with the binocular camera, and the target detection device applies the target detection method in the shielding door scene according to any one of claims 1 to 8 to recognize the corresponding point cloud data based on the left-eye images and the right-eye images, and obtain the center point, size, and contour of the obstacle point cloud data; A display device is in communication connection with the target detection device in the shielding door scene, and is used to display the two-dimensional bounding box of the abnormal object and the class of the abnormal object recognized. ​ ​

Images