Intelligent patrol method and system for subway protection area

Abstract

The invention relates to an intelligent patrol method and system for a subway protection district, and the method comprises the steps: carrying out the preprocessing of a real-time image collected by an unmanned plane, carrying out the target detection of each frame of aerial photo in the preprocessed real-time image, and obtaining all target detection results; constructing an environment field corresponding to the moving target based on the bounding box of each moving target, performing environment feature extraction on the environment field, performing self feature extraction on each moving target, and combining the obtained self features with the target environment to generate final detection features; and matching the final detection features with known detection features in a database, carrying out duplicate removal on a target detection result according to a matching result, carrying out track prediction on the moving target, and generating alarm information according to an obtained track prediction result to give an alarm. Therefore, accurate duplicate removal and dynamic tracking of the moving target are realized, the accuracy and efficiency of patrol of the subway protection area are improved, and the safety of the subway protection area is improved.

Description

Technical Field

[0001] This invention relates to the field of rail transit management technology, and in particular to an intelligent inspection method and system for subway protection zones. Background Technology

[0002] Safety inspections within subway protection zones are a core aspect of rail transit operation and maintenance, directly impacting the structural safety and operational stability of subway lines. Currently, drone inspections are increasingly being used as a supplementary method. Drone aerial surveys identify and track moving targets within the subway protection zone, providing timely warnings of any intrusions.

[0003] When using drones for aerial surveying to identify moving targets, they mainly rely on the characteristics of the moving targets themselves. However, due to the special nature of subway protection zones, moving targets are often in a dynamic state, and their characteristics and positions change frequently. Relying solely on their own characteristics for identification often results in the same target being detected multiple times, affecting the accuracy and efficiency of the patrol and impacting the safety of the subway protection zone. Summary of the Invention

[0004] The technical problem to be solved by this invention is: to provide an intelligent patrol method and system for subway protection zones, which can achieve accurate deduplication and dynamic tracking of moving targets, improve the accuracy and efficiency of subway protection zone patrols, and improve the safety of subway protection zones.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: In a first aspect, the present invention provides an intelligent patrol method for subway protection zones, comprising: Real-time images of the subway protection zone are collected by drones, and the real-time images are preprocessed, including geometric correction and image registration, to obtain preprocessed real-time images. The target detection model is used to perform target detection on each frame of the pre-processed real-time image to obtain all target detection results. The target detection results include moving targets and bounding boxes of moving targets in the pre-processed real-time image. Based on the bounding box of each moving target, the corresponding environment domain of the moving target is constructed. The environment domain of each moving target is then used to extract environment features to obtain all target environment features. At the same time, the self-feature of each moving target is extracted to obtain all self-features. The self-features of each moving target are combined with the corresponding target environment features to generate the final detection features. The final detection feature is matched with known detection features in the database to obtain a matching result. Based on the matching result, all target detection results are deduplicated to obtain deduplicated target detection results. The trajectory of the moving target in the deduplicated target detection results is predicted to obtain a trajectory prediction result. An alarm message is generated based on the trajectory prediction result to issue an alarm.

[0006] The beneficial effects of this invention are as follows: Preprocessing real-time images of subway protection zones collected by UAVs through geometric correction and image registration eliminates distortion caused by UAV shooting angles and terrain undulations, improving the quality of the preprocessed real-time images. Utilizing the generalization ability of a pre-trained target detection model, target detection is performed on each frame of the preprocessed real-time images, reducing missed and false detections while improving target detection efficiency. Unlike traditional methods that rely solely on the moving target's own characteristics for identification, this invention constructs a unique environmental domain for each moving target and extracts environmental features. The obtained target environmental features are combined with the target's own features for deduplication; that is, the obtained target environmental features and the target's own features are used together as the identity identifier of the moving target. This overcomes the problem of frequent changes in the moving target's own characteristics and position, achieving accurate deduplication of moving targets. Furthermore, trajectory prediction is performed on the moving targets, and alarm information is generated based on the trajectory prediction results, enabling dynamic tracking of moving targets, improving the accuracy and efficiency of subway protection zone patrols, and enhancing the safety of the subway protection zone.

[0007] Optionally, constructing the environmental domain corresponding to each moving target based on its bounding box includes: Using the bounding box of each moving target as the center, draw an annular region with a preset radius to obtain the annular region of each moving target, and use the annular region as the environmental domain of the corresponding moving target; and / or Using the bounding box of each moving target as the center, expand it in four directions according to the preset pixel value to obtain the expanded bounding box. Use the expanded bounding box as the environment of the corresponding moving target. and / or For each moving target, the diagonal length ratio is calculated. The bounding box of each moving target is then expanded according to the corresponding diagonal length ratio to obtain the expanded bounding box. The expanded bounding box is used as the environmental domain of the corresponding moving target.

[0008] As described above, diverse environmental generation methods enhance scene adaptability. Drawing a ring-shaped region with a preset radius centered on the bounding box ensures a symmetrical distribution of the resulting environmental domain, unaffected by the shape of the moving target. This uniformly covers the 360° environmental space surrounding the moving target, preventing the omission of environmental features in a particular direction due to differences in the target's shape. Expanding the bounding box in four directions according to preset pixel values ​​eliminates the need for complex geometric calculations, reducing computational complexity and improving overall processing efficiency. Furthermore, the expanded bounding box focuses on the surrounding environment of the moving target, minimizing the influence of irrelevant environments. Expanding according to the diagonal length ratio of the bounding box allows for precise matching of the environmental domain with the actual duty cycle of the moving target, improving the correlation between the two. The diagonal length ratio expansion also better reflects the moving target's operational status, enhancing the realism of the resulting target environmental features.

[0009] Optionally, the target environment features include texture features and structural features, and the extraction of environmental features from the environment domain to obtain the target environment features includes: The real-time image corresponding to the environmental domain is converted into a grayscale image, the grayscale co-occurrence matrix of the grayscale image is calculated, and the contrast, energy value, entropy value and homogeneity are extracted from the grayscale co-occurrence matrix. The contrast, energy value, entropy value and homogeneity are converted into texture features that can describe the surface state through a large model. The texture features include soil, grass and concrete. Structural features were extracted from the grayscale image using the Canny edge detection method and the Hough transform method. The structural features included fences, subway road edges, and foundation pit ramps. Based on the texture features and the structural features, the target environment features are obtained.

[0010] As described above, environmental features include not only texture features but also structural features. When extracting texture features, the real-time image is first converted to grayscale to avoid color deviations caused by changes in lighting. Then, a large model is used to convert the obtained contrast, energy, entropy, and homogeneity values ​​into texture features that can describe the surface state, ensuring the accuracy and comprehensiveness of the obtained texture features. When extracting structural features, diverse extraction methods are employed. The Canny edge detection method can accurately filter noise interference and clearly extract structural features. The Hough transform method excels at converting discrete edge points into continuous geometric shapes, achieving a structured description of structural features and avoiding the inability to identify structural features due to edge breaks, thus improving the accuracy and comprehensiveness of the obtained structural features.

[0011] Optionally, the target environment features further include color features, and obtaining the target environment features based on the texture features and the structural features includes: The real-time image corresponding to the environmental domain is converted into an HSV color image. The color histogram of each channel in the HSV color image is calculated. All color histograms are stitched together to obtain color features. The color feature, texture feature, and structural feature are normalized respectively to obtain normalized color feature, normalized texture feature, and normalized structural feature. The normalized color feature, normalized texture feature, and normalized structural feature are then spliced ​​together to obtain the target environment feature.

[0012] As described above, the real-time conversion of the environmental domain into an HSV color image in advance is used for color feature extraction. This avoids the loss of color information due to the reliance of texture features on grayscale images. Furthermore, HSV color images have strong anti-interference capabilities, ensuring that the color features of the same environment remain stable under different time periods and lighting conditions, thereby improving the accuracy of the obtained color features.

[0013] Optionally, the target detection result includes the target type of the moving target, and the matching of the final detection feature with known detection features in the database includes: Calculate the cosine similarity between the final detected feature and the known detected features in the database, and determine whether the target type is an excavator. If the target type is an excavator, determine whether the cosine similarity exceeds the first similarity threshold. If yes, the match is successful; otherwise, the match fails. If the target type is not an excavator, then determine whether the target type is a dump truck. If the target type is a dump truck, then determine whether the cosine similarity exceeds the second similarity threshold. If yes, the match is successful; otherwise, the match fails. If the target type is not a dump truck, then determine whether the cosine similarity exceeds the third similarity threshold. If yes, the match is successful; otherwise, the match fails. Wherein, the first similarity threshold, the second similarity threshold, and the third similarity threshold are not the same, and the first similarity threshold is greater than the second similarity threshold.

[0014] As described above, different similarity thresholds are used to determine whether a match is successful for different target types. Differentiated similarity thresholds can effectively avoid false deduplication of high-risk moving targets, while avoiding false deduplication of low-risk moving targets, thus improving the accuracy of deduplication.

[0015] Optionally, the step of predicting the trajectory of the moving target in the deduplicated target detection result to obtain the trajectory prediction result includes: Obtain the movement position of each moving target in the time series, and connect the movement positions to generate the historical movement trajectory of each moving target; The historical motion trajectory of each moving target is input into a pre-trained trajectory prediction model to predict the trajectory and obtain the corresponding trajectory prediction result.

[0016] As described above, predicting the future trajectory of a moving target based on its historical trajectory ensures the objectivity and accuracy of the predicted trajectory.

[0017] Optionally, generating alarm information based on the trajectory prediction result and issuing an alarm includes: Obtain a preset risk grid map of the subway protection zone, and align the preset risk grid map with the preprocessed real-time image to a unified geographic coordinate system to obtain a fully perceptive subway map. Obtain the current location of the moving target, and obtain the current risk level from the fully-sensing subway map based on the current location; The risk trajectory of the moving target is obtained based on the trajectory prediction results and the full-sensory subway map, and the trajectory risk level is obtained based on the risk trajectory. The target type of the moving target is obtained from the target detection results, and the risk level of the type is obtained based on the target type; The risk type level, the current risk level, and the trajectory risk level are input into the risk calculation formula to calculate the risk, generate the current risk level, and generate alarm information based on the current risk level to issue an alarm. The risk calculation formula is: ; Where R represents the current level of risk, Indicates the type of risk level. The first weight representing the type of risk level that affects the outcome. Indicates the current risk level. This indicates the second weight that influences the current risk level. Indicates the risk level of the trajectory. This represents the third weight that influences the risk level of a trajectory.

[0018] As described above, constructing a multi-dimensional risk assessment system not only considers the type and level of risk and the current risk level, but also the risk movement trajectory. This avoids misjudgments based on a single dimension of risk while combining static assessment with dynamic prediction. The risk movement trajectory essentially incorporates future dynamic risks, compensating for the limitations of static assessment, improving the accuracy of the current risk level, and making the risk assessment more forward-looking. Furthermore, by constructing a fully perceptive subway map using a pre-set risk grid map and real-time images, the risk information of the subway protection zone is visualized, solving the problem that the risk level cannot be determined based solely on the current location.

[0019] Optionally, generating alarm information and issuing an alarm based on the current risk level includes: If the current risk level exceeds a first risk threshold, an emergency alarm is generated. If the current risk level does not exceed the first risk threshold, the system determines whether the current risk level exceeds a second risk threshold. If the current risk level exceeds the second risk threshold, a warning alarm is generated. If the current risk level does not exceed the second risk threshold, the system determines whether the current risk level exceeds a third risk threshold. If the current risk level exceeds the third risk threshold, a warning alarm is generated. If the current risk level does not exceed the third risk threshold, a notification alarm is generated. When issuing alarms, alarms are issued in the first alarm order, which is: urgent alarms > warning alarms > alarms of concern > alert alarms.

[0020] As described above, alarm classification and priority sorting are implemented, and alarm information is classified according to the current risk level. This avoids the problem of emergency and warning alarm information being overwhelmed due to indiscriminate alarms, as well as excessive interference from prompt alarm information, thereby improving the pertinence of emergency response and the timeliness of processing.

[0021] Secondly, the present invention provides an intelligent patrol system for a subway protection zone, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the intelligent patrol method for a subway protection zone described in the first aspect.

[0022] The technical effects of the intelligent inspection system for subway protection zones provided in the second aspect are the same as those of the intelligent inspection method for subway protection zones provided in the first aspect. Attached Figure Description

[0023] Figure 1 A flowchart illustrating an intelligent patrol method for a subway protection zone provided in this embodiment; Figure 2 This is a schematic diagram of the overall process of an intelligent inspection method for a subway protection zone provided in this embodiment; Figure 3 This is a schematic diagram of the structure of an intelligent patrol system for a subway protection zone provided in this embodiment.

[0024] Explanation of reference numerals in the attached figures 1. The structure of an intelligent patrol system for a subway protection zone; 2. Processor; 3. Memory. Detailed Implementation

[0025] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present invention can be understood more clearly and thoroughly, and that the scope of the present invention can be fully conveyed to those skilled in the art.

[0026] Example 1 Please refer to Figures 1 to 2 This invention provides an intelligent patrol method for subway protection zones, comprising the following steps: S1. Real-time images of the subway protection zone are collected by drone, and the real-time images are preprocessed, including geometric correction and image registration, to obtain preprocessed real-time images. In this embodiment, as Figure 2 As shown, real-time images of the subway protection zone are collected by drones. The drones will collect images according to a preset flight path. Each frame of the real-time images carries information such as GPS coordinates, shooting timestamp, flight altitude and camera attitude. Geometric correction and image registration are performed on the real-time images to eliminate distortion caused by changes in drone attitude, resulting in pre-processed real-time images.

[0027] S2. Target detection is performed on each frame of aerial photograph in the pre-processed real-time image using a pre-trained target detection model to obtain all target detection results. The target detection results include moving targets and bounding boxes of moving targets in the pre-processed real-time image. In this embodiment, as Figure 2As shown, pre-trained target detection models, such as YOLO target detection model and Faster R-CNN target detection model, are used to detect targets in each frame of aerial photographs in the pre-processed real-time image, and all target detection results are obtained. The target detection results include, but are not limited to: moving target, temporary ID of moving target, target type of moving target, confidence of moving target, and bounding box of moving target in pre-processed real-time image.

[0028] S3. Construct the corresponding environment domain of each moving target based on the bounding box of each moving target, extract the environment features of each moving target environment domain to obtain all target environment features, and extract the self-features of each moving target to obtain all self-features. Combine the self-features of each moving target with the corresponding target environment features to generate the final detection features. In this embodiment, as Figure 2 As shown, an environmental domain is constructed for each moving target, and environmental features are extracted from the environmental domain. The obtained target environmental features are combined with the target's own features to generate the final detection features. At this point, the final detection features serve as the unique identifier of the moving target.

[0029] At this point, the construction of the corresponding environment domain for each moving target based on the bounding box of each moving target in step S3 includes: S30. Using the bounding box of each moving target as the center, draw an annular area with a preset radius to obtain the annular area of ​​each moving target, and use the annular area as the environmental domain of the corresponding moving target. and / or S31. Taking the bounding box of each moving target as the center, expand it in four directions according to the preset pixel value to obtain the bounding box after expansion in four directions. Use the bounding box after expansion in four directions as the environment of the corresponding moving target. and / or S32. Calculate the diagonal length ratio of the bounding box of each moving target to obtain the diagonal length ratio of each moving target. Expand the bounding box of each moving target according to the corresponding diagonal length ratio to obtain the bounding box after diagonal length ratio expansion. Use the bounding box after diagonal length ratio expansion as the environmental domain of the corresponding moving target.

[0030] In this embodiment, the environmental domain can be constructed in different ways: First, a ring-shaped area is drawn centered on the bounding box of each moving target, with a preset radius of 3 meters, and the resulting ring-shaped area is taken as the environmental domain. Second, the bounding box of each moving target is expanded in four directions according to a preset pixel value, with the preset pixel value being [30, 50] pixels, and the expanded bounding box is taken as the environmental domain. Third, the diagonal length ratio of the bounding box of each moving target is calculated, and the bounding box of each moving target is expanded according to the corresponding diagonal length ratio, and the expanded bounding box is taken as the environmental domain.

[0031] At this point, the target environment features mentioned in step S3 include texture features and structural features, and the environmental feature extraction of the environmental domain to obtain the target environment features includes: S33. Convert the real-time image corresponding to the environmental domain into a grayscale image, calculate the grayscale co-occurrence matrix of the grayscale image, and extract contrast, energy value, entropy value and homogeneity from the grayscale co-occurrence matrix. Convert the contrast, energy value, entropy value and homogeneity into texture features that can describe the surface state through a large model. The texture features include soil, grass and concrete. S34. Structural features are extracted from the grayscale image using the Canny edge detection method and the Hough transform method. The structural features include fences, subway road edges, and foundation pit ramps. S35. Based on the texture features and the structural features, the target environment features are obtained.

[0032] At this time, the target environment features also include color features, and step S35 includes: S351. Convert the real-time image corresponding to the environmental domain into an HSV color image, calculate the color histogram of each channel in the HSV color image, and stitch all the color histograms together to obtain color features. S352. Normalize the color feature, texture feature, and structural feature respectively to obtain normalized color feature, normalized texture feature, and normalized structural feature. Then, splice the normalized color feature, normalized texture feature, and normalized structural feature to obtain the target environment feature.

[0033] In this embodiment, as Figure 2As shown, the real-time environmental domain corresponding to the image is pre-converted into a grayscale image. The grayscale co-occurrence matrix (GCMM) of the grayscale image is calculated, and contrast, energy, entropy, and homogeneity are extracted from the GCMM to convert it into texture features that can describe the surface state through a large model. At the same time, structural features are extracted from the grayscale image using the Canny edge detection method and the Hough transform method. The real-time image corresponding to the environmental domain is converted into an HSV color image. The color histograms calculated for each channel are stitched together to obtain color features. The color features, texture features, and structural features are normalized respectively, with L2 norm normalization used. The normalized color features, normalized texture features, and normalized structural features are stitched together to obtain the target environmental features. Dimensionality reduction techniques such as principal component analysis are used to reduce the dimensionality of the target environmental features to obtain the final target environmental features.

[0034] S4. Match the final detection feature with the known detection features in the database to obtain a matching result. Based on the matching result, deduplicate all the target detection results to obtain deduplicated target detection results. Perform trajectory prediction on the moving targets in the deduplicated target detection results to obtain trajectory prediction results. Generate alarm information based on the trajectory prediction results to issue an alarm.

[0035] In this embodiment, as Figure 2 As shown, the database stores the target environment features and self-features of known moving targets, i.e., known detection features. The final detection features are matched with the known detection features in the database. Based on the matching results, the target detection results are deduplicated to avoid repeated detection of the same moving target, resulting in deduplicated target detection results. The trajectory of the moving targets in the deduplicated target detection results is predicted, and alarm information is generated based on the obtained trajectory prediction results.

[0036] At this point, the target detection result in step S4 includes the target type of the moving target, and matching the final detection feature with known detection features in the database includes: S40. Calculate the cosine similarity between the final detected feature and the known detected features in the database, and determine whether the target type is an excavator. If the target type is an excavator, determine whether the cosine similarity exceeds the first similarity threshold. If yes, the match is successful; otherwise, the match fails. S41. If the target type is not an excavator, then determine whether the target type is a dump truck. If the target type is a dump truck, then determine whether the cosine similarity exceeds the second similarity threshold. If yes, the match is successful; otherwise, the match fails. S42. If the target type is not a dump truck, determine whether the cosine similarity exceeds the third similarity threshold. If yes, the match is successful; otherwise, the match fails. Wherein, the first similarity threshold, the second similarity threshold, and the third similarity threshold are not the same, and the first similarity threshold is greater than the second similarity threshold.

[0037] In this embodiment, when matching the target detection result with known detection features in the database, the cosine similarity between the two is calculated for judgment. Different similarity thresholds are used for different target types. When the target type is an excavator, if the cosine similarity exceeds the first similarity threshold, the match is successful; otherwise, the match fails. When the target type is a dump truck, if the cosine similarity exceeds the second similarity threshold, the match is successful; otherwise, the match fails. When the target type is neither an excavator nor a dump truck, if the cosine similarity exceeds the third similarity threshold, the match is successful; otherwise, the match fails. Furthermore, the first, second, and third similarity thresholds are all different. Considering that the surrounding environment of an excavator generally does not change much when it is moving, while the surrounding environment of a dump truck usually moves along the subway line, where the surrounding environment changes significantly, the first similarity threshold is greater than the second similarity threshold.

[0038] At this point, the trajectory prediction of the moving target in the deduplicated target detection result in step S4, to obtain the trajectory prediction result, includes: S43. Obtain the movement position of each moving target in the time series, and connect the movement positions to generate the historical movement trajectory of each moving target; S44. Input the historical motion trajectory of each moving target into the pre-trained trajectory prediction model to predict the trajectory and obtain the corresponding trajectory prediction result.

[0039] In this embodiment, as Figure 2 As shown, the movement position of each moving target in the time series is obtained, and the movement positions are spliced ​​to generate a historical movement trajectory. Based on the historical movement trajectory and the pre-trained trajectory prediction model, the trajectory of the moving target is predicted to obtain the trajectory prediction result.

[0040] At this point, step S4, which involves generating alarm information based on the trajectory prediction result, includes: S45. Obtain the preset risk grid map of the subway protection zone, and align the preset risk grid map with the preprocessed real-time image to a unified geographic coordinate system to obtain a fully perceptive subway map. S46. Obtain the current location of the moving target, and obtain the current risk level from the fully-perceived subway map based on the current location; S47. Obtain the risk movement trajectory of the moving target based on the trajectory prediction result and the full-sensing subway map, and obtain the trajectory risk level based on the risk movement trajectory; S48. Obtain the target type of the moving target from the target detection results, and obtain the type risk level based on the target type; S49. Input the type risk level, the current risk level, and the trajectory risk level into the risk calculation formula to calculate the risk, generate the current risk level, and generate alarm information based on the current risk level to issue an alarm. The risk calculation formula is: ; Where R represents the current level of risk, Indicates the type of risk level. The first weight representing the type of risk level that affects the outcome. Indicates the current risk level. This indicates the second weight that influences the current risk level. Indicates the risk level of the trajectory. This represents the third weight that influences the risk level of a trajectory.

[0041] In this embodiment, as Figure 2 As shown, a preset risk grid map of the subway protection zone is obtained. This preset risk grid map not only includes the geographical information of the subway protection zone but also the gridding and risk level classification of the zone. For example, the area directly above the subway tunnel structure is defined as a "Level 1 Risk Zone: High Risk Zone," and the surrounding area is defined as a "Level 2 Risk Zone: Medium Risk Zone." The preset risk grid map is aligned with the pre-processed real-time image to a unified geographic coordinate system, resulting in a fully-perceived subway map. The current position of the moving target is obtained. Based on the position of the current target on the fully-perceived subway map, the current risk level can be determined. Based on the trajectory prediction results combined with the fully-perceived subway map, the risk movement trajectory of the moving target can be obtained, i.e., whether it moves from a high-risk area to a low-risk area or from a low-risk area to a high-risk area, thus enabling the determination of the risk level. The trajectory risk level is obtained from the risk movement trajectory, and the type risk level is obtained from the target type of the moving target obtained from the target detection results. The type risk level is preset according to the target type, and the trajectory risk level is preset according to the pattern of the risk movement trajectory. For example: when the risk movement trajectory is from a high-risk area to a low-risk area, the trajectory risk level is 2; when the risk movement trajectory is from a low-risk area to a high-risk area, the trajectory risk level is 10; when the risk movement trajectory is staying in a high-risk area, the trajectory risk level is 8. When the target type is an excavator, the type risk level is 10; when the target type is a loader, the type risk level is 8; when the target type is a dump truck, the type risk level is 5; and when the target type is a person, the type risk level is 3.

[0042] Input the type risk level, current risk level, and trajectory risk level into the risk calculation formula to calculate the risk and obtain the current risk level. Then, generate alarm information based on the current risk level to issue an alarm.

[0043] At this point, step S49, which involves generating alarm information based on the current risk level, includes: S491. Determine whether the current risk level exceeds a first risk threshold. If the current risk level exceeds the first risk threshold, generate an emergency alarm message. If the current risk level does not exceed the first risk threshold, determine whether the current risk level exceeds a second risk threshold. If the current risk level exceeds the second risk threshold, generate a warning alarm message. If the current risk level does not exceed the second risk threshold, determine whether the current risk level exceeds a third risk threshold. If the current risk level exceeds the third risk threshold, generate a warning alarm message. If the current risk level does not exceed the third risk threshold, generate a prompt alarm message. S492. When issuing alarms, alarms shall be issued in the first alarm order, which is: urgent alarms > warning alarms > alarms of concern > alert alarms.

[0044] In this embodiment, different alarm messages are generated and different alarm processing is performed for different current risk levels. When the current risk level exceeds the first risk threshold, an urgent alarm message is generated, and the urgent alarm message is issued first. When the current risk level does not exceed the first risk threshold but exceeds the second risk threshold, a warning alarm message is generated, which is the next best priority. That is, the urgent alarm message is issued first, followed by the warning alarm message. When the current risk threshold does not exceed the second risk threshold but exceeds the third risk threshold, a concern alarm message is generated, which has a lower priority than the warning alarm message. That is, the urgent alarm message is issued first, followed by the warning alarm message, and then the concern alarm message. When the current risk threshold does not exceed the third risk threshold, a prompt alarm message is generated, and the prompt alarm message is issued last. In other words, the alarm messages are issued in the first alarm order, which is: urgent alarm message > warning alarm message > concern alarm message > prompt alarm message. Furthermore, different alarm messages will be displayed in different colors, and different alarm messages will be handled using different solutions.

[0045] Example 2 Please refer to Figure 3 The present invention provides an intelligent patrol system 1 for a subway protection zone, including a memory 3, a processor 2, and a computer program stored on the memory 3 and run on the processor 2. When the processor 2 executes the computer program, it implements the steps in Embodiment 1.

[0046] Since the systems / devices described in the above embodiments of the present invention are systems / devices used to implement the methods of the above embodiments of the present invention, those skilled in the art can understand the specific structure and modifications of the systems / devices based on the methods described in the above embodiments of the present invention, and therefore will not be repeated here. All systems / devices used in the methods of the above embodiments of the present invention fall within the scope of protection of the present invention.

[0047] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0048] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions.

[0049] It should be noted that any reference numerals placed between parentheses in the claims should not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In claims that enumerate several means, several of these means may be embodied by the same hardware. The use of the terms first, second, third, etc., is merely for convenience of expression and does not indicate any order. These terms can be understood as part of the component names.

[0050] Furthermore, it should be noted that in the description of this specification, the terms "one embodiment," "some embodiments," "embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0051] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the claims should be interpreted to include both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0052] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, then this invention should also include these modifications and variations.

Claims

1. A smart patrol method for subway protection zones, characterized in that, include: Real-time images of the subway protection zone are collected by drones, and the real-time images are preprocessed, including geometric correction and image registration, to obtain preprocessed real-time images. The target detection model is used to perform target detection on each frame of the pre-processed real-time image to obtain all target detection results. The target detection results include moving targets and bounding boxes of moving targets in the pre-processed real-time image. Based on the bounding box of each moving target, the corresponding environment domain of the moving target is constructed. The environment domain of each moving target is then used to extract environment features to obtain all target environment features. At the same time, the self-feature of each moving target is extracted to obtain all self-features. The self-features of each moving target are combined with the corresponding target environment features to generate the final detection features. The final detection feature is matched with known detection features in the database to obtain a matching result. Based on the matching result, all target detection results are deduplicated to obtain deduplicated target detection results. The trajectory of the moving target in the deduplicated target detection results is predicted to obtain a trajectory prediction result. An alarm message is generated based on the trajectory prediction result to issue an alarm.

2. The intelligent patrol method for a subway protection zone as described in claim 1, characterized in that, The construction of the environmental domain corresponding to each moving target based on the bounding box of each moving target includes: Using the bounding box of each moving target as the center, draw an annular region with a preset radius to obtain the annular region of each moving target, and use the annular region as the environmental domain of the corresponding moving target; and / or Using the bounding box of each moving target as the center, expand it in four directions according to the preset pixel value to obtain the expanded bounding box. Use the expanded bounding box as the environment of the corresponding moving target. and / or For each moving target, the diagonal length ratio is calculated. The bounding box of each moving target is then expanded according to the corresponding diagonal length ratio to obtain the expanded bounding box. The expanded bounding box is used as the environmental domain of the corresponding moving target.

3. The intelligent patrol method for a subway protection zone as described in claim 1, characterized in that, The target environment features include texture features and structural features. The extraction of environmental features from the environmental domain to obtain the target environment features includes: The real-time image corresponding to the environmental domain is converted into a grayscale image, the grayscale co-occurrence matrix of the grayscale image is calculated, and the contrast, energy value, entropy value and homogeneity are extracted from the grayscale co-occurrence matrix. The contrast, energy value, entropy value and homogeneity are converted into texture features that can describe the surface state through a large model. The texture features include soil, grass and concrete. Structural features were extracted from the grayscale image using the Canny edge detection method and the Hough transform method. The structural features included fences, subway road edges, and foundation pit ramps. Based on the texture features and the structural features, the target environment features are obtained.

4. The intelligent patrol method for a subway protection zone as described in claim 3, characterized in that, The target environment features also include color features, and obtaining the target environment features based on the texture features and the structural features includes: The real-time image corresponding to the environmental domain is converted into an HSV color image. The color histogram of each channel in the HSV color image is calculated. All color histograms are stitched together to obtain color features. The color feature, texture feature, and structural feature are normalized respectively to obtain normalized color feature, normalized texture feature, and normalized structural feature. The normalized color feature, normalized texture feature, and normalized structural feature are then spliced ​​together to obtain the target environment feature.

5. The intelligent patrol method for a subway protection zone as described in claim 1, characterized in that, The target detection result includes the target type of the moving target, and the matching of the final detection feature with known detection features in the database includes: Calculate the cosine similarity between the final detected feature and the known detected features in the database, and determine whether the target type is an excavator. If the target type is an excavator, determine whether the cosine similarity exceeds the first similarity threshold. If yes, the match is successful; otherwise, the match fails. If the target type is not an excavator, then determine whether the target type is a dump truck. If the target type is a dump truck, then determine whether the cosine similarity exceeds the second similarity threshold. If yes, the match is successful; otherwise, the match fails. If the target type is not a dump truck, then determine whether the cosine similarity exceeds the third similarity threshold. If yes, the match is successful; otherwise, the match fails. Wherein, the first similarity threshold, the second similarity threshold, and the third similarity threshold are not the same, and the first similarity threshold is greater than the second similarity threshold.

6. The intelligent patrol method for a subway protection zone as described in claim 1, characterized in that, The process of predicting the trajectory of the moving target in the deduplicated target detection result to obtain the trajectory prediction result includes: Obtain the movement position of each moving target in the time series, and connect the movement positions to generate the historical movement trajectory of each moving target; The historical motion trajectory of each moving target is input into a pre-trained trajectory prediction model to predict the trajectory and obtain the corresponding trajectory prediction result.

7. The intelligent patrol method for a subway protection zone as described in claim 1, characterized in that, The step of generating alarm information and issuing an alarm based on the trajectory prediction result includes: Obtain a preset risk grid map of the subway protection zone, and align the preset risk grid map with the preprocessed real-time image to a unified geographic coordinate system to obtain a fully perceptive subway map. Obtain the current location of the moving target, and obtain the current risk level from the fully-sensing subway map based on the current location; The risk trajectory of the moving target is obtained based on the trajectory prediction results and the full-sensory subway map, and the trajectory risk level is obtained based on the risk trajectory. The target type of the moving target is obtained from the target detection results, and the risk level of the type is obtained based on the target type; The risk type level, the current risk level, and the trajectory risk level are input into the risk calculation formula to calculate the risk, generate the current risk level, and generate alarm information based on the current risk level to issue an alarm. The risk calculation formula is: ； Where R represents the current level of risk, Indicates the type of risk level. The first weight representing the type of risk level that affects the outcome. Indicates the current risk level. This indicates the second weight that influences the current risk level. Indicates the risk level of the trajectory. This represents the third weight that influences the risk level of a trajectory.

8. The intelligent patrol method for a subway protection zone as described in claim 7, characterized in that, The step of generating alarm information and issuing an alarm based on the current risk level includes: If the current risk level exceeds a first risk threshold, an emergency alarm is generated. If the current risk level does not exceed the first risk threshold, the system determines whether the current risk level exceeds a second risk threshold. If the current risk level exceeds the second risk threshold, a warning alarm is generated. If the current risk level does not exceed the second risk threshold, the system determines whether the current risk level exceeds a third risk threshold. If the current risk level exceeds the third risk threshold, a warning alarm is generated. If the current risk level does not exceed the third risk threshold, a notification alarm is generated. When issuing alarms, alarms are issued in the first alarm order, which is: urgent alarms > warning alarms > alarms of concern > alert alarms.

9. An intelligent patrol system for a subway protection zone, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 8.

Images