Railway foreign object alarm method, device, equipment and storage medium

Abstract

The application relates to a railway foreign matter alarm method, device and equipment and a storage medium, and relates to the technical field of image processing. The method comprises the following steps: acquiring a to-be-detected image comprising a railway; determining a to-be-detected region comprising the railway in the to-be-detected image according to the to-be-detected image and a semantic segmentation model; in the case where the number of to-be-detected regions is one, judging whether a target foreign matter exists in the to-be-detected region according to the to-be-detected image and a foreign matter classification model; the foreign matter classification model is used for detecting foreign matters of a preset type in the to-be-detected image, and the type of the target foreign matter is a type other than a train in the preset type; and in the case where the target foreign matter exists in the to-be-detected region, an alarm is given. The application is used for improving the accuracy of railway foreign matter detection.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, device, equipment and storage medium for alarming foreign objects on railways. Background Technology

[0002] Currently, artificial intelligence technology is developing rapidly, and applying AI to railway foreign object detection has become a common practice. Specifically, railway foreign object detection can be performed using the following two methods: 1. Using deep learning models to detect the types of railway foreign objects; 2. Using background difference models for railway foreign object detection.

[0003] However, due to the limited types of foreign objects that can be classified using deep learning models for railway foreign object detection, false negatives can occur. Furthermore, weather changes can lead to false positives in background difference models. Therefore, improving the accuracy of railway foreign object detection is a pressing technical problem that needs to be solved. Summary of the Invention

[0004] This application provides a method, apparatus, device, and storage medium for alarming foreign objects on railways, to at least solve the problem of inaccurate detection of foreign objects on railways in the prior art. The technical solution of this application is as follows:

[0005] According to a first aspect of this application, a method for alarming foreign objects on railways is provided. The method includes: acquiring an image to be detected that includes a railway; determining a region to be detected that includes a railway in the image to be detected based on the image to be detected and a semantic segmentation model; if there is only one region to be detected, determining whether a target foreign object exists in the region to be detected based on the image to be detected and a foreign object classification model; the foreign object classification model is used to detect foreign objects of a preset type in the image to be detected, and the type of the target foreign object is a type other than a train in the preset types; and issuing an alarm if a target foreign object exists in the region to be detected.

[0006] In one possible implementation, the method further includes issuing an alarm when there are multiple areas to be detected and the detected areas are not continuous.

[0007] In one possible implementation, the method further includes: if it is determined that there are no foreign objects of a preset type in the detection area, determining the background difference result of the video to be detected based on the video to be detected and the background difference model; the video to be detected includes multiple images, the multiple images include the image to be detected, and the background difference result is used to characterize the changes of the multiple images; if the background difference result meets preset conditions, an alarm is issued; the preset conditions include: the background difference result is greater than or equal to a first threshold, the duration of the background difference result is greater than or equal to a second threshold, the background difference result is within the detection area, and the type of the background difference result is not a preset safety type.

[0008] According to a second aspect of this application, a railway foreign object alarm device is provided. The device includes an acquisition unit, a determination unit, and an alarm unit. The acquisition unit is used to acquire a detection image including a railway. The determination unit is used to determine, after the acquisition unit acquires the detection image including a railway, a detection region including a railway in the detection image based on the detection image and a semantic segmentation model. The determination unit is further used to determine whether a target foreign object exists in the detection region when there is only one detection region, based on the detection image and a foreign object classification model. The foreign object classification model is used to detect foreign objects of a preset type in the detection image, and the type of the target foreign object is a type other than a train among the preset types. The alarm unit is used to issue an alarm when a target foreign object exists in the detection region.

[0009] In one possible implementation, the alarm unit is specifically used to issue an alarm when there are multiple areas to be detected and the detected areas are not continuous.

[0010] In one possible implementation, the determining unit is further configured to, when it is determined that there are no foreign objects of a preset type in the area to be detected, determine the background difference result of the video to be detected based on the video to be detected and the background difference model; the video to be detected includes multiple images, the multiple images include the image to be detected, and the background difference result is used to characterize the changes of the multiple images; the alarm unit is further configured to issue an alarm when the background difference result meets preset conditions; the preset conditions include: the background difference result is greater than or equal to a first threshold, the duration of the background difference result is greater than or equal to a second threshold, the background difference result is located in the area to be detected, and the type of the background difference result is not a preset security type.

[0011] According to a third aspect of this application, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute instructions to implement the method of the first aspect described above and any possible implementation thereof.

[0012] According to a fourth aspect of this application, a computer-readable storage medium is provided, wherein when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the methods described in the first aspect and any possible implementation thereof.

[0013] According to a fifth aspect of this application, a computer program product is provided, comprising computer instructions that, when executed on an electronic device, cause the electronic device to perform the method described in the first aspect and any possible implementation thereof.

[0014] The technical solution of the first aspect of this application brings at least the following beneficial effects: Existing technologies typically detect railway foreign objects by employing either deep learning models or background subtraction models, but neither of these methods can accurately detect railway foreign objects. This application obtains the detection region of the image to be detected based on the image including the railway and a semantic segmentation model. If there are multiple discontinuous detection regions, it indicates the presence of a large foreign object in the image, triggering an alarm. If there is only one detection region, it further determines whether a target foreign object exists within the detection region based on the image and a foreign object classification model. If a target foreign object is present within the detection region, an alarm is triggered. Therefore, this application detects large foreign objects using a semantic segmentation model. Even when no large foreign object is present in the image, the foreign object classification model further determines whether a target foreign object, such as a person, branch, or stone, exists in the detection region. This avoids false alarms caused by changes in the detection region due to camera movement or PTZ camera movement, thus improving the accuracy of railway foreign object detection.

[0015] It should be noted that the technical effects of any of the implementation methods in aspects two through five can be found in the technical effects of the corresponding implementation methods in aspect one, and will not be repeated here.

[0016] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application, and do not constitute an undue limitation of this application.

[0018] Figure 1 This is a schematic diagram illustrating a railway foreign object alarm system according to an exemplary embodiment;

[0019] Figure 2 This is a flowchart illustrating a method for alarming foreign objects on a railway, according to an exemplary embodiment;

[0020] Figure 3 This is a schematic diagram illustrating a detection area of ​​one according to an exemplary embodiment;

[0021] Figure 4 This is a schematic diagram illustrating, according to an exemplary embodiment, that the number of regions to be detected is multiple and that the multiple regions to be detected are discontinuous;

[0022] Figure 5This is a flowchart illustrating yet another method for alarming foreign objects on a railway, according to an exemplary embodiment;

[0023] Figure 6 This is a schematic diagram illustrating the difference result according to an exemplary embodiment;

[0024] Figure 7 This is a block diagram illustrating a foreign object alarm device for railways according to an exemplary embodiment;

[0025] Figure 8 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0026] To enable those skilled in the art to better understand the technical solutions of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0027] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0028] Before providing a detailed introduction to the railway foreign object alarm method provided in this application, a brief introduction to the implementation environment (implementation architecture) involved in this application will be given first.

[0029] The alarm method for foreign objects on railways provided in this embodiment of the invention can be applied to alarm systems for foreign objects on railways. Figure 1 A schematic diagram of a foreign object alarm system for railway tracks is shown. Figure 1 As shown, the railway foreign object alarm system 10 includes an electronic device 11 and a railway foreign object alarm device 12. The electronic device 11 is connected to the railway foreign object alarm device 12. The electronic device 11 and the railway foreign object alarm device 12 can be connected by wired or wireless means, and this embodiment of the invention does not limit the connection.

[0030] Electronic device 11 can be used to acquire an image of the railway to be detected and send the image of the railway to an alarm device 12 for foreign objects on the railway.

[0031] In one scenario, electronic device 11 receives the video to be detected captured and sent by a camera, and acquires images of the railway to be detected by setting the image acquisition speed, such as 1 image per second.

[0032] In another scenario, electronic device 11 acquires the video to be detected, and by setting the image acquisition speed, such as 1 image per second, it acquires the image to be detected, including the railway.

[0033] The railway foreign object alarm device 12 can be used to receive the image to be detected sent by the electronic device 11.

[0034] The railway foreign object alarm device 12 can also be used to process the received image to be detected. For example, based on the image to be detected and a semantic segmentation model, it can determine the region to be detected that includes a railway. When there is only one region to be detected, it can determine whether a target foreign object exists within the region based on the image to be detected and a foreign object classification model. If a target foreign object exists within the region, an alarm is triggered.

[0035] In the first scenario, the electronic device 11 in the aforementioned railway foreign object alarm system 10 can be a server for processing data or performing calculations, or a server cluster consisting of multiple servers. In this case, the electronic device 11 can be connected to a surveillance camera, a video camera, a terminal with shooting function, etc.

[0036] In the second scenario, the electronic device 11 in the aforementioned railway foreign object alarm system 10 can be a user's personal terminal, such as a mobile phone, tablet computer, handheld computer, personal computer (PC), wearable device, smart TV, etc. In this case, the electronic device 11 is a terminal with shooting function, such as a surveillance camera, camcorder, etc.

[0037] In different application scenarios, the electronic device 11 and the railway foreign object alarm device 12 can be independent devices or integrated into the same device. This embodiment of the invention does not make specific limitations on this.

[0038] When electronic device 11 and railway foreign object alarm device 12 are integrated into the same device, the data transmission method between electronic device 11 and railway foreign object alarm device 12 is the data transmission between modules within the device. In this case, the data transmission process between the two is the same as the data transmission process when electronic device 11 and railway foreign object alarm device 12 are independent of each other.

[0039] In the following embodiments provided by the present invention, the present invention is illustrated by taking the integration of electronic device 11 and railway foreign object alarm device 12 into the same device as an example.

[0040] For ease of understanding, the alarm method for foreign objects on railways provided in this application will be described in detail below with reference to the accompanying drawings.

[0041] Figure 2 This is a flowchart illustrating a method for detecting foreign objects in railway equipment according to an exemplary embodiment. This method can be applied to electronic devices or to railway foreign object detection devices connected to electronic devices. Furthermore, this method can also be applied to devices similar to electronic devices or railway foreign object detection devices. The following description uses the application of this method to an electronic device as an example to illustrate the method. Figure 2 As shown, the alarm method for foreign objects on the railway includes the following steps:

[0042] S201. Obtain the image to be detected, including the railway.

[0043] As one possible implementation, electronic devices receive images of the railway to be detected, captured and transmitted by surveillance cameras.

[0044] For example, a surveillance camera monitors an area along a railway line and sends images of the railway to be detected to electronic devices.

[0045] As another possible implementation, electronic devices acquire images of the railway to be inspected.

[0046] For example, electronic devices monitor areas along railway lines and collect images of the railway to be detected.

[0047] S202. Based on the image to be detected and the semantic segmentation model, determine the regions in the image to be detected that include railways.

[0048] As one possible implementation, the electronic device inputs a training dataset into a semantic segmentation model to obtain a trained semantic segmentation model. Further, the electronic device inputs the image to be detected into the trained semantic segmentation model, and identifies the railway region in the image output by the semantic segmentation model as the region to be detected in the image.

[0049] For example, real railway images captured by surveillance cameras along the railway line are manually collected, railway regions in the real railway images are labeled, and a training dataset is created. An electronic device inputs the training dataset into an untrained semantic segmentation model to obtain a trained semantic segmentation model. Further, the electronic device inputs railway images into the trained semantic segmentation model to determine the railway regions in the railway images.

[0050] It should be noted that the semantic segmentation model is used to process the input image to be detected and output the railway region and segmentation region corresponding to the image to be detected.

[0051] The area to be detected can be one or more non-contiguous railway areas. When there is only one area to be detected, such as... Figure 3 As shown, this indicates that there are no large foreign objects in the area to be inspected, which can be understood as the railway area. When there are multiple areas to be inspected that are not consecutive, such as... Figure 4 As shown, the image to be detected contains three regions: region 1, region 2, and region 3. These regions are discontinuous, indicating the presence of a large foreign object. It can be understood that regions 1 and 3 represent the railway area, while region 2 is the area occupied by the fallen tree. The large foreign object could be a fallen tree or a railway fence blown down by the wind.

[0052] Understandably, the semantic segmentation model in this application is used to acquire railway regions in railway images. When large foreign objects such as fallen trees or wind-blown railway fences exist in a railway region, the railway region is segmented into multiple discontinuous railway regions by these large foreign objects. Furthermore, the railway regions obtained by inputting the railway image into the semantic segmentation model are also multiple discontinuous railway regions. When no large foreign objects exist in a railway region, the railway region is a continuous railway region, and the railway region obtained by inputting the railway image into the semantic segmentation model is also a continuous railway region. Therefore, the semantic segmentation model in this application can detect the presence of large foreign objects in the image to be detected and acquire the region to be detected, so as to subsequently determine whether other types of foreign objects exist in the image to be detected.

[0053] S203. When there is only one region to be detected, determine whether there is a target foreign object in the region to be detected based on the image to be detected and the foreign object classification model.

[0054] Among them, the foreign object classification model is used to detect foreign objects of a preset type in the image to be detected. The type of the target foreign object is the type other than the train among the preset types.

[0055] As one possible implementation, when there is only one region to be detected, the electronic device inputs the image to be detected into a foreign object classification model and determines the target foreign object in the image based on the classification map output by the model. Further, the electronic device determines whether the target foreign object is within the region to be detected.

[0056] The foreign object classification model is used to process the input image to be detected and output a classification map corresponding to the image to be detected. The classification map includes foreign objects of a preset type.

[0057] It should be noted that the preset type and the type of target foreign object can be preset in the electronic equipment by the maintenance personnel.

[0058] For example, taking a railway image as the image to be detected, a railway region as the region to be detected, and a railway classification image as the classification image to be detected, the foreign objects are manually set to preset types: people, stones, landslide sand, branches, and trains. The target foreign objects are people, stones, landslide sand, and branches. Real railway foreign object images including people, stones, landslide sand, branches, and trains are manually collected, or railway foreign object images are obtained by simulating real scenes. Furthermore, the preset types of foreign objects in the railway foreign object images are labeled to create a foreign object training dataset.

[0059] Electronic devices input the foreign object training dataset into the untrained foreign object classification model to obtain a trained foreign object classification model.

[0060] Furthermore, when there is only one railway area, the electronic device inputs the railway image into a trained foreign object classification model to obtain a railway classification map. The railway classification map includes foreign objects such as people, rocks, and tree branches. Based on the railway classification map (people, rocks, and tree branches) and the type of target foreign object (people, rocks, landslide sand, tree branches), the electronic device determines that the target foreign object in the railway image is a person, a rock, or a tree branch, and determines whether the person, rock, or tree branch is within the railway area.

[0061] S204. An alarm will be triggered if a foreign object is found in the area to be detected.

[0062] As one possible approach, if a target foreign object is present in the area to be detected, the electronic device will transmit an audible and visual alarm to the personnel on site through a buzzer and light-emitting diode (LED) flashing.

[0063] For example, in the case where the person or stone is within the railway area, the electronic equipment provides an audible and visual alarm via a buzzer and LED.

[0064] It should be noted that if there is no target foreign object in the area to be detected, the electronic device will use the image to be detected as the image to be detected in the subsequent alarm method for foreign objects on railways.

[0065] S205. When there are multiple areas to be detected, and the detected areas are not continuous, an alarm is triggered.

[0066] As one possible implementation, when there are multiple areas to be detected and the detected areas are not continuous, the electronic device transmits an audible and visual alarm to the on-site personnel through a buzzer and LED flashing.

[0067] For example, when there are fallen trees in the railway area, the semantic segmentation model outputs multiple and discontinuous railway areas. Electronic devices transmit sound and light alarms to on-site personnel through buzzers and LED flashing lights.

[0068] Understandably, existing technologies typically employ either deep learning models or background subtraction models to detect foreign objects on railways. However, neither of these methods can accurately detect foreign objects. This application uses a semantic segmentation model to obtain the detection region of the image containing the railway. If there are multiple discontinuous detection regions, it indicates the presence of a large foreign object, triggering an alarm. If there is only one detection region, the system further determines whether a target foreign object exists within that region based on the image and a foreign object classification model. If a target foreign object is present, an alarm is triggered. Therefore, this application uses a semantic segmentation model to detect large foreign objects. Even when no large foreign object is present in the image, the system further determines whether a target foreign object, such as a person, branch, or stone, exists in the detection region based on a foreign object classification model. This avoids false alarms caused by changes in the detection region due to camera movement or PTZ camera movement, thus improving the accuracy of railway foreign object detection.

[0069] In some embodiments, such as Figure 5 As shown in the embodiments of this application, the alarm method for foreign objects on railways also includes:

[0070] S301. If it is determined that there are no foreign objects of a preset type in the area to be detected, determine the background difference result of the video to be detected based on the video to be detected and the background difference model.

[0071] The video to be detected includes multiple images, and the multiple images include the images to be detected. The background subtraction results are used to characterize the changes in the multiple images.

[0072] As one possible implementation, assuming no foreign objects of a predetermined type are present in the area to be detected, the electronic device acquires a video containing the image to be detected. Further, the electronic device inputs the video to be detected into a background subtraction model to obtain the difference results between every two adjacent images, thus obtaining the background subtraction result of the video to be detected. The background subtraction result of the video to be detected includes multiple such difference results.

[0073] Specifically, when the image to be detected is the first frame of the video to be detected, the electronic device sets the difference result of the image to be detected to be the difference result between the image to be detected and the preset background image.

[0074] If the image to be detected is not the first frame in the video to be detected, the electronic device sets the difference result of the image to be detected to the difference result between the image to be detected and the previous adjacent image.

[0075] It should be noted that the background difference model can be a Gaussian mixture model.

[0076] For example, taking a railway image as the image to be detected and a railway video as the video to be detected, if there are no people, rocks, landslide sand, branches, or trains in the railway area of ​​the railway image, the electronic device acquires the railway video including the railway image. Further, the electronic device inputs the railway video into a Gaussian mixture model to obtain the difference results of multiple images, and then obtains the background difference result of the railway video.

[0077] Background subtraction models are used to extract differences between two images. For example, ... Figure 6 As shown, one image contains a square and a circle, and the other image contains a square. The background difference model processes these two images and the difference result is a circle.

[0078] S302. Determine whether the background difference result meets the preset conditions.

[0079] The preset conditions include four conditions: the first condition, the second condition, the third condition, and the fourth condition. The first condition is that the background difference result is greater than or equal to the first threshold; the second condition is that the duration of the background difference result is greater than or equal to the second threshold; the third condition is that the background difference result is within the detection area; and the fourth condition is that the type of the background difference result is not a preset security type.

[0080] As one possible implementation, the electronic device determines whether the background difference result of the video to be detected satisfies the first, second, third, and fourth conditions mentioned above. If the background difference result of the video to be detected simultaneously satisfies all four conditions, the electronic device determines that the background difference result of the video to be detected satisfies the preset conditions.

[0081] Regarding the first condition mentioned above, the electronic device uses a rectangular bounding box to select the multiple difference results, obtaining rectangular bounding boxes corresponding to the multiple difference results. Further, the electronic device determines whether the rectangular bounding boxes corresponding to the multiple difference results are greater than or equal to a first threshold. If a preset number of rectangular bounding boxes corresponding to the difference results are all greater than or equal to the first threshold, the electronic device determines that the background difference result satisfies the first condition.

[0082] It should be noted that the first threshold and the preset quantity can be preset in the electronic equipment by the maintenance personnel. The first threshold can be a rectangle with fixed length and width.

[0083] Regarding the second condition mentioned above, the electronic device determines the number of detectable images with the same difference result as the duration of the background difference result. If the duration of the background difference result is greater than or equal to the second threshold, the background difference result is determined to satisfy the second condition.

[0084] It should be noted that the second threshold can be preset in the electronic device by the maintenance personnel.

[0085] Regarding the third condition mentioned above, the electronic device identifies multiple regions to be detected corresponding to the aforementioned difference results and determines whether the difference results are within the regions to be detected. If a preset number of difference results are all within the regions to be detected, the background difference result is determined to satisfy the third condition.

[0086] Regarding the fourth condition mentioned above, the electronic device inputs multiple differential results into the classification model to determine the types of the multiple differential results. If there are foreign objects whose types are not of the preset safety type among the preset number of differential results, the background differential result is determined to satisfy the fourth condition.

[0087] It should be noted that the classification model processes multiple difference results from the input and outputs a classification map corresponding to the difference results of the image to be detected. The classification map includes foreign objects of a preset safety type and other types. If the type of foreign object is not the preset safety type, the electronic device determines that the type of foreign object is other types.

[0088] The default safe type of foreign object can be a change in light or a swaying branch. Other types of foreign objects can be birds or raindrops.

[0089] The first, second, third, and fourth conditions mentioned above can be applied sequentially or simultaneously.

[0090] During the sequential judgment process, if it is determined that the background difference result does not meet any of the above four conditions, the judgment process is stopped, and it is determined that the background difference result does not meet the above preset conditions.

[0091] Meanwhile, during the sequential judgment process, the order of judgment for the above four conditions is not limited. For example, it can be first determined whether the background difference result meets the first condition. If the background difference result meets the first condition, it is then determined whether the background difference result meets the second condition. If the background difference result meets the second condition, it is then determined whether the background difference result meets the third condition. If the background difference result meets the third condition, it is then determined whether the background difference result meets the fourth condition. If the background difference result meets the fourth condition, it is determined that the background difference result meets the preset condition.

[0092] S303. If the background difference result meets the preset conditions, an alarm will be triggered.

[0093] As one possible implementation, if the background difference result meets the preset conditions, the electronic device will transmit an audible and visual alarm to the on-site personnel through a buzzer and LED flashing.

[0094] Understandably, the first condition is that the background difference result is greater than or equal to the first threshold, which can identify foreign objects with a certain volume. The second condition is that the duration of the background difference result is greater than or equal to the second threshold, which can exclude foreign objects such as birds and raindrops that stay in the video for a short time. The third condition is that the background difference result is within the detection area, which is used to determine whether the foreign object is within the detection area. The fourth condition is that the type of the background difference result is not a preset safety type, which can exclude changes in lighting and swaying tree branches. Therefore, the preset conditions avoid false alarms caused by changes in lighting or rain and snow, and at the same time, take into account uncommon foreign objects such as wild animals, thus improving the accuracy of railway foreign object detection.

[0095] Figure 7 This is an exemplary embodiment of a railway foreign object alarm device 500, which includes an acquisition unit 501, a determination unit 502, and an alarm unit 503.

[0096] The acquisition unit 501 is used to acquire an image to be detected, including a railway.

[0097] The determining unit 502 is used to determine the region to be detected that includes the railway in the image to be detected based on the image to be detected and the semantic segmentation model after the acquiring unit 501 acquires the image to be detected that includes the railway.

[0098] The determining unit 502 is further configured to, when there is only one region to be detected, determine whether a target foreign object exists within the region to be detected based on the image to be detected and the foreign object classification model. The foreign object classification model is used to detect foreign objects of a preset type in the image to be detected, and the type of the target foreign object is a type other than a train among the preset types.

[0099] The alarm unit 503 is used to issue an alarm when a target foreign object is present in the area to be detected.

[0100] Optional, such as Figure 7 As shown, the alarm unit 503 provided in this embodiment is specifically used for:

[0101] An alarm is triggered when there are multiple areas to be detected, and the detected areas are not consecutive.

[0102] Optional, such as Figure 7 As shown in the embodiment of this application, the determining unit 502 is further configured to, when it is determined that there are no foreign objects of a preset type in the area to be detected, determine the background difference result of the video to be detected based on the video to be detected and the background difference model. The video to be detected includes multiple images, and the multiple images include the image to be detected. The background difference result is used to characterize the changes in the multiple images.

[0103] The alarm unit 503 is also used to issue an alarm when the background difference result meets preset conditions. The preset conditions include: the background difference result is greater than or equal to a first threshold, the duration of the background difference result is greater than or equal to a second threshold, the background difference result is within the detection area, and the type of the background difference result is not a preset security type.

[0104] Figure 8 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Figure 8 As shown, the electronic device 600 includes, but is not limited to, a processor 601 and a memory 602.

[0105] The memory 602 described above is used to store the executable instructions of the processor 601. It is understood that the processor 601 is configured to execute instructions to implement the railway foreign object alarm method in the above embodiment.

[0106] It should be noted that those skilled in the art will understand that Figure 8 The electronic device structure shown does not constitute a limitation on the electronic device; the electronic device may include, but is not limited to, other electronic devices. Figure 8 This may indicate more or fewer components, or a combination of certain components, or a different arrangement of components.

[0107] Processor 601 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in memory 602, and by calling data stored in memory 602, it performs various functions and processes data, thereby providing overall monitoring of the electronic device. Processor 601 may include one or more processing units. Optionally, processor 601 may integrate an application processor and a modem processor. The application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into processor 601.

[0108] The memory 602 can be used to store software programs and various data. The memory 602 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and application programs (such as determination units and acquisition units) required by at least one functional module. Furthermore, the memory 602 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0109] In an exemplary embodiment, a computer-readable storage medium including instructions, such as a memory including instructions, is also provided, which can be executed by a processor of an electronic device to implement the railway foreign object alarm method in the above embodiments.

[0110] In actual implementation, the functions of the acquisition unit 501, the determination unit 502, and the alarm unit 503 can all be provided by [the relevant entity / component]. Figure 8 The processor 601 calls the computer program stored in the memory 602 to implement the function. The specific execution process can be found in the description of the railway foreign object alarm method in the previous embodiment, and will not be repeated here.

[0111] Optionally, the computer-readable storage medium may be a non-transitory computer-readable storage medium, such as a read-only memory (ROM), random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.

[0112] In an exemplary embodiment, this application also provides a computer program product including one or more instructions, which can be executed by a processor of an electronic device to perform the methods described above.

[0113] It should be noted that when one or more instructions in the computer-readable storage medium or computer program product are executed by the processor of an electronic device, they implement the various processes of the above method embodiments and achieve the same technical effect as the above method. To avoid repetition, they will not be described again here.

[0114] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0115] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0116] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0117] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0118] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0119] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for detecting foreign objects on railway tracks, characterized in that, The method includes: Acquire images of the object to be inspected, including those containing railway lines; Based on the image to be detected and the semantic segmentation model, a region in the image to be detected that includes the railway is determined. An alarm is triggered when there are multiple regions to be detected, and the detected regions are not continuous. When there is only one region to be detected, the presence of a target foreign object is determined based on the image to be detected and the foreign object classification model. The foreign object classification model is used to detect foreign objects of a preset type in the image to be detected, and the type of the target foreign object is a type other than a train among the preset types. An alarm will be triggered if the target foreign object is present in the area to be detected. If it is determined that there are no foreign objects of the preset type in the area to be detected, the background difference result of the video to be detected is determined according to the video to be detected and the background difference model; the video to be detected includes multiple images, the multiple images include the image to be detected, and the background difference result is used to characterize the changes of the multiple images; If the background difference result meets the preset conditions, an alarm is triggered; the preset conditions include: the background difference result is greater than or equal to a first threshold, the duration of the background difference result is greater than or equal to a second threshold, the background difference result is located within the detection area, and the type of the background difference result is not a preset security type.

2. A railway foreign object alarm device, characterized in that, The alarm device includes an acquisition unit, a determination unit, and an alarm unit; The acquisition unit is used to acquire an image to be detected, including a railway. The determining unit is configured to, after the acquiring unit acquires the image to be detected including the railway, determine the region to be detected in the image to be detected that includes the railway based on the image to be detected and the semantic segmentation model. The alarm unit is used to issue an alarm when there are multiple areas to be detected and the multiple areas to be detected are not continuous. The determining unit is further configured to, when the number of regions to be detected is one, determine whether there is a target foreign object in the region to be detected based on the image to be detected and the foreign object classification model; the foreign object classification model is used to detect foreign objects of a preset type in the image to be detected, and the type of the target foreign object is a type other than a train among the preset types; The alarm unit is used to issue an alarm when the target foreign object is present in the area to be detected; The determining unit is further configured to determine the background difference result of the video to be detected based on the video to be detected and the background difference model when it is determined that there are no foreign objects of the preset type in the area to be detected. The video to be detected includes multiple images, and the multiple images include the image to be detected. The background subtraction result is used to characterize the changes in the multiple images. The alarm unit is also used to issue an alarm when the background difference result meets the preset conditions; The preset conditions include: the background difference result is greater than or equal to a first threshold, the duration of the background difference result is greater than or equal to a second threshold, the background difference result is within the detection area, and the type of the background difference result is not a preset security type.

3. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the method as described in claim 1.

4. A computer-readable storage medium, characterized in that, When the computer-executable instructions stored in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is able to perform the method as described in claim 1.

Images