Safety early warning methods, devices, storage media and equipment for rail transit scenarios

Abstract

This disclosure provides a safety early warning method, apparatus, storage medium, and device for rail transit scenarios. The method includes: acquiring multiple scene images from multiple perspectives using multiple image acquisition devices; wherein the rail transit scene includes at least tracks and traffic signal equipment; processing the multiple scene images using a pre-trained first neural network to determine a first detection result; wherein the first detection result includes at least one of the following: whether the rail transit scene includes personnel, and the status of the traffic signal equipment; processing the multiple scene images using a second model to determine a second detection result; wherein the second detection result includes whether rail transit equipment appears in the rail transit scene; and determining whether to issue a safety early warning message based on the first detection result and the second detection result.

Description

Technical Field

[0001] This disclosure relates to computer vision technology, and in particular to a safety early warning method, apparatus, storage medium, and device for rail transit scenarios. Background Technology

[0002] Rail transit refers to a type of transportation or system where vehicles operate on specific tracks. With the diversified development of train and railway technology, rail transit has evolved into increasingly diverse types, not only prevalent in long-distance land transportation but also widely used in short- and medium-distance urban public transportation. The railway system, with its vast network of lines, requires the coordinated work of complex systems such as train operations, locomotive operations, track maintenance, signaling, rolling stock, and power supply to ensure the normal operation and safe production of railway transportation. Currently, train operations within the railway system are characterized by numerous stations and long lines, with varying and complex environments at each station, adding many challenging problems to the daily production management of the train operations system. Summary of the Invention

[0003] To address the aforementioned technical problems, this disclosure is proposed. Embodiments of this disclosure provide a safety early warning method, apparatus, storage medium, and device for rail transit scenarios.

[0004] According to one aspect of the present disclosure, a safety early warning method for rail transit scenarios is provided, comprising:

[0005] Multiple images of the rail transit scene are obtained by acquiring images from multiple perspectives using multiple image acquisition devices; wherein, the rail transit scene includes at least tracks and traffic signal equipment;

[0006] The first detection result is determined by processing the multiple scene images through a pre-trained first neural network; wherein the first detection result includes at least one of the following: whether the rail transit scene includes personnel, and the status of the traffic signal equipment;

[0007] The second model is used to process the multiple scene images to determine a second detection result; wherein, the second detection result includes whether rail transit equipment appears in the rail transit scene;

[0008] Based on the first detection result and the second detection result, determine whether to issue a safety warning.

[0009] Optionally, the step of processing the plurality of scene images using a pre-trained first neural network to determine the first detection result includes:

[0010] The plurality of scene images are input into the first neural network, which outputs at least one human detection box, the confidence score of each human detection box, at least one traffic signal device detection box, and the confidence score of each traffic signal device detection box.

[0011] Based on the at least one human detection box and the confidence level of each human detection box, it is determined whether the rail transit scene includes personnel;

[0012] The state of the traffic signal device is determined based on the at least one traffic signal device detection frame and the confidence level of each traffic signal device detection frame.

[0013] Optionally, the first neural network is an improved YOLOv5 model with at least one downsampling layer added to the neck layer by a preset factor; wherein the preset factor is less than the original multiple downsampling factors in the neck layer of the initial YOLOv5 model.

[0014] Optionally, determining the state of the traffic signal device based on the at least one traffic signal device detection frame and the confidence level of each traffic signal device detection frame includes:

[0015] Based on the confidence level of each traffic signal device detection frame, at least one target device detection frame corresponding to at least one traffic signal device included in the rail transit scenario is determined;

[0016] The at least one target device detection box is segmented from the scene image to obtain at least one traffic signal region;

[0017] The state of the traffic signal equipment in the rail transit scenario is determined based on the pixel values ​​of multiple pixels included in the at least one traffic signal area.

[0018] Optionally, determining the state of the traffic signal equipment in the rail transit scenario based on the pixel values ​​of multiple pixels included in the at least one traffic signal area includes:

[0019] For each traffic signal zone, the traffic signal zone is converted into a preset color space;

[0020] In the preset color space, a color mask is generated based on at least one preset boundary threshold corresponding to at least one color; wherein, the preset boundary threshold includes a preset upper boundary threshold and a preset lower boundary threshold;

[0021] The valid pixels in the traffic signal area are determined based on the color mask, and the state of the traffic signal device is determined based on the pixel values ​​of the valid pixels.

[0022] Optionally, determining the effective pixels in the traffic signal area based on the color mask, and determining the state of the traffic signal device based on the pixel values ​​of the effective pixels, includes:

[0023] The effective pixels in the traffic signal area are determined based on the color mask;

[0024] Determine a first number of effective pixels with a first preset pixel value, and a second number of pixels with a second preset pixel value;

[0025] The state of the traffic signal equipment is determined based on the relationship between the first quantity and the second quantity.

[0026] Optionally, the second model is a Gaussian mixture model;

[0027] The step of processing the multiple scene images using a second model to determine a second detection result includes:

[0028] The multiple scene images are input into the Gaussian mixture model to obtain the foreground and background regions in each scene image;

[0029] By performing at least one preset processing on the foreground area, it is determined whether the rail transit scene includes the rail transit equipment and the track area corresponding to the track.

[0030] Optionally, determining whether the rail transit equipment and the location of the track are included in the rail transit scene by performing at least one preset processing on the foreground area includes:

[0031] Image morphological processing is performed on the foreground region to obtain the detected contour;

[0032] Based on whether the area of ​​the detected contour is greater than a preset area threshold, it is determined whether the rail transit equipment is included in the rail transit scene;

[0033] In response to the rail transit scenario including the rail transit equipment, a device detection frame of the rail transit equipment is determined based on the detection profile, and the position of the track is determined based on the device detection frame.

[0034] Optionally, determining whether to issue a security warning based on the first detection result and the second detection result includes:

[0035] In response to the traffic signal equipment being in a first state, and the positions of the people in the rail transit scenario and the positions of the tracks meeting preset conditions, the safety warning information is issued; or,

[0036] In response to the traffic signal device being in a second state, and the rail transit scenario including the rail transit device, and the positions of the personnel in the rail transit scenario and the positions of the tracks meeting preset conditions, the safety warning information is issued.

[0037] Optionally, before processing the plurality of scene images using the first neural network to determine the first detection result, the method further includes:

[0038] The first neural network is trained using a set of sample images; wherein the set of sample images includes multiple sample images, and each sample image includes corresponding personnel annotation information and traffic signal equipment annotation information.

[0039] According to another aspect of the present disclosure, a safety early warning device for rail transit scenarios is provided, comprising:

[0040] An image acquisition module is used to acquire images of a rail transit scene from multiple perspectives using multiple image acquisition devices, thereby obtaining multiple scene images; wherein, the rail transit scene includes at least tracks and traffic signal equipment;

[0041] The first detection module is used to process the multiple scene images respectively through a pre-trained first neural network to determine a first detection result; wherein, the first detection result includes at least one of the following information: whether the rail transit scene includes personnel, and the status of the traffic signal equipment;

[0042] The second detection module is used to process the multiple scene images using a second model to determine a second detection result; wherein, the second detection result includes whether rail transit equipment appears in the rail transit scene;

[0043] The safety warning module is used to determine whether to issue a safety warning message based on the first detection result and the second detection result.

[0044] Optionally, the first detection module includes:

[0045] The network prediction unit is used to input the plurality of scene images into the first neural network and output at least one human detection box, the confidence score of each human detection box, at least one traffic signal device detection box, and the confidence score of each traffic signal device detection box.

[0046] A personnel detection unit is used to determine whether the rail transit scene includes personnel based on the at least one human body detection frame and the confidence level of each human body detection frame.

[0047] A traffic signal detection unit is used to determine the state of the traffic signal device based on the at least one traffic signal device detection frame and the confidence level of each traffic signal device detection frame.

[0048] Optionally, the first neural network is an improved YOLOv5 model with at least one downsampling layer added to the neck layer by a preset factor; wherein the preset factor is less than the original multiple downsampling factors in the neck layer of the initial YOLOv5 model.

[0049] Optionally, the traffic signal detection unit is specifically configured to determine at least one target device detection frame corresponding to at least one traffic signal device included in the rail transit scene based on the confidence level of each traffic signal device detection frame; the at least one target device detection frame is obtained by segmenting at least one traffic signal region from the scene image; and the state of the traffic signal device in the rail transit scene is determined based on the pixel values ​​of multiple pixels included in the at least one traffic signal region.

[0050] Optionally, when the traffic signal detection unit determines the state of the traffic signal equipment in the rail transit scenario based on the pixel values ​​of multiple pixels included in the at least one traffic signal area, it is used to convert the traffic signal area into a preset color space for each traffic signal area; generate a color mask in the preset color space based on at least one preset boundary threshold corresponding to at least one color; wherein the preset boundary threshold includes a preset upper boundary threshold and a preset lower boundary threshold; determine the effective pixels in the traffic signal area based on the color mask, and determine the state of the traffic signal equipment based on the pixel values ​​of the effective pixels.

[0051] Optionally, when the traffic signal detection unit determines the effective pixels in the traffic signal area based on the color mask and determines the state of the traffic signal device based on the pixel values ​​of the effective pixels, it is used to determine the effective pixels in the traffic signal area based on the color mask; determine a first number of effective pixels with pixel values ​​of a first preset pixel value and a second number of effective pixels with pixel values ​​of a second preset pixel value; and determine the state of the traffic signal device based on the relationship between the first number and the second number.

[0052] Optionally, the second model is a Gaussian mixture model;

[0053] The second detection module is specifically used to input the multiple scene images into the Gaussian mixture model to obtain the foreground region and background region in each scene image; and to determine whether the rail transit scene includes the rail transit equipment and the track region corresponding to the track by performing at least one preset processing on the foreground region.

[0054] Optionally, when the second detection module determines whether the rail transit scene includes the rail transit equipment and the track region corresponding to the track by performing at least one preset processing on the foreground region, it performs image morphological processing on the foreground region to obtain a detection contour; determines whether the rail transit scene includes the rail transit equipment based on whether the area of ​​the detection contour is greater than a preset area threshold; and, in response to the rail transit scene including the rail transit equipment, determines the equipment detection box of the rail transit equipment based on the detection contour, and determines the track region corresponding to the track based on the equipment detection box.

[0055] Optionally, the safety warning module is specifically used to issue the safety warning information in response to the traffic signal device being in a first state and the location of the person in the rail transit scenario meeting a preset condition with respect to the location of the track; or, in response to the traffic signal device being in a second state and the rail transit scenario including the rail transit device, and the location of the person in the rail transit scenario meeting a preset condition with respect to the location of the track.

[0056] Optionally, the device further includes:

[0057] The network training module is used to train the first neural network using a sample image set; wherein the sample image set includes multiple sample images, and each sample image includes corresponding personnel annotation information and traffic signal equipment annotation information.

[0058] According to another aspect of the present disclosure, a computer-readable storage medium is provided, the storage medium storing a computer program for executing the safety early warning method for rail transit scenarios described in any of the above embodiments.

[0059] According to another aspect of the embodiments of this disclosure, an electronic device is provided, the electronic device comprising:

[0060] processor;

[0061] Memory used to store the processor's executable instructions;

[0062] The processor is configured to read the executable instructions from the memory and execute the instructions to implement the safety early warning method for rail transit scenarios described in any of the above embodiments.

[0063] Based on the above embodiments of this disclosure, a safety early warning method, apparatus, storage medium, and device for rail transit scenarios are provided. Multiple image acquisition devices acquire images of the rail transit scenario from multiple perspectives, resulting in multiple scene images. The rail transit scenario includes at least tracks and traffic signal equipment. A pre-trained first neural network processes the multiple scene images to determine a first detection result. The first detection result includes at least one of the following: whether personnel are present in the rail transit scenario, and the status of the traffic signal equipment. A second model processes the multiple scene images to determine a second detection result. The second detection result includes whether rail transit equipment is present in the rail transit scenario. Based on the first and second detection results, it is determined whether to issue a safety early warning message. This embodiment simultaneously detects the status of track areas, rail transit equipment, and traffic signal equipment in a rail transit scenario using a first neural network and a second model, improving detection accuracy and enabling timely safety warnings for personnel appearing on the track, thus enhancing the safety of personnel and rail transit equipment.

[0064] The technical solutions of this disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0065] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps;

[0066] Figure 1 This is a flowchart illustrating a safety early warning method for a rail transit scenario provided in an exemplary embodiment of this disclosure;

[0067] Figure 2 This is a public announcement Figure 1 A flowchart illustrating step 104 in the illustrated embodiment;

[0068] Figure 3 This is a public announcement Figure 1 A flowchart illustrating step 106 in the illustrated embodiment;

[0069] Figure 4 This is a schematic diagram of the structure of a safety early warning device for a rail transit scenario provided in an exemplary embodiment of this disclosure;

[0070] Figure 5 This is a structural diagram of an electronic device provided in an exemplary embodiment of this disclosure. Detailed Implementation

[0071] Hereinafter, exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. It is obvious that the described embodiments are merely some embodiments of the present disclosure, and not all embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described herein.

[0072] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0073] Those skilled in the art will understand that the terms "first," "second," etc., in the embodiments of this disclosure are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they indicate a necessary logical order between them.

[0074] It should also be understood that in the embodiments disclosed herein, "a plurality of" may refer to two or more, and "at least one" may refer to one, two or more.

[0075] It should also be understood that any component, data or structure mentioned in the embodiments of this disclosure can generally be understood as one or more unless expressly defined or given to the contrary in the context.

[0076] Furthermore, the term "and / or" in this disclosure is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this disclosure generally indicates that the preceding and following related objects have an "or" relationship.

[0077] It should also be understood that the description of the various embodiments in this disclosure emphasizes the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, they will not be described in detail.

[0078] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0079] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0080] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0081] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0082] The embodiments disclosed herein can be applied to electronic devices such as terminal devices, computer systems, and servers, and can operate together with a wide range of other general-purpose or special-purpose computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments, and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, and servers include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems, etc.

[0083] Electronic devices such as terminal devices, computer systems, and servers can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in distributed cloud computing environments, where tasks are executed by remote processing devices linked through communication networks. In distributed cloud computing environments, program modules can reside on local or remote computing system storage media, including storage devices.

[0084] Exemplary methods

[0085] Figure 1 This is a flowchart illustrating a safety early warning method for a rail transit scenario provided in an exemplary embodiment of this disclosure. This embodiment can be applied to electronic devices, such as… Figure 1 As shown, it includes the following steps:

[0086] Step 102: Collect images of the rail transit scene from multiple perspectives using multiple image acquisition devices to obtain multiple scene images.

[0087] The rail transit scenario includes at least the rails and traffic signal equipment in the rail transit equipment.

[0088] Optionally, the image acquisition device can be a monocular camera or a multi-view camera (e.g., a tri-view camera). To ensure the safety of people in rail transit, it is necessary not only to acquire images of the track but also to acquire images of objects in the rail transit scene, such as traffic signal equipment (e.g., traffic lights). Therefore, this embodiment requires image acquisition of the rail transit scene from multiple perspectives. Optionally, the rail transit scene can be a railway intersection or an urban rail transit intersection; image acquisition of the rail transit scene can be achieved by setting up multiple multi-view cameras on the light bridge erected at the intersection.

[0089] Step 104: Process multiple scene images using a pre-trained first neural network to determine the first detection result.

[0090] The first detection result includes at least one of the following: whether there are people in the rail transit scenario, and the status of traffic signal equipment.

[0091] In one embodiment, a first neural network processes each of the multiple scene images to predict at least one detection box in each scene image. Optionally, the detection boxes in this embodiment include at least two types: human detection boxes and traffic signal device detection boxes (the specific objects corresponding to the detection boxes can be determined based on training, such as traffic lights). By outputting human detection boxes and traffic signal device detection boxes, the detection of personnel and traffic signal devices is achieved. Optionally, the first neural network can be a YOLOv5 model, etc. The YOLOv5 model is a commonly used one-stage algorithm. One-stage algorithms have fast processing speed, meeting the real-time requirements of detection; and reduce the consumption of computing resources, improving the detection accuracy.

[0092] Step 106: Process multiple scene images using the second model to determine the second detection result.

[0093] The second detection result includes whether rail transit equipment is present in the rail transit scenario.

[0094] Optionally, rail transit equipment is a moving object in a rail transit scene. In order to identify moving objects in a rail transit scene, this embodiment processes multiple scene images using a second model to identify moving objects in the rail transit scene. Optionally, the second model can be a Gaussian mixture model, etc., and the output of the second model is further processed to determine whether rail transit equipment (e.g., trains, etc.) appears in the rail transit scene.

[0095] In this embodiment, steps 104 and 106 can be executed simultaneously; or steps 104 can be executed first and then steps 106 can be executed; or steps 106 can be executed first and then steps 104 can be executed. The execution order does not affect the technical effect achieved by this application.

[0096] Step 108: Based on the first and second detection results, determine whether to issue a safety warning.

[0097] Optionally, the first and second detection results are combined to determine whether a safety hazard exists in the current rail transit scenario, so as to issue a safety warning. By combining the two detection results to integrate various environmental, equipment and personnel information in the rail transit scenario, the accuracy of scenario recognition is improved, the accuracy of safety warnings for dangerous situations is improved, and ineffective warnings are avoided. Optionally, the safety warning information may include, but is not limited to, at least one of the following: sound, light, smell, etc. For example, a warning may be given by flashing high-frequency lights combined with playing a preset voice (e.g., repeatedly playing "Danger, please avoid"); or a warning may be given by flashing high-frequency lights, playing a preset voice, and releasing a non-toxic irritating gas.

[0098] This disclosure provides a safety early warning method for rail transit scenarios, which involves acquiring images of the rail transit scenario from multiple perspectives using multiple image acquisition devices to obtain multiple scene images. The rail transit scenario includes at least tracks and traffic signal equipment. A pre-trained first neural network processes the multiple scene images to determine a first detection result. The first detection result includes at least one of the following: whether personnel are present in the rail transit scenario, and the status of the traffic signal equipment. A second model processes the multiple scene images to determine a second detection result. The second detection result includes whether rail transit equipment is present in the rail transit scenario. Based on the first and second detection results, it is determined whether to issue a safety early warning. This embodiment simultaneously detects the status of track areas, rail transit equipment, and traffic signal equipment in a rail transit scenario using a first neural network and a second model, improving detection accuracy and enabling timely safety warnings for personnel appearing on the track, thus enhancing the safety of personnel and rail transit equipment.

[0099] like Figure 2 As shown above, in the above Figure 1 Based on the illustrated embodiment, step 104 may include the following steps:

[0100] Step 1041: Input multiple scene images into the first neural network and output at least one human body detection box, the confidence score of each human body detection box, at least one traffic signal device detection box, and the confidence score of each traffic signal device detection box.

[0101] Optionally, the first neural network can process multiple scene images simultaneously; or, the first neural network can process multiple scene images separately multiple times (processing one scene image at a time); or, multiple first neural networks can process multiple scene images separately, with each first neural network processing one scene image; since each scene image corresponds to a different perspective of the rail transit scene, the output results corresponding to all scene images are combined to output the prediction result of the corresponding rail transit scene.

[0102] Step 1042: Determine whether the rail transit scene includes people based on at least one human detection box and the confidence level of each human detection box.

[0103] Optionally, each human detection box corresponds to a confidence level, which represents the probability that the human detection box includes a human body. By pre-setting a confidence level threshold (the specific value is determined according to the actual application scenario), human detection boxes with a confidence level greater than or equal to the confidence level threshold are determined to include people, and human detection boxes with a confidence level less than the confidence level threshold are determined to not include people. This can determine whether there are people in the rail transit scenario, and the location information of people can be known based on the position of the human detection box.

[0104] Step 1043: Determine the state of the traffic signal device based on at least one traffic signal device detection frame and the confidence level of each traffic signal device detection frame.

[0105] In this embodiment, traffic signal equipment (e.g., traffic lights) corresponds to different traffic conditions under different states. For example, in an optional example, the traffic signal equipment is a traffic light corresponding to a train. When the traffic light is red, it means that the train will not pass temporarily. At this time, if there are people in the rail transit scenario, no alarm is needed. However, if the traffic light is green, it means that the train is about to pass through the track section in the rail transit scenario. At this time, an alarm needs to be issued to people in the rail transit scenario to prevent people from crossing the tracks and causing danger.

[0106] Optionally, the first neural network is an improved YOLOv5 model with at least one downsampling layer added to the neck layer by a preset factor; wherein the preset factor is less than the original multiple downsampling factors in the neck layer of the initial YOLOv5 model.

[0107] In this embodiment, to improve the detection accuracy of the first neural network for people, i.e., to more accurately identify small targets in the image, a downsampling layer with a preset downsampling factor (e.g., 4x downsampling) can be added to the original downsampling factor in the YOLOv5 model to enhance the mining of small target features. Multi-scale feedback is used to introduce global contextual information to improve the ability to identify small targets in the image. An attention mechanism and an SE module are added to the original YOLOv5 model to improve the model's detection performance.

[0108] The SE module, short for Squeeze-and-Excitation Networks, consists of two main parts: the Squeeze part and the Excitation part. The Squeeze part compresses the original feature map to 11C, effectively reducing the HW dimension to one dimension. This is typically achieved using global average pooling. After compression, this one-dimensional parameter gains access to the previous H*W global field of view, resulting in a wider receptive area. The Excitation part, after obtaining the 11C representation from the Squeeze, adds a fully connected (FC) layer to predict the importance of each channel. The importance of different channels is then applied (excited) to the corresponding channels in the previous feature map before further processing. The SE module's flexibility lies in its direct application to existing network structures. This embodiment improves the detection performance by adding the SE module to the original YOLOv5 model.

[0109] Furthermore, regarding the improvements to the YOLOv5 model mentioned above, due to the addition of a 4x downsampling process, the corresponding prior boxes of a certain size are added to the training data (the larger the downsampling factor, the larger the prior box). For example, the size of the prior boxes is increased to [5,6] (representing a prior box of 5*6 pixels), [8,14] (representing a prior box of 8*14 pixels), and [15,11] (representing a prior box of 15*11 pixels).

[0110] Optionally, step 1043 may include:

[0111] Based on the confidence level of each traffic signal device detection frame, determine at least one target device detection frame corresponding to at least one traffic signal device included in the rail transit scenario;

[0112] At least one traffic signal region is obtained by segmenting the scene image based on at least one target device detection bounding box;

[0113] The state of traffic signal equipment in a rail transit scenario is determined based on the pixel values ​​of multiple pixels included in at least one traffic signal area.

[0114] In this embodiment, the confidence level of each traffic signal device detection frame can be compared with a preset threshold (the specific value is determined according to the actual application scenario). Traffic signal device detection frames with a confidence level greater than or equal to the preset threshold are identified as target device detection frames. Based on the target device detection frames, the traffic signal area corresponding to the target device can be determined, and at least one traffic signal area can be segmented from the scene image for separate processing. The state of the traffic signal device can be determined based on the pixel values ​​of multiple pixels included in each traffic signal area. For example, red and green correspond to different pixel values. Therefore, the pixel values ​​corresponding to the pixels included in the traffic signal area can determine whether the state of the traffic signal device is red or green.

[0115] Optionally, the state of traffic signal equipment in a rail transit scenario is determined based on the pixel values ​​of multiple pixels included in at least one traffic signal area, including:

[0116] For each traffic signal zone, convert the traffic signal zone into a preset color space;

[0117] In a preset color space, a color mask is generated based on at least one preset boundary threshold corresponding to at least one color; wherein, the preset boundary threshold includes a preset upper boundary threshold and a preset lower boundary threshold.

[0118] The effective pixels in the traffic signal area are determined based on the color mask, and the status of the traffic signal equipment is determined based on the pixel values ​​of the effective pixels.

[0119] Optionally, the traffic signal area is a part of the scene image and corresponds to the same color space as the scene image. In this embodiment, the traffic signal area is converted to a preset color space to improve the efficiency of color recognition. For example, the preset color space is the HSV color space or other color spaces that can clearly distinguish between red and green. A color mask with a preset boundary threshold corresponding to at least one color is determined in the preset color space. For example, color masks for red, green and yellow are determined respectively. After determining the preset upper threshold and preset lower threshold corresponding to each color, the corresponding color mask can be generated in the traffic signal area using the cv2.inRange() function. The mask generation process can be smoothed by median filtering.

[0120] Optionally, the effective pixels in the traffic signal area are determined based on a color mask, and the state of the traffic signal device is determined based on the pixel values ​​of the effective pixels, including:

[0121] Determine the effective pixels in the traffic signal area based on a color mask;

[0122] Determine a first number of valid pixels with a first preset pixel value, and a second number of pixels with a second preset pixel value;

[0123] The status of the traffic signal equipment is determined based on the relationship between the first and second quantities.

[0124] In this embodiment, a mask region corresponding to at least one color can be obtained through at least one color mask, with each color corresponding to one mask region; all pixels corresponding to the mask regions are valid pixels, that is, the pixels in the mask regions have valid colors, and the traffic signal region includes not only the valid pixels corresponding to the traffic lights, but also other pixels that are not useful for identifying the traffic light status. In this embodiment, valid pixels are extracted through color mask, which improves the accuracy of status recognition. The obtained valid pixels may include pixels of multiple colors. A first preset pixel value corresponds to one color (e.g., red), and a second preset pixel value corresponds to another color (e.g., green). When the number of pixels of one color accounts for a large proportion of the valid pixels (or the number is much greater than that of pixels of another color), the state of the traffic signal device can be determined by that color. For example, when the number of red pixels accounts for a larger proportion of the valid pixels than a preset proportion (determined according to the actual scenario), the traffic light color is determined to be red. By determining the state of the traffic signal device by the proportion of the number of pixels, the simultaneous display of red and green lights due to display errors in some LED beads (e.g., multiple LED beads may be displaying errors) is avoided. This embodiment determines the traffic light state by using the color corresponding to the pixel value with a larger proportion of the number of pixels, thus solving the above problem.

[0125] like Figure 3 As shown above, in the above Figure 1 Based on the illustrated embodiment, step 106 may include the following steps:

[0126] Step 1061: Input multiple scene images into a Gaussian mixture model to obtain the foreground and background regions in each scene image.

[0127] The second model can be a Gaussian mixture model.

[0128] Optionally, when the scene image is the current frame, the background region in the rail transit scene is determined by combining the scene images of multiple frames before the current frame with the Gaussian mixture model.

[0129] Optionally, multiple image acquisition devices continuously acquire and process images of the rail transit scene in real time. Therefore, the multiple scene images can be considered as a single frame in a video. For example, the multiple scene images currently input to the Gaussian mixture model can be used as the (n+1)th frame in the video acquired by the image acquisition devices, and the background of the rail transit scene can be determined based on the previous n frames of video images acquired before this (n+1)th frame.

[0130] Optionally, the specific process of using Gaussian mixture model to segment the background and foreground regions may include: initializing the background of the rail transit scene: firstly, using the first n consecutive video stream images to construct the background region of the rail transit scene, and representing the pixel value of each pixel in each of the first n frames using a weighted sum of K Gaussian distributions; when receiving the (n+1)th frame image of the video (corresponding to multiple scene images acquired in this embodiment), updating the parameters of the Gaussian mixture model using the (n+1)th frame image to adapt to changes in the moving scene; when moving rail transit equipment appears in the (n+1)th frame image, performing foreground and background region segmentation.

[0131] Step 1062: By performing at least one preset processing on the foreground area, determine whether the rail transit scene includes rail transit equipment and the corresponding track area.

[0132] In this embodiment, for each scene image, a Gaussian mixture model is used to segment the scene image into a foreground region and a background region. The foreground refers to any meaningful moving object (in this embodiment, rail transit equipment) assuming the background is stationary. Extracting the foreground from the rail transit scene image using the Gaussian mixture model aims to make the background more closely resemble the background of the current video frame. After determining the foreground image in the current frame's scene image, at least one processing step is used to further identify and determine the moving objects in the rail transit scene, thereby identifying the equipment region of the rail transit equipment in the scene image, and based on this equipment region, determining the corresponding track region.

[0133] Optionally, step 1062 may include:

[0134] Perform image morphological processing on the foreground region to obtain the detected contour;

[0135] Optionally, image morphological processing may include, but is not limited to, opening and closing operations; wherein, opening operation is erosion followed by dilation; and closing operation is dilation followed by erosion; for example, in an optional example, the opening operation uses a 3*3 convolution kernel size, and the closing operation uses a 5*5 convolution kernel size.

[0136] Based on whether the area of ​​the detected contour is greater than a preset area threshold, it is determined whether the rail transit scene includes rail transit equipment.

[0137] In determining the area of ​​the detection contour, Green's formula can be used to calculate the number of pixel values ​​included in the detection contour, thereby determining the area corresponding to the detection contour.

[0138] In response to rail transit scenarios, including rail transit equipment, the equipment detection frame of the rail transit equipment is determined based on the detection profile, and the position of the track is determined based on the equipment detection frame.

[0139] Optionally, the Gaussian mixture model outputs a binarized image. Image morphological processing is performed on the binarized image to determine the detection contours (corresponding to rail transit equipment). These contours are then identified using a preset area threshold. Contours with areas greater than the preset area threshold are identified as including the rail transit equipment. If all detection contours are smaller than the preset area threshold, it indicates that the rail transit scene does not include the rail transit equipment. The preset area threshold identification removes noise from the image and eliminates small moving objects (e.g., birds) in the rail transit scene, preventing irrelevant moving objects from affecting the detection results. In this embodiment, after identifying the rail transit equipment, since the equipment relies on the track for operation, it must be located on the track; that is, the position of the track in the image can be determined based on the position of the rail transit equipment.

[0140] For example, in some optional examples, the foreground region is binarized to obtain a binarized image. The image is then eroded twice using a 3*3 erosion convolution kernel. The eroded image is then dilated twice using an 8*3 dilation convolution kernel. Contour detection is performed on the obtained image to obtain the position information of all contours. For each detected contour, the area of ​​the image in the contour is calculated using Green's formula. If the number of pixels in the area is greater than a preset number (e.g., 1000 pixels), it is considered that there is operating rail transit equipment in the current rail transit scene.

[0141] In some alternative embodiments, step 108 may include:

[0142] In response to the traffic signal equipment being in its first state, and the positions of people and tracks in the rail transit scenario meeting preset conditions, a safety warning is issued; or,

[0143] When the traffic signal equipment is in the second state, and the rail transit scenario includes rail transit equipment, and the positions of the personnel in the rail transit scenario and the positions of the tracks meet preset conditions, a safety warning message is issued.

[0144] In this embodiment, safety warning information is issued to personnel through at least one decision. Optionally, the first state is a green light, and the second state is a red light or a yellow light. Since the position of the track is determined in the second detection result, when the traffic signal device is green, it indicates that the rail transit equipment is about to appear in the rail transit scene. Therefore, in this case, a safety warning information is issued as soon as personnel are present at the track position. In addition, when the traffic signal device is red or yellow, if the second detection result indicates that there is a specified moving transportation device (e.g., a train) in the rail transit scene, and the distance between the personnel's position and the track position is less than a preset distance value, a safety warning information will be issued to prompt personnel to evacuate and reduce the occurrence of traffic safety accidents.

[0145] In some optional embodiments, prior to performing step 104, the following may also be included:

[0146] The first neural network is trained using a set of sample images.

[0147] The sample image set includes multiple sample images, each of which includes corresponding personnel annotation information and traffic signal equipment annotation information.

[0148] In this embodiment, the acquisition of sample images can involve image acquisition from multiple different rail transit scenarios. For each rail transit scenario, images are acquired from multiple perspectives using multiple multi-view cameras. Multiple sample images acquired in the same rail transit scenario are uniformly labeled, for example, by manually labeling the status of personnel and traffic signal equipment in the sample images. In addition, since manual labeling requires a lot of manpower, the acquisition of sample images is quite difficult. In order to improve the performance of the first neural network, a large number of sample images are needed to train the first neural network. Optionally, the acquired and labeled sample images are augmented and expanded to simulate complex scenarios in the natural environment that do not appear in the training dataset. The first neural network is trained using the data augmented and expanded sample image set. Optionally, data augmentation and expansion include, but are not limited to, at least one of the following: random scaling, random cropping, random arrangement, rotation, cropping, color conversion, etc.

[0149] The safety warning method for any rail transit scenario provided in this disclosure can be executed by any suitable device with data processing capabilities, including but not limited to terminal devices and servers. Alternatively, the safety warning method for any rail transit scenario provided in this disclosure can be executed by a processor, such as by a processor executing the safety warning method for any rail transit scenario mentioned in this disclosure by calling corresponding instructions stored in memory. Further details will not be elaborated below.

[0150] Exemplary device

[0151] Figure 4 This is a structural schematic diagram of a safety early warning device for a rail transit scenario provided in an exemplary embodiment of this disclosure. Figure 4 As shown, the apparatus provided in this embodiment includes:

[0152] The image acquisition module 41 is used to acquire images of the rail transit scene from multiple perspectives through multiple image acquisition devices, thereby obtaining multiple scene images.

[0153] In the rail transit scenario, at least tracks and traffic signal equipment are included.

[0154] The first detection module 42 is used to process multiple scene images through a pre-trained first neural network to determine the first detection result.

[0155] The first detection result includes at least one of the following: whether there are people in the rail transit scenario, and the status of traffic signal equipment.

[0156] The second detection module 43 is used to process multiple scene images using the second model to determine the second detection result.

[0157] The second detection result includes whether rail transit equipment is present in the rail transit scenario.

[0158] The safety warning module 44 is used to determine whether to issue a safety warning message based on the first detection result and the second detection result.

[0159] This disclosure provides a safety early warning device for rail transit scenarios, which acquires images of the rail transit scenario from multiple perspectives using multiple image acquisition devices, resulting in multiple scene images. The rail transit scenario includes at least tracks and traffic signal equipment within the rail transit system. A first neural network processes the multiple scene images to determine a first detection result, which includes at least one of the following: whether personnel are present in the rail transit scenario, and the status of the traffic signal equipment. A second model processes the multiple scene images to determine a second detection result, which includes whether rail transit equipment is present in the rail transit scenario. Based on the first and second detection results, a safety early warning message is issued. This embodiment, through the first neural network and the second model, improves the detection accuracy and enables timely safety warnings for personnel appearing on the tracks, thereby enhancing the safety of personnel and rail transit equipment.

[0160] Optionally, the first detection module 42 includes:

[0161] The network prediction unit is used to input the plurality of scene images into the first neural network and output at least one human detection box, the confidence score of each human detection box, at least one traffic signal device detection box, and the confidence score of each traffic signal device detection box.

[0162] A personnel detection unit is used to determine whether the rail transit scene includes personnel based on the at least one human body detection frame and the confidence level of each human body detection frame.

[0163] A traffic signal detection unit is used to determine the state of the traffic signal device based on the at least one traffic signal device detection frame and the confidence level of each traffic signal device detection frame.

[0164] Optionally, the first neural network is an improved YOLOv5 model with at least one downsampling layer added to the neck layer by a preset factor; wherein the preset factor is less than the original multiple downsampling factors in the neck layer of the initial YOLOv5 model.

[0165] Optionally, the traffic signal detection unit is specifically used to determine at least one target device detection box corresponding to at least one traffic signal device included in the rail transit scene based on the confidence of each traffic signal device detection box; at least one target device detection box is segmented from the scene image to obtain at least one traffic signal region; and the state of the traffic signal device in the rail transit scene is determined based on the pixel values ​​of multiple pixels included in the at least one traffic signal region.

[0166] Optionally, when the traffic signal detection unit determines the state of traffic signal equipment in a rail transit scenario based on the pixel values ​​of multiple pixels included in at least one traffic signal area, it converts the traffic signal area into a preset color space for each traffic signal area; generates a color mask in the preset color space based on at least one preset boundary threshold corresponding to at least one color; wherein the preset boundary threshold includes a preset upper threshold and a preset lower threshold; determines the valid pixels in the traffic signal area based on the color mask, and determines the state of the traffic signal equipment based on the pixel values ​​of the valid pixels.

[0167] Optionally, when the traffic signal detection unit determines the effective pixels in the traffic signal area based on the color mask and determines the state of the traffic signal device based on the pixel values ​​of the effective pixels, it is used to determine the effective pixels in the traffic signal area based on the color mask; determine a first number of effective pixels with pixel values ​​of a first preset pixel value and a second number of effective pixels with pixel values ​​of a second preset pixel value; and determine the state of the traffic signal device based on the relationship between the first number and the second number.

[0168] Optionally, the second model is a Gaussian mixture model;

[0169] The second detection module 43 is specifically used to input multiple scene images into a Gaussian mixture model to obtain the foreground region and background region in each scene image; and to determine whether the rail transit scene includes rail transit equipment and the location of the track by performing at least one preset processing on the foreground region.

[0170] Optionally, when the second detection module determines whether the rail transit scene includes rail transit equipment and the corresponding track area by performing at least one preset processing on the foreground region, it performs image morphological processing on the foreground region to obtain a detection contour; determines whether the rail transit scene includes rail transit equipment based on whether the area of ​​the detection contour is greater than a preset area threshold; and, in response to the rail transit scene including rail transit equipment, determines the equipment detection box of the rail transit equipment based on the detection contour, and determines the corresponding track area based on the equipment detection box.

[0171] In some optional embodiments, the safety warning module 44 is specifically used to issue a safety warning message in response to the traffic signal equipment being in a first state and the position of the person in the rail transit scenario meeting a preset condition; or, in response to the traffic signal equipment being in a second state and the rail transit scenario including rail transit equipment, and the position of the person in the rail transit scenario meeting a preset condition.

[0172] In some optional embodiments, the apparatus provided in this embodiment further includes:

[0173] The network training module is used to train the first neural network using a set of sample images.

[0174] The sample image set includes multiple sample images, each of which includes corresponding personnel annotation information and traffic signal equipment annotation information.

[0175] Exemplary electronic devices

[0176] Below, for reference Figure 5 This describes an electronic device according to embodiments of the present disclosure. The electronic device may be either or both of a first device and a second device, or a standalone device independent of them, which may communicate with the first device and the second device to receive acquired input signals from them.

[0177] Figure 5 A block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0178] like Figure 5 As shown, the electronic device 50 includes one or more processors 51 and memory 52.

[0179] The processor 51 may be a central processing unit (CPU) or other form of processing unit with data processing and / or instruction execution capabilities, and may control other components in the electronic device 50 to perform desired functions.

[0180] The memory 52 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 51 may execute the program instructions to implement the safety warning methods for rail transit scenarios in the various embodiments of this disclosure described above, and / or other desired functions. Various contents such as input signals, signal components, and noise components may also be stored in the computer-readable storage medium.

[0181] In one example, the electronic device 50 may also include an input device 53 and an output device 54, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0182] For example, when the electronic device is a first device or a second device, the input device 53 can be the microphone or microphone array described above, used to capture the input signal from the sound source. When the electronic device is a standalone device, the input device 53 can be a communication network connector, used to receive the acquired input signal from the first device and the second device.

[0183] In addition, the input device 53 may also include, for example, a keyboard, a mouse, etc.

[0184] The output device 54 can output various information to the outside, including determined distance information, direction information, etc. The output device 54 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0185] Of course, for the sake of simplicity, Figure 5 Only some of the components of the electronic device 50 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 50 may include any other suitable components depending on the specific application.

[0186] Exemplary computer program products and computer-readable storage media

[0187] In addition to the methods and devices described above, embodiments of this disclosure may also be computer program products, including computer program instructions that, when executed by a processor, cause the processor to perform the steps in the safety warning methods for rail transit scenarios according to various embodiments of this disclosure as described in the "Exemplary Methods" section of this specification.

[0188] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this disclosure. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0189] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions that, when executed by a processor, cause the processor to perform the steps in the safety warning method for rail transit scenarios according to various embodiments of this disclosure as described in the "Exemplary Methods" section of this specification.

[0190] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0191] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0192] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0193] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0194] The methods and apparatus of this disclosure may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustrative purposes only, and the steps of the methods of this disclosure are not limited to the order specifically described above unless otherwise specifically stated. Furthermore, in some embodiments, this disclosure may also be implemented as a program recorded on a recording medium, the program including machine-readable instructions for implementing the methods according to this disclosure. Thus, this disclosure also covers recording media storing programs for performing the methods according to this disclosure.

[0195] It should also be noted that in the apparatus, devices, and methods of this disclosure, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions to this disclosure.

[0196] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0197] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A safety early warning method for rail transit scenarios, characterized in that, include: Multiple images of the rail transit scene are obtained by acquiring images from multiple perspectives using multiple image acquisition devices; wherein, the rail transit scene includes at least tracks and traffic signal equipment; The first detection result is determined by processing the multiple scene images through a pre-trained first neural network; wherein the first detection result includes at least one of the following: whether the rail transit scene includes personnel, and the status of the traffic signal equipment; The second model is used to process the multiple scene images to determine a second detection result; wherein, the second detection result includes whether rail transit equipment appears in the rail transit scene; Based on the first detection result and the second detection result, determine whether to issue a safety warning message; The step of processing the multiple scene images using a pre-trained first neural network to determine the first detection result includes: The multiple scene images are input into the first neural network, which outputs at least one traffic signal device detection box and the confidence score of each traffic signal device detection box. Based on the confidence level of each traffic signal device detection frame, at least one target device detection frame corresponding to at least one traffic signal device included in the rail transit scenario is determined; At least one traffic signal region is obtained by segmenting the scene image based on the at least one target device detection box; The state of the traffic signal equipment in the rail transit scenario is determined based on the pixel values ​​of multiple pixels included in the at least one traffic signal area.

2. The method according to claim 1, characterized in that, The step of processing the multiple scene images using a pre-trained first neural network to determine the first detection result further includes: The multiple scene images are input into the first neural network, which outputs at least one human detection box and the confidence score of each human detection box. Whether the rail transit scene includes people is determined based on the at least one human detection box and the confidence level of each human detection box.

3. The method according to claim 2, characterized in that, The first neural network is an improved YOLOv5 model with at least one downsampling layer added to the neck layer by a preset factor; wherein the preset factor is less than the original multiple downsampling factors in the neck layer of the initial YOLOv5 model.

4. The method according to claim 1, characterized in that, Determining the state of traffic signal equipment in the rail transit scenario based on the pixel values ​​of multiple pixels included in the at least one traffic signal area includes: For each traffic signal zone, the traffic signal zone is converted into a preset color space; In the preset color space, a color mask is generated based on at least one preset boundary threshold corresponding to at least one color; wherein, the preset boundary threshold includes a preset upper boundary threshold and a preset lower boundary threshold; The valid pixels in the traffic signal area are determined based on the color mask, and the state of the traffic signal device is determined based on the pixel values ​​of the valid pixels.

5. The method according to claim 4, characterized in that, The step of determining the effective pixels in the traffic signal area based on the color mask, and determining the state of the traffic signal device based on the pixel values ​​of the effective pixels, includes: The effective pixels in the traffic signal area are determined based on the color mask; Determine a first number of effective pixels with a first preset pixel value, and a second number of pixels with a second preset pixel value; The state of the traffic signal equipment is determined based on the relationship between the first quantity and the second quantity.

6. The method according to any one of claims 1-5, characterized in that, The second model is a Gaussian mixture model; The step of processing the multiple scene images using a second model to determine a second detection result includes: The multiple scene images are input into the Gaussian mixture model to obtain the foreground and background regions in each scene image; By performing at least one preset processing on the foreground area, it is determined whether the rail transit scene includes the rail transit equipment and the track area corresponding to the track.

7. The method according to claim 6, characterized in that, The step of determining whether the rail transit scene includes the rail transit equipment and the corresponding track area by performing at least one preset processing on the foreground area includes: Image morphological processing is performed on the foreground region to obtain the detected contour; Based on whether the area of ​​the detected contour is greater than a preset area threshold, it is determined whether the rail transit equipment is included in the rail transit scene; In response to the rail transit scenario including the rail transit equipment, a device detection frame of the rail transit equipment is determined based on the detection contour, and a track area corresponding to the track is determined based on the device detection frame.

8. The method according to any one of claims 1-5, characterized in that, The step of determining whether to issue a security warning based on the first detection result and the second detection result includes: In response to the traffic signal equipment being in a first state, and the positions of the people in the rail transit scenario and the positions of the tracks meeting preset conditions, the safety warning information is issued; or, In response to the traffic signal device being in a second state, and the rail transit scenario including the rail transit device, and the positions of the personnel in the rail transit scenario and the positions of the tracks meeting preset conditions, the safety warning information is issued.

9. The method according to any one of claims 1-5, characterized in that, Before processing the multiple scene images using the first neural network to determine the first detection result, the process further includes: The first neural network is trained using a set of sample images; wherein the set of sample images includes multiple sample images, and each sample image includes corresponding personnel annotation information and traffic signal equipment annotation information.

10. A safety early warning device for rail transit scenarios, characterized in that, include: An image acquisition module is used to acquire images of a rail transit scene from multiple perspectives using multiple image acquisition devices, thereby obtaining multiple scene images; wherein, the rail transit scene includes at least tracks and traffic signal equipment; The first detection module is used to process the multiple scene images respectively through a pre-trained first neural network to determine a first detection result; wherein, the first detection result includes at least one of the following information: whether the rail transit scene includes personnel, and the status of the traffic signal equipment; The second detection module is used to process the multiple scene images using a second model to determine a second detection result; wherein, the second detection result includes whether rail transit equipment appears in the rail transit scene; A security warning module is used to determine whether to issue a security warning message based on the first detection result and the second detection result; The first detection module includes: A network prediction unit is used to input the multiple scene images into the first neural network and output at least one traffic signal device detection box and the confidence score of each traffic signal device detection box; A traffic signal detection unit is configured to determine at least one target device detection frame corresponding to at least one traffic signal device included in the rail transit scene based on the confidence level of each traffic signal device detection frame; the at least one target device detection frame is obtained by segmenting at least one traffic signal region from the scene image; and the state of the traffic signal device in the rail transit scene is determined based on the pixel values ​​of multiple pixels included in the at least one traffic signal region.

11. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which is used to execute the safety early warning method for rail transit scenarios as described in any one of claims 1-9.

12. An electronic device, characterized in that, The electronic device includes: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the safety early warning method for rail transit scenarios as described in any one of claims 1-9.

Images