A signal plane diagram signal machine recognition method and system based on deep learning

Abstract

The application provides a signal plane diagram signal machine recognition method and system based on deep learning, which comprises the following steps: obtaining a signal plane diagram to be recognized; using a signal machine recognition network to recognize the signal machine in the signal plane diagram; and the signal machine recognition network is a neural network based on YOLOv5-s. The signal plane diagram signal machine recognition method and system based on deep learning take YOLOv5-s network as the basis, propose a Signal-Net recognition network suitable for signal plane diagrams, and can accurately detect the signal machine target in a high-resolution drawing. While ensuring the recognition accuracy of the network for the signal machine, the model parameter amount is moderate, and the real-time operation requirement on the edge computing device is met.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of rail transit, and particularly relates to a signal plan signal machine identification method and system based on deep learning. BACKGROUND

[0002] For the signal field of rail transit, in the process of drawing a plan, mainly including drawing route signal machines, warning signs, and station entrance signal machines, and they must meet the relevant requirements. Given the large number of signal types in the station, even the same type of signal machine needs to be composed in different ways according to the actual situation of the on-site line, therefore, accurately identifying various signal machines of the station signal plan layout (referred to as signal plan) is a time-consuming and laborious work.

[0003] As one of the important algorithms for target detection, YOLOv3 (YOLO, You Only Look Once) has high precision and fast inference speed, which makes engineering and technical personnel see the dawn of deploying computer vision in the industrial field. As a milestone in the development of YOLO series algorithms, this algorithm has attracted the attention of a large number of scholars and engineering and technical personnel for a long time. The framework of YOLOv3 is Darknet, which can utilize GPU (Graphics Processing Unit, graphics processor) parallel operation, and enhances the detection ability of small targets in the image. With the appearance of YOLOv4 and YOLOv5, the overall performance of the target detection algorithm has been greatly improved.

[0004] As shown in Figure 1 The signal machines in the signal plan drawing have fixed drawing specifications, and the signal machines are represented by circles, and one or more connected circles represent a signal machine. Compared with the entire drawing, the proportion of the signal machine target is relatively small, so there are two difficulties in applying deep learning technology to signal machine identification:

[0005] (a) Since the signal machine target is small, the local area of the signal machine needs to be clear and obvious, so the plan must have high resolution, such as 6750x3200, to ensure that the small target signal machine has strong readability in the drawing.

[0006] (b) Due to the high resolution of the signal plan, a new problem arises, that is, how to input data into the deep neural network for training and testing.

[0007] For difficulty (a), with the update of hardware facilities, it is easy to solve in technology with corresponding software. It is no longer difficult to increase the resolution to improve the identification effect of small targets. Today, super high-resolution images are widely used, such as remote sensing satellite images and images taken from high altitude by unmanned aerial vehicles. The description of signal equipment can also obtain high-resolution signal plan through conversion tools.

[0008] For difficulty (b), due to the large image resolution, the difficulty of detecting small targets increases, mainly reflected in the limitation of the receptive field. The image size input into the neural network is not the same, and the image is processed into an image size suitable for the network in the network preprocessing stage. Taking the image size 640x640 input into the network as an example, the receptive field is analyzed. Since YOLOv5 has three scale feature maps, it is obtained by downsampling 3, 4 and 5 times respectively, and the size of the feature map obtained by detection is 20x20 (640 / 2 5 ), 40x40 (640 / 2 4 ), 80x80 (640 / 2 3 ). Among them, the feature map with the largest size of 80x80 is used to detect small target categories, and the receptive field of the feature map corresponding to the input image is 8x8 (640 / 80) corresponding to the input image. If the area occupied by a single target image is lower than 8x8, it cannot be detected.

[0009] For signal plan image with a resolution of 10400x3160, if the original image is directly input, many small targets cannot be detected.

[0010] In addition, when the multiple of downsampling of a large resolution image is too large, data information is easily lost. Even if the network is trained, the forward propagation of the network has certain requirements on the memory, that is, the storage of many feature maps is completed, so a large amount of computing resources is occupied, which is easy to cause memory explosion and cannot be normally trained and reasoned.

[0011] Therefore, there is an urgent need for a signal plan recognition method for efficiently detecting extremely small targets in large resolution or super large resolution. SUMMARY

[0012] In view of the above problems, the present application aims to solve the problem that the current deep neural network is difficult to detect extremely small targets in large resolution or super large resolution. A deep learning target detection algorithm is combined with a traditional image processing method to recognize signal machines in signal plan images. A network for signal machine recognition in signal plan images, namely Signal-Net network, is designed to recognize signal machines and improve the efficiency and accuracy of artificial recognition.

[0013] The present application provides a signal machine recognition method for signal plan based on deep learning,

[0014] obtaining a signal plane diagram to be recognized;

[0015] recognizing signal machines in the signal plane diagram by using a signal machine recognition network;

[0016] The signal machine recognition network is a neural network based on YOLOv5-s.

[0017] Further, obtaining the signal plane diagram to be recognized comprises:

[0018] segmenting the whole signal plane diagram to form a plurality of local area images;

[0019] screening the plurality of local area images to obtain a local area image containing a signal machine.

[0020] Further, segmenting the whole signal plane diagram comprises:

[0021] segmenting the whole signal plane diagram by using a sliding window based on an overlap rate, so that each signal machine appears completely in at least one local area image.

[0022] Further, screening the plurality of local area images to obtain a local area image containing a signal machine as the signal plane diagram to be recognized comprises:

[0023] graying the local area image;

[0024] further performing median filtering on the local area image, so as to achieve the effect of noise suppression;

[0025] performing Hough circle detection on the processed local area image according to Hough detection parameters;

[0026] The Hough detection parameters are determined according to the shape of the signal machine in the signal plane diagram.

[0027] According to the Hough circle detection result, remove the image that does not contain the relevant circular area.

[0028] Further, after obtaining the local area image containing the signal machine,

[0029] annotating the signal machine of the local area image by an anchor box to form a self-defined data set as input data of the signal machine recognition network.

[0030] Further, recognizing the signal machine in the signal plane diagram by using the signal machine recognition network comprises:

[0031] At the input end of the signal machine recognition network, a self-defined anchor box size suitable for the size of the signal machine is set, a self-defined data set labeled with the self-defined anchor box is taken as input data, and the selection of the anchor box is automatically completed on the self-defined data set by the signal machine recognition network.

[0032] Further, the width coefficient of the signal machine recognition network is 3 times that of the YOLOv5-s model.

[0033] Further, in the first CSP structure CSP1 in the signal machine recognition network, the number of residual components after the first CBL structure of each CSP1 is 3 times that of the corresponding YOLOv5-s model.

[0034] In the second CSP structure CSP2 in the signal machine recognition network, the number of CBL components after the first CBL structure of each CSP2 is 3 times that of the corresponding YOLOv5-s model.

[0035] Further, the signal machine recognition network increases the number of convolution kernels of Focus and CBL, so that the network depth coefficient is 1.25 times that of the YOLOv5-s model.

[0036] Further, the loss function of the signal machine recognition network includes a target positioning loss function, a target confidence loss function and a target category loss function.

[0037] The weight of the target confidence loss function is set to reduce the influence caused by the uneven number, that is, the number of boundary prediction boxes containing targets in the signal plan view is less than the number of boundary prediction boxes not containing targets.

[0038] The application provides a signal machine recognition system for signal plan based on deep learning, comprising:

[0039] An acquisition module is configured to acquire a signal plan to be recognized.

[0040] An identification module is configured to identify signal machines in the signal plan by using a signal machine recognition network.

[0041] The signal machine recognition network is a neural network based on YOLOv5-s.

[0042] Further, the acquisition module comprises a segmentation unit and a screening unit.

[0043] The segmentation unit is configured to segment the overall signal plan to form a plurality of local area images.

[0044] The screening unit is configured to screen the plurality of local area images to obtain a local area image containing a signal machine.

[0045] Further, the width coefficient of the signal recognition network is 3 times that of the YOLOv5-s model, and the depth coefficient is 1.25 times that of the YOLOv5-s model.

[0046] The application provides a signal plane diagram signal machine recognition system based on deep learning,

[0047] The system comprises at least one processor and at least one memory.

[0048] The memory stores a computer program for executing the above-mentioned deep learning-based signal plane diagram signal machine recognition method, and the processor calls the computer program in the memory to execute the above-mentioned deep learning-based signal plane diagram signal machine recognition method.

[0049] The deep learning-based signal plane diagram signal machine recognition method and system of the application are based on a YOLOv5-s network, propose a Signal-Net recognition network suitable for signal plane diagrams, and can accurately detect signal machine targets in large-resolution drawings. While ensuring the recognition accuracy of the network for signal machines, the model parameter amount is moderate, meeting the real-time operation requirements on edge computing devices.

[0050] In addition, by performing segmentation, screening and other pretreatments on the image, the efficiency and accuracy of subsequent labeling processing are improved. The Signal-Net network automatically obtains anchor boxes, simplifying the processing and analysis of specific data sets. By using a custom data set, the signal plane diagram is automatically labeled, further improving the labeling efficiency and shortening the preprocessing time.

[0051] The image recognition result of the application is accurate, and the efficiency of manually rechecking drawings in engineering application is improved.

[0052] YOLOv5 uses Pytorch as an implementation framework, which facilitates the training of custom data sets, the entire code library uniformly uses python as the programming language, does not need to be compiled, does not depend on the system, and is easy to configure the environment on the server side; can directly process various data input sources such as images, videos and network cameras, and the model weight is easy to deploy on the mobile end, which is an important advantage.

[0053] Other features and advantages of the application will be set forth in the following description of the application, and in part will become apparent to those skilled in the art, or will be learned by practice of the application. The objects and other advantages of the application will be realized and attained by the structures particularly pointed out in the written description and claims, and the appended drawings. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings described below are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0055] Figure 1 A partial station yard signal plane layout diagram of a certain station according to the prior art is shown;

[0056] Figure 2 A flowchart for acquiring an AoI partial area image according to an embodiment of the present application is shown;

[0057] Figure 3 A schematic diagram of splitting a signal plane diagram using a sliding window according to an embodiment of the present application is shown;

[0058] Fig. 4(a) shows an example diagram of splitting results of a signal plane diagram containing an incomplete signal machine after setting an overlap rate according to an embodiment of the present application;

[0059] Fig. 4(b) shows an example diagram of splitting results of a signal plane diagram containing a complete signal machine after setting an overlap rate according to an embodiment of the present application;

[0060] Figure 5 An example diagram of adjacent partial area images generated by splitting a signal plane diagram according to an embodiment of the present application is shown;

[0061] Figure 6 A Hough circle detection effect diagram of a signal plane diagram according to an embodiment of the present application is shown;

[0062] Figure 7 A partial area image containing a signal machine according to an embodiment of the present application is shown;

[0063] Figure 8 A Focus structure slicing operation schematic diagram according to an embodiment of the present application is shown;

[0064] Figure 9 A feature map pyramid schematic diagram according to an embodiment of the present application is shown;

[0065] Fig. 10(a) shows a Signal-Net model structure diagram according to an embodiment of the present application;

[0066] Fig. 10(b) shows a structure schematic diagram of a first CSP structure according to an embodiment of the present application;

[0067] Fig. 10(c) shows a structure schematic diagram of a second CSP structure according to an embodiment of the present application;

[0068] Fig. 10(d) shows a structural schematic diagram of a first CSP structure employing 3 residual components according to an embodiment of the present application;

[0069] Fig. 10(e) shows a structural schematic diagram of a second CSP structure employing 6 groups of convolutions according to an embodiment of the present application;

[0070] Figure 11 A YOLOv5-s model convolution kernel quantity diagram according to an embodiment of the present application is shown.

[0071] Figure 12 A result diagram of Signal-Net network performing detection according to an embodiment of the present application is shown.

[0072] Figure 13 A structural schematic diagram of a deep learning-based signal planar diagram signal machine recognition system according to an embodiment of the present application is shown.

[0073] Figure 14 Another structural schematic diagram of a deep learning-based signal planar diagram signal machine recognition system according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0074] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0075] The embodiments of the present application provide a deep learning-based signal planar diagram signal machine recognition method, and provide a deep learning-based signal planar diagram signal machine recognition network, i.e. Signal-Net network.

[0076] The Signal-Net network of the embodiments of the present application is implemented based on a Pytorch framework, and is used for signal machine recognition of high-resolution or super-high-resolution signal planar diagrams. Pytorch is an open-source Python machine learning library based on Torch, and is used in application fields such as computer vision and natural language processing, and provides GPU computing and deep neural networks.

[0077] Figure 2 A flowchart of acquiring an AoI (Area of Interest) local area image according to an embodiment of the present application is shown.

[0078] Before the identification, it also includes the step of splitting and labeling the large resolution signal plane. By splitting and labeling, the AoI local area image is obtained, as shown in Figure 3 The same size and different filling patterns of each adjacent circle represent the lamp of the signal machine, which jointly constitutes a signal machine. Due to the particularity of the signal machine, when the image is split, in order to avoid that the signal machine cannot be completely contained in the local area image during the image splitting, the overlap rate parameter is set, and in addition, the rejection rate parameter is set for the rejection of part of the image, and the rejection rate is not a key parameter.

[0079] (1) The signal plane is split by using a sliding window, as shown in Figure 3 The image is split by using a sliding window with a fixed size H (height) x W (width) such as (608x608), to obtain a local area image with the same size as the sliding window. In order to avoid the edge of the signal machine being incomplete when the sliding window is used, the signal plane is overlapped by an overlap rate During the sliding process, the sliding window covers all the ranges of the image, and the places passed are stored in the software and the region to be processed as local area images obtained by splitting the signal plane with a fixed size (608x608).

[0080] Taking a signal plane with a resolution of 10400x3160 as an example, if the overlap rate is not set, there will be a local area image containing an incomplete signal machine. After the overlap rate is reasonably set, although there is still an image of an incomplete signal machine, as shown in Fig. 4(a), it can be ensured that each signal machine is completely displayed in at least one local area image, as shown in Fig. 4(b). In Fig. 4(a), the signal machine on the left side of the image is not completely included in the splitting result, and there is a phenomenon that a column of circles and part of the circle edge are cut off. In Fig. 4(b), the signal machine is completely displayed in the splitting result.

[0081] From the sliding of the sliding window, it has a certain overlap rate, and an example of the adjacent local area images generated after splitting is shown in Figure 5

[0082] Exemplarily, in the embodiment of the present application, through experiments, the image splitting parameters are determined as follows: the overlap rate is 0.6, the image size is 608x608, and the rejection rate is 0.4.

[0083] Table 1 Image splitting parameters

[0084]

[0085] (2) Hough transform screening local area image

[0086] ​The split images are too many, which is not conducive to subsequent image labeling, etc. Due to the circular characteristics of the signal machine, Hough transform is used to detect the circles in each split image, and the local area images without circles are deleted.

[0087] The Hough transform screening step of the local area image includes:

[0088] The local area image is subjected to grayscale processing.

[0089] Then, median filtering processing is performed, so as to achieve the effect of noise suppression.

[0090] The processed local area image is subjected to Hough circle detection according to the Hough detection parameters.

[0091] The Hough detection parameters are determined according to the shape of the signal machine in the signal plane diagram.

[0092] The Hough transform has been encapsulated in the open source computer vision library opencv, so the HoughCircles (Hough transform circle detection) function for detecting circles is set with related parameters.

[0093] Table 2 Hough detection parameter table

[0094]

[0095] The target result obtained by the Hough circle detection is shown in Figure 6 Figure 6 The circular contour lines of different filling patterns are the detection results (the signal machine display mode in Figure 5 can be compared: some signal machines do not have obvious contour lines). In addition, Figure 6 the table and the text in are only the contents on the signal plane diagram, and have no effect on the essence of the method of the embodiment of the present application, and are not described. According to the Hough circle detection result, most of the images without related circular regions are automatically removed, and the number of images is greatly reduced. Exemplarily, a complete signal plane diagram is split by a sliding window to obtain 14713 images with a size of 608x608, most of which do not contain signal machines. After Hough transform detection, the number of images is reduced to 3733, which is 22.9% of the original number of images, and the labeling workload is greatly reduced.

[0096] In the embodiment of the present application, the large-resolution picture is first regularly split in a sliding window manner to obtain a fixed-size local area image, and then detection and screening are performed. The effective area of the signal machine in the signal plane diagram is small, and the sliding window split image will generate a large number of images that do not contain the signal machine. Therefore, according to the inherent characteristics of the circular signal machine, on the basis of the sliding window split signal plane diagram, the Hough transform is used to detect the circle, and then the AoI containing the circle is obtained, thereby improving the efficiency and reducing the occupation of computing resources.

[0097] The fixed-size local area image (an example of a split result containing a signal machine, Figure 7 is recognized. In the embodiment of the present application, the s network model structure of YOLOv5 is deeply analyzed, and on the basis thereof, the Signal-Net network for signal machine recognition is designed according to the requirements of recognition accuracy and speed.

[0098] Since the YOLOv5 network has high flexibility, in the embodiment of the present application, the s model of YOLOv5 is taken as a benchmark, and the Signal-Net network is designed according to the trade-off between accuracy and operation speed, and model training is performed.

[0099] The network structure of the Signal-Net network in the embodiment of the present application is divided into four parts of an input end, a Backbone (main network), a Neck (neck network) and a Prediction (output end).

[0100] (1) Input end

[0101] When processing the training data, the data enhancement method is mainly used. YOLOv5 provides image enhancement methods such as color space adjustment, mosaic enhancement, scaling and cropping of input images.

[0102] YOLOv5 provides adaptive anchor box selection. For the COCO (Common Objects in Context) dataset, the configuration file of YOLOv5 presets the anchor box size of the input image under 640x640, a total of three groups:

[0103] [10, 13, 16, 30, 33, 23], [30, 61, 62, 45, 59, 119], [116, 90, 156, 198, 373, 326]

[0104] The large data set contains common targets, but cannot meet the anchor box size of the signal machine in the embodiment of the application. Therefore, the Signal-Net network in the embodiment of the application sets a custom anchor box size suitable for the size of the signal machine for signal planar graph recognition, takes a custom data set (image) labeled with a custom anchor box as input data, and automatically completes the selection of the anchor box for the custom data set through the Signal-Net network, which can accelerate the training convergence speed of the model and achieve the purpose of saving training time.

[0105] Adaptive picture scaling: in common target detection algorithms, different pictures have different lengths and widths, so the common way is to uniformly scale the original picture to a standard size and then send it into the detection network. However, in actual projects, many pictures have different aspect ratios, so after scaling and padding, the size of the black edges at both ends is different, and if the padding is more, there is information redundancy, which affects the inference speed. Therefore, Yolov5 is modified to adaptively add the least black edge to the original image. In the embodiment of the application, the size of the split image is fixed to 608x608, and the width-to-height ratio of the image is 1, so adaptive picture scaling is not needed.

[0106] (2) Backbone

[0107] The Backbone adopts a Focus structure, as shown in Figure 8 The Focus structure is used to perform slicing operation, and a 3x4x4 image is sliced to obtain a 12x2x2 feature map. Taking a 3x608x608 image as an example, the Focus structure is input, and slicing operation is performed, thereby generating a corresponding feature map, and the specific specification is 12x304x304. In the convolution process, 32 convolution kernels are used for one convolution operation, and thereby a 32x304x304 feature map is obtained.

[0108] (3) Neck part of the network

[0109] The Neck part adopts a combination of FPN and PAN, and is used to generate a feature pyramid, as shown in Figure 9 The feature pyramid enhancement model is used to detect objects of different scaling sizes, so as to identify the same object of different sizes and scales. The PANet is used as a feature fusion network, and the PANet is based on the Mask-Rcnn and FPN framework.

[0110] (4) Output end

[0111] (4.1) Bounding Box loss function

[0112] YOLOv5 uses GIOU_Loss as the loss function for bounding box regression

[0113] (4.2) NMS (non maximum suppression) non maximum suppression

[0114] In the post-processing of target detection, NMS operation is performed for screening of many target boxes.

[0115] The signal recognition network of the application, i.e. Signal-Net network, is designed based on YOLOv5-s model, and the design goal is to ensure the recognition accuracy of the network for signal machines; and under the premise of improving the accuracy, the model parameter amount is moderate to ensure the real-time requirement on the edge computing device.

[0116] Specifically, the network width and depth of the Signal-Net network are set as multiples of the construction parameters of the YOLOv5-s model. For different network depth and network width configuration models, the total amount of parameters is obviously different, and the difference in the amount of parameters needs to be verified through a large number of experiments. Through the design of multiple models and verification, the network width coefficient of Signal-Net is set to 3 times that of the s model, and the network depth coefficient is set to 1.25 times that of the s model, and the parameter comparison is shown in Table 3:

[0117] Table 3 Network parameter amount comparison

[0118]

[0119] Based on the comparison and analysis in Table 3, compared with the YOLOv5-m model, the number of network layers of the Signal-Net network increases by 27%, but the total parameter amount only increases by 2.8%. Since the more the number of layers, the more the obtained features, the detection accuracy of the model is ensured; compared with the YOLOv5-l model, the Signal-Net has the same number of model layers, but the parameter amount is only 47% of the latter. Therefore, the design goal is generally achieved. The width direction parameter (width coefficient) of the Signal-Net network is set to 1, and the depth direction parameter (depth coefficient) is set to 0.625, and the parameter setting overview of each model is shown in the table:

[0120] Table 4 Model construction parameter table

[0121]

[0122] The Signal-Net network structure of the embodiment of the application is based on the s model of YOLOv5, and is self-defined according to the network width and network depth, and the network width of the Signal-Net network is 3 times that of YOLOv5-s, and the network depth is 1.25 times that of YOLOv5-s.

[0123] The network width improvement of the Signal-Net network is described below.

[0124] The correspondence between the s model network of YOLOv5 and the specific components of the Signal-Net network is shown in Table 5.

[0125] The Signal-Net model network structure is shown in FIG. 10(a). In the network structure diagram, two CSP (CrossStage Partial Network) structures are included: the first CSP structure CSP1 and the second CSP structure CSP2. Among them, the structure of CSP1 is mainly used in Backbone, and the structure of CSP2 is mainly used in Neck structure. The first CSP structure CSP1 refers to a CSP structure in which n (n≥1) ResUnits (residual components) are executed after the first CBL structure (convolution block) is executed, as shown in FIG. 10(b); the second CSP structure CSP2 refers to a CSP structure in which n (n≥1) CBL components are executed after the first CBL structure is executed, as shown in FIG. 10(c). Among them, the CBL component also refers to a group of continuously executed CBL structures.

[0126] In YOLOv5s, in the first CSP1, 1 residual component is used, denoted as CSP1_1. In Signal-Net, the network width is increased, and in the first CSP1, 3 residual components are used, denoted as CSP1_3, as shown in FIG. 10(d). In the second CSP1, 9 residual components are used, denoted as CSP1_9. In the third CSP1, 9 residual components are used, denoted as CSP1_9.

[0127] In the second CSP structure, CSP2 structure, the same way is used. Taking the first CSP2 structure as an example, in the Yolov5s component, 2×1=2 groups of CBL components are used, denoted as CSP2_1. In Signal-Net, 2×3=6 groups of CBL components are used, as shown in FIG. 10(e). The network depth is further deepened, and the ability of network feature extraction and feature fusion is continuously increased.

[0128] Conv is a convolution operation; SPP is a network structure that can output fixed-length images without considering image size; Contact represents a connection operation.

[0129] Table 5 Component quantity correspondence

[0130]

[0131] The network depth improvement of the Signal-Net network is described below.

[0132] As Figure 11 shown in Figure 2, the partial model structure of YOLOv5-s is extracted for analysis, and the number of convolution kernels of the relevant components is shown in the figure. Since the traffic signals are close to each other, it is difficult to distinguish them, and the same color signs form different traffic signals, which has certain difficulty in recognition. Therefore, the number of convolution kernels of the CBL module of Signal-Net is adjusted to 1.25 times that of the s model to increase the number of convolution kernels. The number of convolution kernels of the two is listed in detail in Table 6. Specifically, the number of Focus convolution kernels in the Signal-Net network is set to 40, the number of convolution kernels of the first CBL is set to 80, the number of convolution kernels of the second CBL is set to 160, the number of convolution kernels of the third CBL is set to 320, and the number of convolution kernels of the fourth CBL is set to 640.

[0133] Table 6 Number of convolution kernels

[0134]

[0135] Further, the Signal-Net network also improves the YOLO loss function based on the Pytorch framework. The loss function of the YOLO framework is as follows:

[0136]

[0137] wherein, is the target positioning loss function, is the target confidence loss function, is the target category loss function, describes the target category, describes the probability of whether it is a target.

[0138] (1) Target positioning loss function

[0139] The target positioning loss function can be obtained by taking the square sum of the actual deviation value of the Ground truth box (true annotation box) and the anchor box (anchor box) and the deviation value of the predicted rectangular boundary box and the anchor box.

[0140] (2) Target confidence loss function

[0141] The confidence is the probability of the existence of a certain target in the predicted boundary rectangular box, and the target confidence loss function is as follows.

[0142]

[0143] As can be seen, this term consists of two parts. The rectangular boundary prediction box has two cases for detecting the target, including detecting the target and not including detecting the target, so the first term of the formula represents a confidence loss function containing the target to be detected, and the latter represents a confidence loss function not containing the target to be detected. Since most of the split subgraphs do not contain the signal machine target after splitting in a to-be-detected image, the weight of the target confidence loss function is set to less than 1, and the influence caused by the uneven quantity is reduced by adjusting . Exemplarily, is equal to 0.8. The uneven quantity refers to the number of boundary prediction boxes containing the target in the signal plane graph being less than the number of boundary prediction boxes not containing the target. Therefore, the final overall loss function is modified as:

[0144] (3) Target category loss function

[0145] For the target category loss function, a binary classification cross-entropy loss function is adopted.

[0146] According to the shape characteristics of the signal machine, the method for obtaining the AoI region of the signal machine is proposed in combination with the existing research. Based on the YOLOv5 network basic structure, the Signal-Net network for signal machine recognition is designed by customizing the network depth and width and setting a reasonable loss function. In the process of recognizing the signal machine, various methods are mainly used, such as establishing a signal machine data set and verifying the model. The Signal-Net network of the embodiment of the application is used for signal machine detection, and the detection effect is significantly improved.

[0147] The result verification process is as follows.

[0148] (1) Evaluation standard

[0149] In the process of evaluating the recognition performance, mAP is mainly used as the evaluation standard. AP is the area generated by the precision rate, and the average AP value is mAP value, which is as follows:

[0150]

[0151]

[0152] Here, the signal machine is described, in which TP and FP correspond to correctly detecting positive signal machines and misdetecting true signal machines as negative examples, and FN represents misdetecting true signal machines as negative examples. P (Precision) represents the precision rate, and R (Recall) represents the recall rate. The AP and mAP calculation methods are as follows:

[0153] ​

[0154]

[0155] P represents the precision, R is the recall, Q represents the number of categories, and q represents the category number. Since the embodiment of the application includes signal machine counting, two new indicators, i.e., the number of missed detections and the number of false detections, are added, the former represents the number of missed signal machines, and the latter represents the number of false signal machines, so as to achieve the purpose of judging the good or bad of the model.

[0156] (2) Result analysis

[0157] For deep learning, the accuracy of target detection depends on two factors: image quality and diversity.

[0158] Since it is difficult to obtain a data set, and YOLOv5 has excellent performance on small sample data sets, a data set with only 600 images is used. The final training MAP is as high as 0.94, and the test sample is as shown in Figure 12 , which corresponds to a relatively high confidence, and in the process of detecting small targets, it shows relatively strong detection capability. Figure 12 In the figure, the probabilities of detecting three signal machines (each with 3 lights) from top to bottom as S3 type signal machines are 0.89, 0.89, and 0.84, respectively. In order to facilitate the description, the types and probabilities of the three signal machines are manually labeled in the figure.

[0159] The test set detection results are shown in Table 7: the experimental results show that the Signal-Net has advantages in detection accuracy and speed, and the average accuracy reaches 0.94, and the detection speed on the Jetson Xavier NX end reaches 20ms. The effect of the model is preliminarily verified, and the parameter amount is not significantly increased. Compared with the R-CNN network model and the YOLOV3 network model, the Signal-Net has obvious advantages.

[0160] Table 7 Test set detection results

[0161]

[0162] Based on the same inventive concept, the embodiment of the application also provides a signal plane diagram signal machine recognition system based on deep learning, as shown in Figure 13 , the system comprises:

[0163] An acquisition module is configured to acquire a signal plane diagram to be recognized.

[0164] An identification module is configured to identify signal machines in the signal plane diagram by using a signal machine identification network.

[0165] The signal machine identification network is a neural network based on YOLOv5.

[0166] Further, the acquisition module comprises a segmentation unit and a screening unit, the segmentation unit is used for segmenting the overall signal plan view to form a plurality of local area images, and the screening unit is used for screening the plurality of local area images to obtain a local area image containing the signal machine.

[0167] The width coefficient of the signal machine recognition network is 3 times that of the YOLOv5-s model, and the depth coefficient is 1.25 times that of the YOLOv5-s model.

[0168] The specific implementation mode of the signal plan view signal machine recognition system based on deep learning in the embodiment of the application can be obtained according to the signal plan view signal machine recognition method based on deep learning in any embodiment of the application, and will not be repeated.

[0169] The method of the application can be realized by a computer or an embedded program controlled system. Therefore, corresponding to this, another signal plan view signal machine recognition system based on deep learning is also provided in the embodiment of the application, as shown in the figure, the system comprises at least one processor and at least one memory; the memory stores a computer program for executing the method of any embodiment of the application, and the processor calls the computer program in the memory to execute the method of any embodiment of the application. Figure 14

[0170] Further, the memory can be communicatively connected with the one or more processors, and the memory stores instructions executable by the one or more processors, and the instructions are executed by the one or more processors to enable the one or more processors to realize the method of the application.

[0171] Although the application is described in detail with reference to the foregoing embodiments, it should be understood by those skilled in the art that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the application.​

Claims

1. A signal plane graph signal recognition method based on deep learning, characterized in that, Obtain the signal plane diagram to be identified; The overall signal plane is segmented to form multiple local region images; these local region images are then filtered to obtain the local region image containing the signal. The local region image is converted to grayscale; then, median filtering is applied to the local region image to suppress noise; finally, Hough circle detection is performed on the processed local region image based on the Hough detection parameters. Anchor boxes are used to annotate the signal generators in the local region images after Hough circular detection to form a custom dataset, which serves as the input data for the signal generator recognition network. Based on the input data, a signal recognition network is used to identify the signal in the signal plane; the signal recognition network is a neural network based on YOLOv5-s; wherein: The width factor of the signal recognition network is three times that of the YOLOv5-s model; In the first CSP structure CSP1 of the signal recognition network, the number of residual components after the first CBL structure of each CSP1 is 3 times that of the corresponding YOLOv5-s model. In the second CSP structure CSP2 of the signal recognition network, the number of CBL components after the first CBL structure in each CSP2 is 3 times that of the corresponding YOLOv5-s model; The signal recognition network increases the number of convolution kernels for Focus and CBL, making the network depth coefficient 1.25 times that of the YOLOv5-s model. The loss function of the signal recognition network includes a target localization loss function, a target confidence loss function, and a target category loss function; Set the weights of the target confidence loss function to reduce the impact of the number imbalance, which means that the number of bounding boxes containing the target in the signal plane is less than the number of bounding boxes that do not contain the target. The loss function of the YOLO framework is: ； Where, loss loc To determine the loss function for the target, loss cof For the target confidence loss function, loss cls Let λ be the target category loss function. cof The weights of the target confidence loss function.

2. The signal plane graph signal recognition method based on deep learning according to claim 1, characterized in that, Segmenting the overall signal plane diagram includes: The overall signal plane is segmented using a sliding window based on the overlap rate, so that each signal device appears completely in at least one local region image.

3. The signal plane graph signal recognition method based on deep learning according to claim 1, characterized in that, Filtering multiple local region images to obtain the local region image containing the signal as the signal plane image to be identified includes: The Hough detection parameters are determined based on the signal shape in the signal plane diagram; based on the Hough change circle detection results, images that do not contain the relevant circular regions are removed.

4. The signal plane graph signal recognition method based on deep learning according to any one of claims 1-3, characterized in that, The signal identification network used to identify the signals in the signal plane includes: At the input end of the traffic signal recognition network, a custom anchor box size adapted to the size of the traffic signal is set. A custom dataset labeled with the custom anchor boxes is used as input data, and the traffic signal recognition network automatically selects the anchor boxes from the custom dataset.

5. A signal plane diagram signal recognition system based on deep learning, used to perform the method according to any one of claims 1-4, characterized in that, include: The acquisition module is used to acquire the signal plane diagram to be identified; The identification module is used to identify the signals in the signal plane using a signal identification network; The signal recognition network is a neural network based on YOLOv5-s.

6. The signal plane diagram signal recognition system based on deep learning according to claim 5, characterized in that, The acquisition module includes a segmentation unit and a filtering unit. The segmentation unit is used to segment the overall signal plane image into multiple local region images; The filtering unit is used to filter multiple local area images to obtain the local area image containing the signal machine.

7. A signal plane diagram signal recognition system based on deep learning, characterized in that, The system includes at least one processor and at least one memory; The memory stores a computer program for performing the method of any one of claims 1-4, and the processor invokes the computer program in the memory to perform the method of any one of claims 1-4.

Images