Subway flood prevention door state monitoring method based on image recognition

Abstract

The application discloses a subway flood prevention door state monitoring method based on image recognition, and combines the image recognition technology with the early warning mechanism of the subway flood prevention door; when an abnormal state of the lifting type flood prevention door is recognized, corresponding protection measures are immediately taken to prevent the risk from being enlarged, the safety in the use process of the flood prevention door is effectively improved, and timely alarm is ensured. Meanwhile, the method considers the particularity of the state recognition of the subway flood prevention door, and a subway flood prevention door image recognition model based on transfer learning is invented to perform efficient and accurate recognition operation on real-time images, and the image imaging requirement is reduced through a data enhancement method, and the robustness and generalization ability of the model are further improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image recognition, in particular to a subway floodgate state monitoring method based on image recognition. BACKGROUND

[0002] Due to the high timeliness and high traffic characteristics of the subway system, major cities in various countries will take the subway network as the backbone network of public transportation and transportation, and it is the first choice for most commuters. The subway system is mainly to dig tunnels underground to form a rail network, but when building the network, it is impossible to completely avoid possible water gushing points such as fault zones, river sections, earth sections, or network intersections. In recent years, the climate has become more extreme, and the probability of heavy rain and floods has relatively increased, which will pose a threat to the personal safety of people traveling by subway and cause property losses. In order to avoid this disaster, some cities use lifting floodgates, hydraulic swing arm floodgates, etc. to avoid subway flooding in heavy rain and unnecessary personnel injury and property loss.

[0003] From the above, the lifting floodgate has importance and necessity in the modern subway system, and its disaster prevention and mitigation effect is obvious, but how to control the opening and closing of the floodgate under the conditions of high reliability, high safety and high timeliness, it is urgent to design a more scientific and intelligent method.

[0004] Under the condition that the occurrence time of the disaster cannot be accurately predicted, the subway system maintains the normal operation of the train in the tunnel during the operation time, and the entire tunnel is completely closed after the floodgate is closed. Although disaster prevention measures must be timely and effective, they must be executed after the disaster situation is effectively confirmed to avoid false triggering or malicious activation, so the floodgate control of the lifting floodgate must avoid overdoing and always ensure the stability and reliability of the system. If this requirement needs to be met, intelligent learning algorithms and models such as deep learning can be used. The combination of lifting floodgate state detection and deep learning can greatly improve the detection efficiency, and the automatic means can control the start of the floodgate, saving a lot of manpower and material resources. SUMMARY

[0005] Therefore, the purpose of the present application is to provide a subway floodgate state monitoring method based on image recognition. To overcome the various disadvantages of the lifting floodgate control in the prior art.

[0006] The purpose of the present application is achieved by the following technical solutions:

[0007] A subway floodgate state monitoring method based on image recognition, the method comprises:

[0008] S1, the state map of subway flood prevention door is collected, and multiple subway flood prevention door original images are preprocessed to form a deep neural network image classification model, and the model is trained;

[0009] S2, the model is used for identification of real-time field images obtained on site of the flood prevention door, and once the water level at the position of the flood prevention door appears abnormal condition, a warning signal is sent and further risk avoidance measures are taken.

[0010] S3, the ResLNet network proposed in the application is used for identification and classification of subway lifting type flood prevention door state image;

[0011] S4, based on the scene of transfer learning, the model is trained on large-scale data set ImageNet, and is transferred to the map flood prevention door image data set, the accurate identification and classification of the map flood prevention door state is realized, and the specific model is referred to Figure 4 .

[0012] S5, in order to improve the classification accuracy of the model, the data enhancement technology is used for data enhancement of the collected lifting type flood prevention door state image, rich training samples are obtained for model training, and the robustness and generalization ability of the model are improved.

[0013] S501, the data enhancement technology includes scaling, horizontal / vertical flipping and image random cropping.

[0014] S502, in the model training and processing of real-time field images obtained on site of the flood prevention door, multiple flood prevention door images are processed through OpenCV library, and feature extraction is realized.

[0015] S6, the steps of OpenCV processing are as follows:

[0016] S601, the size is reset in OpenCV by using resize(·) function, and the size of subway lifting type flood prevention door image is unified;

[0017] S602, the input subway lifting type flood prevention door image is randomly cropped by the function retval=img[y:y+h,x:x+w].copy(), as an option, the size of the cropped image can be set to 330, and the input image is randomly cropped to a 330*330 (ultra-low resolution, small space occupation) image, wherein x, y are pixel values, which are coordinate values of the subway lifting type flood prevention door image cropping rectangular region, w, h are pixel values, which are width and height of the subway lifting type flood prevention door image cropping rectangular region;

[0018] S603, respectively according to the default rotation probability, using the functions im1 = opcv.flip(im, 1) and im2 = opcv.flip(im, 0) to flip the subway lift type floodgate image after random cropping horizontally / vertically, wherein opcv represents an image recognition method library, im is a subway lift type floodgate image, 0 represents flipping around the X axis, and 1 represents flipping around the Y axis;

[0019] S604, using opcv.getRotationMatrix2D and opcv.warpAffine to randomly rotate the flipped subway lift type floodgate image, calculating the radiation matrix according to the rotation angle and displacement, and returning the radiation matrix as the input of the subsequent warpAffine operation. For example: the input parameter is 15 degrees, which rotates 45 degrees counterclockwise along the positive direction of the x axis, the purpose is to make the final obtained image not lose any information, for details, see embodiment three;

[0020] S605, using the normalize(·) function to normalize the subway lift type floodgate image after rotation transformation, so that the gradient of model training is average for each picture;

[0021] S606, using the ResLNet network to realize feature extraction, and pre-training the network structure and parameters of ResLNet on the image classification data set.

[0022] S7, in image recognition, based on the scene of transfer learning, the monitored floodgate image sample is fitted by optimizing the loss function and the classifier function, the image is analyzed and the floodgate state is recognized.

[0023] S8, the vector x after feature extraction is divided into N low-dimensional embeddings x i (i∈[1,N]), the low-dimensional representation is transformed by the model, and the dot product operation is performed with the corresponding weight to obtain (w i ·x i ), and all low-dimensional vectors are aggregated to

[0024]

[0025] S9, ResLNet network uses the idea of aggregation transformation, uses a more general function to replace the elementary transformation (w i ·x i ), and the aggregation transformation can be represented as:

[0026]

[0027] Wherein, Z(x i ) is an arbitrary function, Z(x i) projecting x into an embedding representation and then transforming; J represents the number of transformations to be aggregated;

[0028] Finally, the above aggregated transformation F(x) is used as a residual function, and the following is obtained:

[0029]

[0030] Wherein, y is the output feature.

[0031] S10, using a multi-class support vector machine to classify the output feature y;

[0032] Define the loss function C to measure the degree of dissatisfaction with the estimated result; the greater the difference between the output result of the scoring function and the true result, the greater the loss function, and vice versa; the loss function of each sample is summed and averaged.

[0033] C i The loss function of the multi-class SVM of the ith data sample:

[0034]

[0035] For a training set data with N training samples corresponding to N labels, the loss function is defined as:

[0036]

[0037] Wherein, C i The loss function of the multi-class SVM of the ith data sample, k i Indicates the correct classification label, Indicates the score on the correct classification label, r q Indicates the score on the incorrect classification label, and N is the number of samples; by optimizing the loss function, the SVM classifier function fits the image sample so that the output result of most samples meets the target function greater than 0; the farther the image is from the hyperplane, the greater the absolute value of the score, indicating correct classification.

[0038] S11, the control circuit of the subway floodgate and the floodgate image acquisition device are connected with the near-end monitoring system arranged in the subway station, the near-end monitoring system is connected with the far-end monitoring center arranged outside the station, the far-end monitoring center is also connected with the signal system of the subway control center, and the algorithm of the subway floodgate state detection image recognition based on the migration learning technology is executed.

[0039] S12, when the far-end monitoring center identifies that the water level of the on-site image of any subway station is abnormal, it immediately reports to the signal system of the subway control center, determines the tunnel clearance, and remotely closes the lifting type floodgate; when the remote control cannot be normally executed, the near-end monitoring console is authorized to execute the near-end remote control closing operation.

[0040] The beneficial effects of the present application are:

[0041] (1) The present application combines image recognition technology with the prevention mechanism of subway flood prevention doors. When an abnormal state of the lifting type flood prevention door is identified, appropriate protective measures are immediately taken to prevent the risk from expanding, realizing intelligent control of subway flood prevention doors and effectively improving the safety and timeliness during the use of flood prevention doors.

[0042] (2) The present application proposes a ResLNet deep learning network that is different from the traditional ResNet network. The model has fewer parameters, and can improve the accuracy of image recognition without increasing the computational cost. In addition, the ResLNet network structure is simpler and more versatile.

[0043] (3) The present application takes into account the particularity of subway flood prevention door state recognition, and designs a suitable image recognition model for efficient and accurate identification of real-time images. At the same time, through data enhancement methods such as scaling, horizontal / vertical flipping and image random cropping, the requirements of image imaging are reduced, and the robustness and generalization ability of the model are further improved.

[0044] (4) The present application adopts a multi-layer control mode. The control circuit of the subway flood prevention door and the flood prevention door image acquisition device are connected with the near-end monitoring system set in the subway station. The near-end monitoring system is connected with the remote monitoring center set outside the station. The remote monitoring center is also connected with the signal system of the subway control center, and remote real-time control can be realized.

[0045] Other advantages, objects, and features of the present application will be set forth in part in the following specification, and in part will become apparent to those skilled in the art from a consideration of the following specification and from the practice of the application. The objects and other advantages of the present application can be realized and attained by the means recited in the following specification and claimed in the appended claims. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to make the objects, technical solutions and advantages of the present application clearer, the following will further describe the present application in combination with the drawings, in which:

[0047] Figure 1 is a schematic diagram of the method of the present application;

[0048] Figure 2 is an original subway flood prevention door image;

[0049] Figure 3 is a schematic diagram of the SVM multi-classifier;

[0050] Figure 4An image classification model based on transfer learning;

[0051] Figure 5 A subway flood gate image detection model based on transfer learning technology;

[0052] Figure 6 A subway flood gate image recognition accuracy result comparison. DETAILED DESCRIPTION

[0053] The preferred embodiments of the present application will be described in detail below with reference to the accompanying drawings. It should be understood that the preferred embodiments are only for the purpose of illustrating the present application, and are not intended to limit the protection scope of the present application.

[0054] Example one

[0055] As shown in the figure, a subway flood gate state monitoring method based on image recognition of the present application mainly includes: Figure 1

[0056] I. Model establishment and training: collect the state map of the subway flood gate, and pre-process a large number of original images of the subway flood gate to form a deep neural network image classification model, and train the model;

[0057] II. Apply the model to the recognition of on-site images and alarm for abnormal conditions: apply the model to the recognition of real-time on-site images obtained on site of the flood gate, and once an abnormal condition of the water level at the location of the flood gate is found, issue a warning signal and take further risk avoidance measures.

[0058] Among them, in the processing and recognition of images, the present embodiment uses ResLNet network to recognize and classify the state images of the subway lifting type flood gate; based on the scene of transfer learning, the model is trained on a large-scale data set ImageNet, and is migrated to the map flood gate image data set to realize accurate recognition and classification of the map flood gate state; in order to further improve the classification accuracy of the model, the collected lifting type flood gate state images are subjected to data enhancement using data enhancement technology to obtain rich training samples for model training, thereby improving the robustness and generalization ability of the model. Data enhancement techniques include but are not limited to scaling, horizontal / vertical flipping, and image random cropping techniques.

[0059] In terms of specific device settings, the present embodiment adopts a multi-layer control mode, the control circuit of the subway flood gate and the flood gate image acquisition device are connected with the near-end monitoring system set in the subway station, the near-end monitoring system is connected with the far-end monitoring center set outside the station, the far-end monitoring center is also connected with the signal system of the subway control center, and the algorithm of the subway flood gate state detection image recognition based on transfer learning technology is executed.

[0060] ​When the remote monitoring center identifies that the water level of any subway station is abnormal, the signal system of the subway control center is notified immediately, and the lift-type floodgate is closed in a remote manner after determining the tunnel clearance. When the remote control cannot be normally executed, the near-end monitoring console is authorized to execute the near-end remote control closing operation. Taking the existing lift-type subway floodgate as an example, the closing operation is to issue an instruction and start the opening and closing machine of the lifting door leaf through the control line of the subway floodgate, move the baffle trolley that bears the door leaf away from the current position, and push the electric push rod that extends and retracts. When the door is difficult to close due to special circumstances, the downward pressing mechanism should also be started. With the above-mentioned multi-layer control and hierarchical authorization management mechanism, the reliability of system management and the timeliness of system response can be considered.

[0061] Embodiment Two

[0062] In the model training and processing of real-time field images obtained on the site of the floodgate, the present embodiment processes multiple floodgate images through OpenCV technology and realizes feature extraction.

[0063] The steps of processing through OpenCV are as follows:

[0064] (1) The resize(·) function is used to reset the size and unify the input subway lift-type floodgate image size;

[0065] (2) The retval=img[y:y+h,x:x+w].copy() function is used to randomly crop the input subway lift-type floodgate image. In the present embodiment, the size of the cropping is set to 330, and the input image is randomly cropped to a 330*330 image. This parameter setting can achieve the technical effect of ultra-low resolution and small space occupation, where x and y are pixel values, which are the coordinate values of the rectangular region of the subway lift-type floodgate image, and w and h are pixel values, which are the width and height of the rectangular region of the subway lift-type floodgate image;

[0066] (3) The im1=opcv.flip(im,1) and im2=opcv.flip(im,0) functions are used to horizontally / vertically flip the randomly cropped subway lift-type floodgate image according to the default rotation probability, where opcv represents the image recognition method library, im is the subway lift-type floodgate image, 0 is the rotation around the X-axis, and 1 is the rotation around the Y-axis;

[0067] (4) Using opcv.getRotationMatrix2D and opcv.warpAffine to randomly rotate the flipped subway lift-type floodgate image, and according to the rotation angle and displacement to calculate the radiation matrix, return the radiation matrix as the input of the subsequent warpAffine operation. For example: the input parameter is 15 degrees, then rotate 45 degrees counterclockwise along the positive direction of the x axis, the purpose is to make the final image not lose any information.

[0068] The specific process is as follows:

[0069]

[0070] Rotate the image center point (u x, u y ) by β degrees, γ = cosβ, θ = sinβ.

[0071] (a) A certain point in the original image: The point moves from the center point to the origin

[0072] (c) Rotate The point moves from the origin back to the center point

[0073] Where x and y represent the coordinates of a certain point in the original image, (u x, u y ) represents the image center point.

[0074] The above four steps (a)-(d) realize the rotation of the picture around the center point by β degrees.

[0075] (5) Use the normalize(·) function to normalize the subway lift-type floodgate image after rotation transformation, so that the gradient of the model training is average for each picture;

[0076] (6) Use ResLNet to realize feature extraction, and pre-train the network structure and parameters of ResLNet on the image classification dataset.

[0077] Example Three

[0078] As a further improvement, in the specific image recognition, this embodiment is based on the scene of transfer learning, by optimizing the loss function, the classifier function fits the image sample, analyzes the image result, the specific process is as follows:

[0079] (1) The vector x is divided into N low-dimensional embeddings x i , and the low-dimensional representation is transformed by the model, and the dot product with the corresponding weight is obtained (w i ·xi ), all low-dimensional embedding is aggregated to obtain

[0080] (2) ResLNet uses the idea of aggregation transformation, and uses a more general function to replace the elementary transformation (w i ·x i ), and the aggregation transformation can be expressed as:

[0081] (where i≥1)

[0082] where Z(x i ) is an arbitrary function, and Z(x i ) projects x into an embedding representation and then performs transformation.

[0083] (3) In the above formula, J is the number of the set of transformations to be aggregated, called the cardinality, and finally, the above formula aggregates the transformation to be used as a residual function, and obtains:

[0084]

[0085] where y is the output.

[0086] (4) The output feature y is classified by using a multi-class support vector machine.

[0087] Define the loss function C to measure the degree of dissatisfaction with the estimated result. The greater the difference between the output result of the scoring function and the true result, the greater the loss function, and vice versa. The loss function of each sample is summed and averaged.

[0088] C i is the loss function of the multi-class SVM of the ith data sample:

[0089]

[0090] For a training set data with N training samples corresponding to N labels, the loss function is defined as:

[0091]

[0092] where C i is the loss function of the multi-class SVM of the ith data sample, k i represents the correct classification label, r ki represents the score on the correct classification label, r q represents the score on the incorrect classification label, and N is the number of samples. By optimizing the loss function, the SVM classifier function fits the image samples so that the output results of most samples meet the target function greater than 0. The farther the image is from the hyperplane, the greater the absolute value of the score, indicating that the classification is correct.

[0093] To further verify the effectiveness of the image recognition of the present application, experiments are carried out for two different scenes of single background and complex background existing in the image data of the subway lifting type floodgate. In the single background, each image has only one target object. In the single background, the average classification accuracy of the model is as high as 92.91%, verifying the effectiveness of the migration model. When an abnormal state of the lifting type floodgate is recognized, the corresponding abnormal information is immediately sent to the image recognition monitoring point. In the complex background, each subway lifting type floodgate image has multiple objects, including complex background interference. In the complex background, the average classification accuracy of the model can reach 85.31%, which shows that the model has good feature extraction ability for impurities in the complex background and good ability to distinguish the background and foreground of the image.

[0094] It should be noted that any processes or methods described in the flowcharts of the present application or otherwise described herein can be understood as representing code modules, segments, or portions of code that include one or more executable instructions for implementing specific logic functions (or steps) in the process, and that the various embodiments of the present application can include additional or fewer steps performing the same or equivalent functions as those shown or discussed, in different orders, in different combinations, or omitted altogether, as would be understood by one of ordinary skill in the art of the present application.

[0095] The logic and / or steps represented in the flowcharts and / or otherwise described herein, for example, can be considered as a list of executable instructions to implement logic functions, which can be specifically embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor- based system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this specification, a "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection having one or more wires (electrical apparatus), a portable computer diskette (magnetic apparatus), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber (optical apparatus), and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium can even be paper or other suitable medium on which the program can be printed, as the program can be electronically obtained, for example, by optical scanning of the paper or other medium, followed by electronic conversion of the optical scanning into the program, and then storing the program in a computer memory.

[0096] Finally, it is to be explained that the above embodiments are only used to illustrate the technical solutions of the present application but not to limit the present application. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or equivalently replaced without departing from the purpose and scope of the present application, and all of them should be covered in the scope of the claims of the present application.

Claims

1. A subway flood prevention door state monitoring method based on image recognition, characterized in that: The method comprises: Collect the state diagram of the subway floodgate, and preprocess multiple subway floodgate original images to form a deep neural network image classification model, and train the model; Use the model for real-time image recognition obtained on site of the floodgate, and once an abnormal situation is found in the water level at the location of the floodgate, an early warning signal is sent and further risk avoidance measures are taken, and in image recognition, based on the scene of transfer learning, the loss function and the classifier function are optimized to fit the samples, the image results are analyzed, and specifically include: The vector x after feature extraction is divided into N low-dimensional embeddings x i (i∈[1, N]) and the low-dimensional representation is transformed by the model, respectively, and the corresponding weight is multiplied to obtain (w i • x i ), and all low-dimensional vectors are aggregated ; ResLNet uses a more general function instead of elementary transformation (w i • x i ), and the aggregation transformation can be represented as: (where i > 1) Where Z(x) i Z(x) is an arbitrary function. i Project x into an embedded representation and then perform the transformation; J is the number of transformation sets to be aggregated; Finally, the above aggregation conversion F(x) is used as a residual function to obtain: Where y is the output feature; The output feature y is classified by using a multi-class support vector machine; Define a loss function C to measure the degree of dissatisfaction with the estimated result; the greater the difference between the output result of the scoring function and the true result, the greater the loss function, and vice versa; the loss function of each sample is summed and averaged; C i Loss function for multiclass SVM for the i-th data sample: For a training set data with N training samples corresponding to N labels, the loss function is defined as: where C i is the loss function of multi-class SVM for the i-th data sample, k i represents the correct classification label, r ki represents the score on the correct classification label, r q represents the score on the incorrect classification label, and N is the number of samples; by optimizing the loss function, the SVM classifier function fits the image samples so that the results output by most samples meet the target function greater than 0; the farther the image is from the hyperplane, the greater the absolute value of the score, indicating correct classification. 2.The subway flood prevention door state monitoring method based on image recognition according to claim 1, characterized in that: ResLNet network is used for recognition and classification of subway lifting type floodgate state images; To improve the classification accuracy of the model, data enhancement technology is used to enhance the collected lifting type floodgate state images, obtain rich training samples for model training, and improve the robustness and generalization ability of the model. 3.The subway flood door state monitoring method based on image recognition of claim 1 or 2, characterized in that: The data enhancement technology includes scaling, horizontal / vertical flipping and random image cropping.

4. The subway flood door state monitoring method based on image recognition according to claim 1, characterized in that: In the model training and processing of real-time images obtained on site of the floodgate, multiple floodgate images are processed by OpenCV, and feature extraction is realized.

5. The subway flood door state monitoring method based on image recognition according to claim 4, characterized in that: Further, the steps of OpenCV processing are as follows: In OpenCV, the resize(•) function is used to reset the size and unify the input subway lifting type floodgate image size; The retval=img[y:y+h, x:x+w].copy() function is used to randomly crop the input subway lifting type floodgate image, where x, y are pixel values, which are the coordinate values of the subway lifting type floodgate image cropping rectangular region, w, h are pixel values, which are the width and height of the subway lifting type floodgate image cropping rectangular region; The im1=opcv.flip(im, 1) and im2=opcv.flip(im, 0) functions are used to horizontally / vertically flip the randomly cropped subway lifting type floodgate image according to the default rotation probability, where opcv represents the image recognition method library, im is the subway lifting type floodgate image, 0 is the rotation around the X axis, and 1 is the rotation around the Y axis; The opcv.getRotationMatrix2D and opcv.warpAffine functions are used to randomly rotate the flipped subway lifting type floodgate image, calculate the radiation matrix according to the rotation angle and displacement, return the radiation matrix as the input of the subsequent warpAffine operation; The normalized subway lift-type floodgate image after the rotation transformation is normalized by using the normalize(•) function, so that the gradient of the model training is average for each picture.

6. The subway flood door state monitoring method based on image recognition according to claim 5, characterized in that: In the process of randomly cropping the input subway lift-type floodgate image, the size of the crop is set to 330, and the input image is randomly cropped to a 330*330 image.

7. The subway flood door state monitoring method based on image recognition according to claim 1, characterized in that: The control circuit and the floodgate image acquisition device of the subway floodgate are connected with a near-end monitoring system arranged in the subway station, the near-end monitoring system is connected with a far-end monitoring center arranged outside the station, the far-end monitoring center is also connected with a signal system of a subway control center, and an algorithm of subway floodgate state detection image recognition based on a migration learning technology is executed. 8.The subway flood door state monitoring method based on image recognition of claim 7, wherein: When the far-end monitoring center identifies that the water level of the on-site image of any subway station is abnormal, the signal system of the subway control center is immediately notified, the lift-type floodgate is closed in a remote control mode after the tunnel clearance is determined, and when the far-end remote control cannot be normally executed, the near-end monitoring console is authorized to execute a near-end remote control closing operation.

Images