Abnormity recognition method and device applied to tunneling section and electronic equipment

Abstract

The embodiment of the invention discloses an anomaly recognition method and device applied to a tunneling section and electronic equipment. A specific embodiment of the method comprises the following steps: acquiring a real-time image group, a real-time humidity signal and a real-time gas signal group; performing image feature extraction on the real-time image group; performing signal feature extraction on the real-time humidity signal and the real-time gas signal group in parallel; according to the image feature group, the humidity signal feature, the gas signal feature group and a pre-trained tunneling section state prediction model, determining section state information corresponding to the tunneling section; determining an exception type according to the section state information and a pre-constructed exception decision-making tree; and according to the exception type, initiating an exception prompt. According to the embodiment, real-time and automatic abnormal recognition of the tunneling section is realized, and the construction safety is greatly improved.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technologies, and more particularly, to an abnormal recognition method, apparatus, and electronic device applied to a tunneling section. Background Art

[0002] A shield machine is a large-scale tool dedicated to tunnel excavation. Common types of shield machines include, but are not limited to: earth pressure balance shield machines, slurry balance shield machines, coal roadway tunneling machines, and special-shaped shield machines. The tunneling section refers to the working face of the shield machine cutterhead along the tunneling direction. Due to geological changes, water seepage, cave-ins, harmful gas spills may occur in the tunneling section, or the cutterhead of the shield machine may be damaged due to rock layers, etc., thus affecting the safety of tunnel excavation construction. Therefore, how to effectively identify abnormalities in the tunneling section to ensure construction safety has become an urgent problem to be solved.

[0003] The above information disclosed in this background art section is only used to enhance the understanding of the background of the inventive concept, and thus, it may include information that does not form the prior art known to those of ordinary skill in the art. Summary of the Invention

[0004] This summary of the disclosure is intended to introduce concepts in a concise form that will be described in detail in the following detailed description. This summary of the disclosure is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to be used to limit the scope of the claimed technical solution.

[0005] Some embodiments of the present disclosure propose an abnormal recognition method, apparatus, and electronic device applied to a tunneling section to solve one or more of the technical problems mentioned in the above background art section.

[0006] In a first aspect, some embodiments of the present disclosure provide a method for abnormal recognition applied to a tunneling section. The method includes: collecting a group of real-time images, a real-time humidity signal, and a group of real-time gas signals. Among them, the above-mentioned group of real-time images includes: a first real-time image and a second real-time image. The first real-time image is collected by a camera facing the cutter head opening of the shield machine from the side, and the second real-time image is collected by a camera facing the screw conveyor; extracting image features from the above-mentioned group of real-time images to generate a group of image features; through a signal feature extraction model, parallelly extracting signal features from the above-mentioned real-time humidity signal and the above-mentioned group of real-time gas signals to generate a humidity signal feature and a group of gas signal features; according to the above-mentioned group of image features, the above-mentioned humidity signal feature, the above-mentioned group of gas signal features, and a pre-trained tunneling section state prediction model, determining the section state information corresponding to the tunneling section. Among them, the above-mentioned section state information includes: gas state information, soil state information, and cutter head state information; according to the above-mentioned section state information and a pre-constructed abnormal decision tree, determining the abnormal type; according to the above-mentioned abnormal type, initiating an abnormal prompt.

[0007] In a second aspect, some embodiments of the present disclosure provide an abnormal recognition device applied to a tunneling section. The device includes: a collection unit configured to collect a group of real-time images, a real-time humidity signal, and a group of real-time gas signals. Among them, the above-mentioned group of real-time images includes: a first real-time image and a second real-time image. The first real-time image is collected by a camera facing the cutter head opening of the shield machine from the side, and the second real-time image is collected by a camera facing the screw conveyor; an image feature extraction unit configured to extract image features from the above-mentioned group of real-time images to generate a group of image features; a signal feature extraction unit configured to parallelly extract signal features from the above-mentioned real-time humidity signal and the above-mentioned group of real-time gas signals through a signal feature extraction model to generate a humidity signal feature and a group of gas signal features; a first determination unit configured to determine the section state information corresponding to the tunneling section according to the above-mentioned group of image features, the above-mentioned humidity signal feature, the above-mentioned group of gas signal features, and a pre-trained tunneling section state prediction model. Among them, the above-mentioned section state information includes: gas state information, soil state information, and cutter head state information; a second determination unit configured to determine the abnormal type according to the above-mentioned section state information and a pre-constructed abnormal decision tree; an initiation unit configured to initiate an abnormal prompt according to the above-mentioned abnormal type.

[0008] In a third aspect, some embodiments of the present disclosure provide an electronic device, including: one or more processors; a storage device having stored thereon one or more programs, which when executed by the one or more processors, cause the one or more processors to implement the method described in any implementation manner of the first aspect above.

[0009] Fourthly, some embodiments of the present disclosure provide a computer-readable medium storing a computer program thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the above first aspect is implemented.

[0010] The above various embodiments of the present disclosure have the following beneficial effects: Through the abnormal recognition method applied to the tunneling section in some embodiments of the present disclosure, effective and accurate abnormal recognition for the tunneling section is achieved, thereby ensuring construction safety. Specifically, the reason for the inability to guarantee construction safety is that due to geological changes, water seepage, collapse, harmful gas leakage may occur in the tunneling section, or the cutter head of the shield machine may be damaged due to rock layers, etc., thus affecting the safety of tunnel excavation construction. Based on this, the abnormal recognition method applied to the tunneling section in some embodiments of the present disclosure, firstly, collects a real-time image group, a real-time humidity signal, and a real-time gas signal group, wherein the above real-time image group includes: a first real-time image and a second real-time image, the above first real-time image is collected by a camera facing the cutter head incision of the shield machine from the side, and the above second real-time image is collected by a camera facing the screw conveyor. Secondly, image feature extraction is performed on the above real-time image group to generate an image feature group. In this way, an image feature expression for the real-time image is obtained. Then, through a signal feature extraction model, signal feature extraction is performed on the above real-time humidity signal and the above real-time gas signal group in parallel to generate a humidity signal feature and a gas signal feature group. In this way, signal feature expressions for the real-time humidity signal and the real-time gas signal at the same time scale are obtained. Further, according to the above image feature group, the above humidity signal feature, the above gas signal feature group, and a pre-trained tunneling section state prediction model, the section state information corresponding to the tunneling section is determined, wherein the above section state information includes: gas state information, soil state information, and cutter head state information. In this way, the section state of the tunneling section at the current moment is classified by combining images, humidity signals, and gas signals. Further, according to the above section state information and a pre-constructed abnormal decision tree, the abnormal type is determined. By combining the decision tree, the corresponding abnormal type is automatically matched. Finally, according to the above abnormal type, an abnormal prompt is initiated. Through this method, real-time and automated abnormal recognition of the tunneling section is achieved, greatly improving construction safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In combination with the accompanying drawings and with reference to the following specific embodiments, the above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic and the elements and elements are not necessarily drawn to scale.

[0012] Figure 1is a flowchart of some embodiments of an abnormal recognition method applied to a tunneling section according to the present disclosure;

[0013] Figure 2 is a schematic structural diagram of a cutter head of a shield machine;

[0014] Figure 3 is a schematic diagram of the positional relationship among the cutter head, screw conveyor, camera and sensor of a shield machine;

[0015] Figure 4 is a schematic diagram of the model structure of an image feature extraction model;

[0016] Figure 5 is a schematic diagram of the model structure of a signal feature extraction model;

[0017] Figure 6 is a schematic structural diagram of a spatial information capturer;

[0018] Figure 7 is a schematic diagram of the structure of a cross convolution block;

[0019] Figure 8 is a schematic diagram of a data dashboard corresponding to a shield machine twin;

[0020] Figure 9 is a schematic structural diagram of some embodiments of an abnormal recognition device applied to a tunneling section according to the present disclosure;

[0021] Figure 10 is a schematic structural diagram of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed implementation manners

[0022] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0023] In addition, it should be noted that only parts related to the relevant invention are shown in the drawings for the convenience of description. Without conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other.

[0024] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependent relationships.

[0025] It should be noted that the modifications of "one" and "multiple" mentioned in this disclosure are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly specified in the context, it should be understood as "one or more".

[0026] The names of the messages or information exchanged between multiple devices in the embodiments of this disclosure are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0027] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0028] Reference Figure 1 , a flowchart 100 of some embodiments of an abnormal recognition method applied to a tunneling section according to the present disclosure is shown. The abnormal recognition method applied to the tunneling section includes the following steps:

[0029] Step 101, collect a group of real-time images, a real-time humidity signal, and a group of real-time gas signals.

[0030] In some embodiments, the execution subject of the abnormal recognition method applied to the tunneling section (such as Figure 1 the computing device shown) can collect a group of real-time images, a real-time humidity signal, and a group of real-time gas signals through wired connection or wireless connection. Among them, the above-mentioned group of real-time images includes: a first real-time image and a second real-time image. The first real-time image is collected by a camera facing the cutter head opening of the shield machine from the side. The camera for collecting the first real-time image is arranged on the back side of the shield machine cutter head. The second real-time image is collected by a camera facing the screw conveyor. The camera for collecting the second real-time image is arranged on the back side of the shield machine cutter head. The real-time humidity signal is collected by a humidity sensor arranged on the back side of the shield machine cutter head. The group of real-time gas signals is collected by multiple gas sensors arranged on the back side of the shield machine cutter head. The multiple gas sensors include but are not limited to: CH4 gas sensor, CO gas sensor, H2S gas sensor, CO2 gas sensor. In practice, the shield machine cutter head is horizontal with the tunneling section to excavate along the tunneling section. If the camera and sensors are arranged on the front side of the shield machine cutter head, the camera and sensors will directly contact the excavated soil, sand, and gravel, thus increasing the probability of wear and damage of the camera and sensors. Therefore, the camera and sensors are arranged on the back side of the shield machine cutter head. As an example, refer to Figure 2 the structural schematic diagram of the shield machine cutter head shown, where the cutter head cross-section of the shield machine cutter head 1 is circular, and shield machine cutters 2 are uniformly arranged on the shield machine cutter head 1. In practice, the shield machine cutters include: cutting tools and rolling cutters. The cutting tools include but are not limited to: cutting knives, scraping knives, pilot knives, profiling knives. The rolling cutters include but are not limited to: single rolling cutters, double rolling cutters. Among them, the shield machine cutters 2 included in the shield machine cutter head can be selected according to actual needs and are not limited here.

[0031] As another example, refer to Figure 3 the schematic diagram of the positional relationship among the cutter head, screw conveyor, camera, and sensor of the shield machine shown in the figure. Among them, along the tunneling direction, the cutter head 1 of the shield machine is arranged in front of the screw conveyor 3. Specifically, a mud bin can be arranged behind the cutter head 1 of the shield machine. The screw conveyor 3 can transport the soil, sand, and gravel in the mud bin to the conveyor belt, and transport the soil, sand, and gravel out of the tunnel through the conveyor belt. Since the mud bin is a cavity area, both the sensor and the camera can be arranged inside the mud bin. Specifically, the camera 4 for collecting the first real-time image faces the cutter head 1 of the shield machine to collect the image including the back side of the cutter head 1 of the shield machine and the soil, sand, and gravel dug out by the cutter head of the shield machine in real time. The camera 5 for collecting the second real-time image faces the feeding port of the screw conveyor 3 to collect the image of the soil, sand, and gravel dug out by the cutter head 1 of the shield machine and the screw conveyor 3 in real time. The humidity sensor 6 and the gas sensor 7 can be arranged on the support member.

[0032] Optionally, since the mud bin is in a high-temperature, high-humidity, and low-light environment, the cameras arranged in the mud bin may have problems such as being blocked by sludge and poor quality of the collected real-time images due to insufficient light. Therefore, each camera is equipped with a fill light and a camera cleaning device.

[0033] It should be noted that the above wireless connection methods may include but are not limited to 3G / 4G / 5G connections, WiFi connections, Bluetooth connections, WiMAX connections, Zigbee connections, UWB (ultra-wideband) connections, and other currently known or future-developed wireless connection methods.

[0034] It should be noted that the above computing device can be hardware or software. When the computing device is hardware, it can be implemented as a distributed cluster composed of multiple servers or terminal devices, or as a single server or a single terminal device. When the computing device is embodied as software, it can be installed in the above-listed hardware devices. It can be implemented as, for example, multiple software or software modules for providing distributed services, or as a single software or software module. No specific limitation is made here.

[0035] Step 102: Extract image features from the real-time image group to generate an image feature group.

[0036] In some embodiments, the above-mentioned execution entity may perform image feature extraction on the real-time image group to generate an image feature group. In practice, the camera specifications of the camera for collecting the first real-time image and the camera for collecting the second real-time image are the same. Therefore, this ensures that the image sizes of the first real-time image and the second real-time image obtained by collection are the same. Specifically, a convolutional neural network can be used to perform image feature extraction on the first real-time image and the second real-time image in the real-time image group to generate an image feature group. To improve the image feature extraction efficiency, a parallel extraction method can be adopted to perform image feature extraction on the first real-time image and the second real-time image in parallel to obtain an image feature group.

[0037] In some optional implementation manners of some embodiments, the above-mentioned execution entity performs image feature extraction on the above-mentioned real-time image group to generate an image feature group, including:

[0038] First step, for the first real-time image in the above-mentioned real-time image group, perform the following image preprocessing steps:

[0039] First sub-step, determine the image reduction coefficient corresponding to the above-mentioned first real-time image.

[0040] Among them, there is a mapping relationship between the image reduction coefficient and the image size corresponding to the first real-time image. In practice, the mapping relationship between the image reduction coefficient and the image size can be preset, so as to avoid the problem of image feature loss caused by excessive reduction. In addition, through image reduction, the number of pixels for subsequent processing can be reduced, thereby reducing the data processing volume.

[0041] Second sub-step, according to the above-mentioned image reduction coefficient, perform image reduction on the above-mentioned first real-time image to obtain the reduced first real-time image.

[0042] As an example, the image size of the first real-time image can be H×L. The image reduction coefficient can be α. The image size of the reduced first real-time image can be αH×αL.

[0043] Third sub-step, determine the atmospheric light intensity and atmospheric refractive index parameters corresponding to the area where the camera is located.

[0044] In practice, it can be simplified that the atmospheric light intensity and atmospheric refractive index parameters corresponding to the mud sump are fixed. Specifically, first, the dark channel corresponding to the reduced first real-time image can be determined. Specifically, the dark channel can be a color channel in the reduced first real-time image. Among them, the determination of the dark channel can refer to the following formula:

[0045]

[0046] Among them, P darkDenote the dark channel. Ω(x) represents the set of regional points centered at x. P c is a color channel in the first real-time image P after reduction. {r, g, b} represent the three color channels in the first real-time image P after reduction.

[0047] Secondly, for the top 0.1% of the pixels in the dark channel of the first real-time image after reduction, find the brightest points in the corresponding color channels as the estimated value of the atmospheric light intensity. Then, after obtaining the atmospheric light intensity, it can be assumed that the atmospheric refractive index parameter is a constant. Therefore, the atmospheric refractive index parameter can be obtained by combining the dark channel and the atmospheric light intensity. In addition, considering that the atmospheric light intensity and the atmospheric refractive index parameter corresponding to the mud bin are fixed, to avoid repeated calculations, after the first calculation, the atmospheric light intensity and the atmospheric refractive index parameter can be set as fixed values, thereby further improving the data processing speed.

[0048] The fourth sub-step is to perform dark channel image defogging on the first real-time image after reduction according to the above atmospheric light intensity and the above atmospheric refractive index parameter to obtain the first real-time image after defogging.

[0049] In practice, the above execution entity can perform dark channel image defogging on the first real-time image after reduction according to the above atmospheric light intensity and the above atmospheric refractive index parameter through the dark channel defogging formula to obtain the first real-time image after defogging. Specifically, the dark channel defogging formula can be as follows:

[0050] I = J×t + A(1 - t).

[0051] Among them, I represents the first real-time image after reduction. J represents the first real-time image after defogging. A represents the atmospheric light intensity. t represents the atmospheric refractive index parameter.

[0052] In practice, since the camera is set inside the mud bin, the mud bin can be considered as a space with low brightness, approximately airtight and high temperature and humidity. At the same time, during the operation of the shield machine cutter head, soil and sand may be lifted, resulting in an approximate foggy effect on the collected images. Therefore, by combining the first sub-step to the fourth sub-step, image defogging can be quickly carried out.

[0053] The fifth sub-step is to perform image feature extraction on the first real-time image after defogging through a pre-constructed image feature extraction model to obtain the image features corresponding to the first real-time image in the above image feature group.

[0054] Among them, the above image feature extraction model includes: a low-light image enhancer and an image feature extractor.

[0055] As an example, see Figure 4Schematic diagram of the model structure of the shown image feature extraction model. The above image feature extraction model is an inventive point of the present disclosure. Among them, the image feature extraction model includes: a low-light image enhancer 402 and an image feature extractor 401. Among them, the image feature extractor 401 and the low-light image enhancer 402 are serially connected. In practice, since the camera is set in the mud bin, the mud bin can be considered as a low-brightness and approximately airtight space, resulting in low brightness of the collected images. Therefore, the present disclosure designs an image feature extraction model and refines the image feature extraction model into two parts: a low-light image enhancer 402 and an image feature extractor 401. Specifically, the image feature extractor 401 will perform preliminary image feature extraction on the first real-time image after dehazing. Considering the extraction speed and the problem of feature forgetting existing in the convolutional network. The image feature extractor 401 adopts a lightweight network structure, that is, it includes: convolutional layer A1, convolutional layer A2, convolutional layer A3, and convolutional layer A4. In addition, the low-light image enhancer 402 generates corresponding illumination feature maps through the included: convolutional layer B1, convolutional layer B2, convolutional layer B3, convolutional layer B4, convolutional layer B5, convolutional layer B6, and convolutional layer B7. At the same time, in order to further avoid feature forgetting, convolutional layer B1, convolutional layer B2, convolutional layer B3, and convolutional layer B4 are used as a group of feature extraction blocks, and the output of convolutional layer B4 is multiplied by the output of convolutional layer A4 as the input of convolutional layer B5. Immediately afterwards, the low-light image enhancer 402 also includes 2 dual-path feature extraction modules with the same model structure. Specifically, the low-light image enhancer 402 includes: a dual-path feature extraction module C and a dual-path feature extraction module D. The dual-path feature extraction module C includes: convolutional layer C1, convolutional layer C2, convolutional layer C3, convolutional layer C4, convolutional layer C5, convolutional layer C6, convolutional layer C7, and convolutional layer C8. Among them, convolutional layer C1, convolutional layer C2, convolutional layer C3, and convolutional layer C4 are downsampling networks. Convolutional layer C5, convolutional layer C6, convolutional layer C7, and convolutional layer C8 are upsampling networks. The corresponding parts of convolutional layer C1, convolutional layer C2, convolutional layer C3, and convolutional layer C4 are symmetric with the corresponding parts of convolutional layer C5, convolutional layer C6, convolutional layer C7, and convolutional layer C8. The dual-path feature extraction module D includes: convolutional layer D1, convolutional layer D2, convolutional layer D3, convolutional layer D4, convolutional layer D5, convolutional layer D6, convolutional layer D7, and convolutional layer D8. Among them, convolutional layer D1, convolutional layer D2, convolutional layer D3, and convolutional layer D4 are downsampling networks. Convolutional layer D5, convolutional layer D6, convolutional layer D7, and convolutional layer D8 are upsampling networks. The corresponding parts of convolutional layer D1, convolutional layer D2, convolutional layer D3, and convolutional layer D4 are symmetric with the corresponding parts of convolutional layer D5, convolutional layer D6, convolutional layer D7, and convolutional layer D8. Immediately afterwards, by superimposing the output of convolutional layer A4, the output of convolutional layer C8, and the output of convolutional layer D8, it is used as the input of convolutional layer E1.Finally, the sizes of the features are adjusted through convolutional layer E1 and convolutional layer E2 to extract image features from the first real-time image after defogging, and the image features corresponding to the first real-time image in the above image feature group are obtained. The above image feature extraction model can effectively perform image enhancement and image adjustment extraction on low-illuminance images.

[0056] In practice, the above execution subject can execute the above fifth sub-step in parallel to extract image features from the second real-time image after defogging through a pre-constructed image feature extraction model, and obtain the image features corresponding to the second real-time image in the image feature group. Thereby improving the extraction efficiency of image features.

[0057] Step 103, parallelly extract signal features from the real-time humidity signal and the real-time gas signal group through a signal feature extraction model to generate humidity signal features and a gas signal feature group.

[0058] In some embodiments, the above execution subject can parallelly extract signal features from the real-time humidity signal and the real-time gas signal group through a signal feature extraction model to generate humidity signal features and a gas signal feature group. In practice, considering that both the real-time humidity signal and the real-time gas signal are time-series signals, multiple parallelly arranged recurrent neural network models can be used to extract signal features from the real-time gas signal in the real-time humidity signal and the real-time gas signal group to generate humidity signal features and a gas signal feature group.

[0059] Optionally, the signal feature extraction model includes: a model activator and K + 1 sub-signal feature extraction models arranged in parallel. K is the number of real-time gas signals in the above-mentioned real-time gas signal group. The above-mentioned model activator is used to activate the sub-signal feature extraction models. The sub-signal feature extraction model is composed of a signal segmentation layer, a first normalization layer, a spatial information capturer, a second normalization layer, and a cross-convolution block. Specifically, the model activator can be set with multiple gate structures arranged in parallel. The number of gate structures = the number of gas sensors + 1 (humidity sensor). The gate structures are initially in the open state. When there are corresponding signal features, the corresponding gate structures close and activate the corresponding signal feature extraction models. In practice, the model activator can be implemented with a one-dimensional vector. When the vector value is 0, it indicates that the gate structure corresponding to the vector value is in the open state. When the vector value is 1, it indicates that the gate structure corresponding to the vector value is in the closed state. Specifically, considering the complex actual working conditions, there may be problems where the signal cannot be collected due to sensor abnormalities. At this time, if the corresponding signal feature extraction model is still activated, it will cause ineffective consumption of hardware resources (the reason is that the signal feature extraction model does not have the corresponding real-time gas signal as input). Therefore, the model activator is designed to control the model state of the signal feature extraction model. The above-mentioned signal feature extraction model, as another inventive point of the present disclosure, can effectively perform signal feature extraction on real-time humidity signals and real-time gas signals.

[0060] As an example, the gas sensors may include: a CH4 gas sensor, a CO gas sensor, an H2S gas sensor, and a CO2 gas sensor. When all gas sensors are working properly, the real-time gas signal group may include 4 real-time gas signals. When the CO2 gas sensor is working abnormally, the real-time gas signal group may include 3 real-time gas signals, that is, it does not include the real-time gas signal corresponding to the CO2 gas sensor.

[0061] As another example, refer to Figure 5 the schematic diagram of the model structure of the signal feature extraction model shown in Figure 5 The signal feature extraction model shown in

[0062] Further refer to Figure 6Schematic structural diagram of the spatial information capturer shown, where first, the spatial information capturer processes the output of the first normalization layer using the fast Fourier transform. Then, the output after the fast Fourier transform is masked by the adaptive masker. Next, the output of the adaptive masker is cross-multiplied with the local weights, and the output of the fast Fourier transform is cross-multiplied with the global weights. Finally, the output of the adaptive masker cross-multiplied with the local weights and the output of the fast Fourier transform cross-multiplied with the global weights are superimposed and subjected to the inverse Fourier transform process, serving as the input to the second normalization layer. Immediately following, refer to Figure 7 Schematic structural diagram of the cross-convolution block shown, where the cross-convolution block includes 4 convolutional layers. After the outputs of the first 3 convolutional layers are cross-multiplied and superimposed, they serve as the input to the last convolutional layer.

[0063] In some optional implementation manners of some embodiments, the above-mentioned execution entity activates the sub-signal feature extraction model corresponding to the above-mentioned real-time humidity signal through the above-mentioned model activator;

[0064] In the first step, in response to successful activation, the above-mentioned real-time humidity signal is subjected to signal feature extraction through the sub-signal feature extraction model corresponding to the above-mentioned real-time humidity signal to generate the above-mentioned humidity signal feature.

[0065] In the second step, for each real-time gas signal in the above-mentioned real-time gas signal group, the following signal feature extraction steps are performed:

[0066] In the first sub-step, the sub-signal feature extraction model corresponding to the above-mentioned real-time gas signal is activated through the above-mentioned model activator.

[0067] In the second sub-step, in response to successful activation, the above-mentioned real-time gas signal is subjected to signal feature extraction through the sub-signal feature extraction model corresponding to the above-mentioned real-time gas signal to generate the gas signal feature corresponding to the above-mentioned real-time gas signal in the above-mentioned gas signal feature group.

[0068] Step 104, according to the image feature group, humidity signal feature, gas signal feature group, and the pre-trained tunneling section state prediction model, determine the section state information corresponding to the tunneling section.

[0069] In some embodiments, the above-mentioned execution entity determines the section state information corresponding to the tunneling section according to the image feature group, humidity signal feature, gas signal feature group, and the pre-trained tunneling section state prediction model. The section state information includes: gas state information, soil state information, and cutterhead state information. Among them, the gas state information includes the concentration change trend and corresponding confidence level of the gas. The soil state information includes the soil type and corresponding confidence level. The cutterhead state information includes: cutterhead state type and corresponding confidence level.

[0070] Optionally, the tunneling section state prediction model includes: a fine-grained feature extractor and a multi-class predictor. The above multi-class predictor includes: a gas state predictor, a soil state predictor, and a cutterhead state predictor. The fine-grained feature extractor includes: an image feature fine-grained extractor for performing fine-grained image feature extraction on the image feature group, and a signal feature fine-grained extractor for performing fine-grained signal feature extraction on the humidity signal feature and the above gas signal feature group. In practice, the image feature fine-grained extractor can adopt a dual-path convolutional neural network model. Thus, the first real-time image and the second real-time image in the real-time image group can be input into the dual-path convolutional neural network model in parallel. Since the real-time humidity signal and the real-time gas signal have obvious temporal characteristics, the signal feature fine-grained extractor can adopt a recurrent neural network model. Specifically, for the humidity signal feature and each gas signal feature, there is a corresponding recurrent neural network model for fine-grained feature extraction of the signal feature. The gas state predictor, the soil state predictor, and the cutterhead state predictor all adopt multi-classifiers.

[0071] In some optional implementation manners of some embodiments, the above execution subject determines the section state information corresponding to the tunneling section according to the above image feature group, the above humidity signal feature, the above gas signal feature group, and the pre-trained tunneling section state prediction model, including:

[0072] First step, through the above fine-grained feature extractor, perform fine-grained feature extraction on the above image feature group, the above humidity signal feature, and the above gas signal feature group to obtain a fine-grained image feature group, a fine-grained humidity signal feature, and a fine-grained gas signal feature group.

[0073] Second step, according to the above fine-grained image feature group and the above soil state predictor, determine the soil state information included in the above section state information.

[0074] Third step, according to the above fine-grained image feature group and the above cutterhead state predictor, determine the cutterhead state information included in the above section state information.

[0075] Fourth step, according to the above fine-grained gas signal feature group and the above gas state predictor, determine the gas state information included in the above section state information.

[0076] Step 105, determine the abnormal type according to the section state information and the pre-constructed abnormal decision tree.

[0077] In some embodiments, the above execution subject can determine the abnormal type according to the section state information and the pre-constructed abnormal decision tree. In practice, the abnormal decision tree can be a pre-constructed decision tree for mapping the section state information and the abnormal type.

[0078] Optionally, the abnormal decision tree includes: M sub-abnormal decision trees, where M is greater than or equal to 3, and the tree depth of the sub-abnormal decision tree is less than or equal to a preset tree depth. In practice, when the tree depth of the abnormal decision tree is relatively deep, overfitting may occur during the training phase. Considering that the cross-section state information has been refined into gas state information, soil state information, and cutterhead state information, the abnormal decision tree can be divided into at least M (≥3) sub-abnormal decision trees. Specifically, when the depth of a certain sub-abnormal decision tree is relatively deep (greater than the preset tree depth), pruning or tree splitting can be further performed to construct a new sub-abnormal decision tree.

[0079] In some optional implementation manners of some embodiments, the above-mentioned execution subject determines the abnormal type according to the above-mentioned cross-section state information and the pre-constructed abnormal decision tree, including:

[0080] First step, in response to the above-mentioned gas state information indicating gas abnormality, determine the abnormal type according to the above-mentioned gas state information and the sub-abnormal decision tree in the above-mentioned M sub-abnormal decision trees that corresponds to the above-mentioned gas state information.

[0081] In practice, when the concentration change trend included in the gas state information is an upward trend and the increase exceeds the preset increase, it can indicate gas abnormality.

[0082] Second step, in response to the above-mentioned soil state information indicating soil abnormality, determine the abnormal type according to the above-mentioned soil state information and the sub-abnormal decision tree in the above-mentioned M sub-abnormal decision trees that corresponds to the above-mentioned soil state information.

[0083] In practice, when the soil type included in the soil state information generated at time T (the current time) is different from the soil type included in the soil state information generated at time T-1, or the soil type included in the soil state information is the same as the preset soil type and the corresponding confidence level is greater than the preset confidence level, it can indicate soil abnormality.

[0084] Third step, in response to the above-mentioned cutterhead state information indicating the cutterhead abnormality of the shield machine, determine the abnormal type according to the above-mentioned cutterhead state information and the sub-abnormal decision tree in the above-mentioned M sub-abnormal decision trees that corresponds to the above-mentioned cutterhead state information.

[0085] In practice, when the cutterhead state type in the cutterhead state information is consistent with the preset cutterhead state type and the corresponding confidence level is greater than the preset confidence level, it can indicate the cutterhead abnormality of the shield machine.

[0086] Step 106, initiate an abnormal prompt according to the abnormal type.

[0087] In some embodiments, the above-mentioned execution entity may initiate an exception prompt according to the exception type. Among them, the above-mentioned execution entity may determine the exception prompt corresponding to the exception type through a pre-constructed mapping table of exception types and exception prompts, and initiate the exception prompt.

[0088] In some optional implementation manners of some embodiments, the above-mentioned execution entity initiates an exception prompt according to the exception type, including:

[0089] The first step is to determine the exception level corresponding to the exception type.

[0090] In practice, the exception level corresponding to the exception type can be determined by means of mapping.

[0091] The second step is to match the exception prompt information and associated user corresponding to the above-mentioned exception type.

[0092] In practice, the exception prompt information and associated user corresponding to the above-mentioned exception type can be matched according to a pre-configured prompt template. Specifically, the prompt template is pre-configured with exception prompt information and associated users.

[0093] The third step is to send the above-mentioned exception prompt information to the above-mentioned associated user in response to the above-mentioned exception level being less than or equal to the preset exception level.

[0094] In practice, the exception prompt information can be sent to the terminal bound to the associated user.

[0095] The fourth step is to automatically execute the emergency operation corresponding to the above-mentioned exception type and broadcast the above-mentioned exception prompt information in response to the above-mentioned exception level being greater than the preset exception level.

[0096] In practice, the above-mentioned execution entity can control the shield machine to execute a preset emergency operation and broadcast the exception prompt information to all personnel in the construction area in the form of a broadcast.

[0097] Optionally, the above method further includes:

[0098] Synchronize the above-mentioned real-time image group, the above-mentioned real-time humidity signal, the above-mentioned real-time gas signal group, and the above-mentioned exception type to the shield machine twin for visual display on the shield machine twin.

[0099] Among them, the above-mentioned shield machine twin is used for three-dimensional visual display of the tunneling progress and real-time status of the shield machine. For example, see Figure 8Schematic diagram of the data dashboard corresponding to the shield machine twin shown, where the data dashboard includes: the visualized tunneling progress of the shield machine, a real-time image display dashboard for displaying real-time images (real-time image group), a signal display dashboard for displaying real-time signals (real-time humidity signal, real-time gas signal group), and a status dashboard for displaying the current status parameters of the shield machine.

[0100] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: Through the abnormal recognition method applied to the tunneling section in some embodiments of the present disclosure, effective and accurate abnormal recognition of the tunneling section is achieved, thereby ensuring the construction safety. Specifically, the reason for the inability to guarantee construction safety is that due to geological changes, water seepage, collapse, harmful gas overflow may occur in the tunneling section, or the cutter head of the shield machine may be damaged due to the rock layer, etc., thus affecting the safety of tunnel excavation construction. Based on this, in some embodiments of the present disclosure, the abnormal recognition method applied to the tunneling section first collects a real-time image group, a real-time humidity signal, and a real-time gas signal group. Among them, the above-mentioned real-time image group includes: a first real-time image and a second real-time image. The above-mentioned first real-time image is collected by a camera facing the cutter head incision of the shield machine from the side, and the above-mentioned second real-time image is collected by a camera facing the screw conveyor. Secondly, image feature extraction is performed on the above-mentioned real-time image group to generate an image feature group. In this way, an image feature expression for the real-time image is obtained. Then, through a signal feature extraction model, signal feature extraction is performed on the above-mentioned real-time humidity signal and the above-mentioned real-time gas signal group in parallel to generate a humidity signal feature and a gas signal feature group. In this way, signal feature expressions for the real-time humidity signal and the real-time gas signal are obtained at the same time scale. Further, according to the above-mentioned image feature group, the above-mentioned humidity signal feature, the above-mentioned gas signal feature group, and a pre-trained tunneling section status prediction model, the section status information corresponding to the tunneling section is determined. Among them, the above-mentioned section status information includes: gas status information, soil quality status information, and cutter head status information. In this way, the section status of the tunneling section at the current moment is classified by combining images, humidity signals, and gas signals. Further, according to the above-mentioned section status information and a pre-constructed abnormal decision tree, the abnormal type is determined. By combining the decision tree, the corresponding abnormal type is automatically matched. Finally, according to the above-mentioned abnormal type, an abnormal prompt is initiated. Through this method, real-time and automated abnormal recognition of the tunneling section is achieved, greatly improving the construction safety.

[0101] Further reference Figure 9 , as an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of an abnormal recognition device applied to the tunneling section. These device embodiments correspond to Figure 1 the method embodiments shown, and the abnormal recognition device applied to the tunneling section can be specifically applied to various electronic devices.

[0102] As Figure 9 shown, the abnormal recognition device 900 applied to the tunneling section in some embodiments includes: a collection unit 901, an image feature extraction unit 902, a signal feature extraction unit 903, a first determination unit 904, a second determination unit 905, and a triggering unit 906. Among them, the collection unit 901 is configured to collect a real-time image group, a real-time humidity signal, and a real-time gas signal group. Among them, the above real-time image group includes: a first real-time image and a second real-time image. The above first real-time image is collected by a camera facing the cutter head opening of the shield machine from the side, and the above second real-time image is collected by a camera facing the screw conveyor; the image feature extraction unit 902 is configured to perform image feature extraction on the above real-time image group to generate an image feature group; the signal feature extraction unit 903 is configured to perform signal feature extraction on the above real-time humidity signal and the above real-time gas signal group in parallel through a signal feature extraction model to generate a humidity signal feature and a gas signal feature group; the first determination unit 904 is configured to determine the section state information corresponding to the tunneling section according to the above image feature group, the above humidity signal feature, the above gas signal feature group, and a pre-trained tunneling section state prediction model. Among them, the above section state information includes: gas state information, soil state information, and cutter head state information; the second determination unit 905 is configured to determine the abnormal type according to the above section state information and a pre-constructed abnormal decision tree; the triggering unit 906 is configured to trigger an abnormal prompt according to the above abnormal type.

[0103] It can be understood that the units described in the abnormal recognition device 900 applied to the tunneling section correspond to the respective steps in the method described in the reference Figure 1 description. Therefore, the operations, features, and beneficial effects described above for the method also apply to the abnormal recognition device 900 applied to the tunneling section and the units included therein, and will not be repeated here.

[0104] Next, refer to Figure 10 , which shows a schematic structural diagram of an electronic device (for example, a computing device) suitable for implementing some embodiments of the present disclosure. Figure 10 The electronic device shown is only an example and should not impose any limitations on the functions and usage scopes of the embodiments of the present disclosure. As Figure 10As shown, the computer device includes a processor, a memory, and a network interface connected via a system bus. Among them, the memory may include a non-volatile storage medium and an internal memory. The non-volatile storage medium can store an operating system and computer programs. The computer programs include program instructions, which when executed, can cause the processor to execute any one of the front-end page monitoring methods. The processor is used to provide computing and control capabilities to support the operation of the entire computer device. The internal memory provides an environment for the operation of the computer programs in the non-volatile storage medium, and when the computer programs are executed by the processor, the processor can be caused to execute any one of the front-end page monitoring methods. The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art can understand that Figure 10 the structure shown is only a block diagram of some structures related to the solution of the present disclosure and does not constitute a limitation on the computer device to which the solution of the present disclosure is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0105] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0106] Wherein, in one embodiment, the above-mentioned processor is used to run a computer program stored in a memory to implement the following steps: collecting a group of real-time images, a real-time humidity signal, and a group of real-time gas signals, wherein the above-mentioned group of real-time images includes: a first real-time image and a second real-time image, the above-mentioned first real-time image is collected by a camera facing the cutter head opening of the shield machine from the side, and the above-mentioned second real-time image is collected by a camera facing the screw conveyor; extracting image features from the above-mentioned group of real-time images to generate a group of image features; through a signal feature extraction model, parallelly extracting signal features from the above-mentioned real-time humidity signal and the above-mentioned group of real-time gas signals to generate a humidity signal feature and a group of gas signal features; determining the cross-section state information corresponding to the tunneling cross-section according to the above-mentioned group of image features, the above-mentioned humidity signal feature, the above-mentioned group of gas signal features, and a pre-trained tunneling cross-section state prediction model, wherein the above-mentioned cross-section state information includes: gas state information, soil state information, and cutter head state information; determining the type of abnormality according to the above-mentioned cross-section state information and a pre-constructed abnormal decision tree; initiating an abnormality prompt according to the above-mentioned type of abnormality.

[0107] The embodiments of the present disclosure also provide a computer-readable storage medium, on which a computer program is stored, and the computer program includes program instructions, and the method implemented when the program instructions are executed can refer to the various embodiments of the method for abnormal recognition of a tunneling cross-section in the present disclosure.

[0108] Wherein, the above-mentioned computer-readable storage medium may be an internal storage unit of the above-mentioned computer device in the foregoing embodiment, such as the hard disk or memory of the above-mentioned computer device. The above-mentioned computer-readable storage medium may also be an external storage device of the above-mentioned computer device, such as a plug-in hard disk equipped on the above-mentioned computer device, a SmartMedia Card (SMC), a Secure Digital (SD) card, a Flash Card, etc.

[0109] It should be noted that, in this article, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article or system. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, method, article or system including that element.

[0110] The above description is only some preferred embodiments of the present disclosure and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the technical solutions formed by mutually replacing the above features with the technical features (but not limited to) disclosed in the embodiments of the present disclosure that have similar functions.

Claims

1. An abnormal recognition method applied to a tunneling section, characterized in that, Including: Collecting a real-time image group, a real-time humidity signal, and a real-time gas signal group, where the real-time image group includes: a first real-time image and a second real-time image. The first real-time image is collected by a camera facing the cutter head opening of the shield machine from the side, and the second real-time image is collected by a camera facing the screw conveyor. Performing image feature extraction on the real-time image group to generate an image feature group. Through a signal feature extraction model, parallelly performing signal feature extraction on the real-time humidity signal and the real-time gas signal group to generate a humidity signal feature and a gas signal feature group. According to the image feature group, the humidity signal feature, the gas signal feature group, and a pre-trained tunneling section state prediction model, determining the section state information corresponding to the tunneling section, where the section state information includes: gas state information, soil state information, and cutter head state information. Determining the abnormal type according to the section state information and a pre-constructed abnormal decision tree. Initiating an abnormal prompt according to the abnormal type.

2. The method according to claim 1, characterized in that, The abnormal decision tree includes: M sub-abnormal decision trees, where M is greater than or equal to 3, and the tree depth of the sub-abnormal decision tree is less than or equal to a preset tree depth; and The determining the abnormal type according to the section state information and a pre-constructed abnormal decision tree includes: In response to the gas state information indicating gas abnormality, determining the abnormal type according to the gas state information and the sub-abnormal decision tree in the M sub-abnormal decision trees corresponding to the gas state information. In response to the soil state information indicating soil abnormality, determining the abnormal type according to the soil state information and the sub-abnormal decision tree in the M sub-abnormal decision trees corresponding to the soil state information. In response to the cutter head state information indicating an abnormality of the shield machine cutter head, determining the abnormal type according to the cutter head state information and the sub-abnormal decision tree in the M sub-abnormal decision trees corresponding to the cutter head state information.

3. The method according to claim 2, wherein The initiating an abnormal prompt according to the abnormal type includes: Determining the abnormal level corresponding to the abnormal type. Matching the abnormal prompt information and the associated user corresponding to the abnormal type. In response to the abnormal level being less than or equal to a preset abnormal level, sending the abnormal prompt information to the associated user. In response to the abnormal level being greater than the preset abnormal level, automatically executing the emergency operation corresponding to the abnormal type, and broadcasting the abnormal prompt information.

4. The method according to claim 3, wherein The method further includes: Synchronizing the real-time image group, the real-time humidity signal, the real-time gas signal group, and the abnormal type to the shield machine digital twin for visual display in the shield machine digital twin, where the shield machine digital twin is used for three-dimensional visual display of the tunneling progress and real-time state of the shield machine.

5. The method according to claim 4, characterized in that, The performing image feature extraction on the real-time image group to generate an image feature group includes: For the first real-time image in the real-time image group, performing the following image preprocessing steps: Determining the image reduction coefficient corresponding to the first real-time image, where there is a mapping relationship between the image reduction coefficient and the image size corresponding to the first real-time image. Reduce the size of the first real-time image according to the image reduction coefficient to obtain the reduced first real-time image; Determine the atmospheric light intensity and atmospheric refractive index parameters corresponding to the area where the camera is located; Perform dark channel image defogging on the reduced first real-time image according to the atmospheric light intensity and the atmospheric refractive index parameters to obtain the defogged first real-time image; Extract image features from the defogged first real-time image through a pre-constructed image feature extraction model to obtain the image features corresponding to the first real-time image in the image feature group, where the image feature extraction model includes: a low-light image enhancer and an image feature extractor.

6. The method according to claim 5, wherein The signal feature extraction model includes: a model activator and K + 1 sub-signal feature extraction models arranged in parallel, where K is the number of real-time gas signals in the real-time gas signal group. The model activator is used to activate the sub-signal feature extraction models, and the sub-signal feature extraction models are composed of a signal segmentation layer, a first normalization layer, a spatial information capturer, a second normalization layer, and a cross convolution block; and The parallel signal feature extraction of the real-time humidity signal and the real-time gas signal group through the signal feature extraction model to generate humidity signal features and a gas signal feature group includes: Activate the sub-signal feature extraction model corresponding to the real-time humidity signal through the model activator; In response to successful activation, perform signal feature extraction on the real-time humidity signal through the sub-signal feature extraction model corresponding to the real-time humidity signal to generate the humidity signal features; For each real-time gas signal in the real-time gas signal group, perform the following signal feature extraction steps: Activate the sub-signal feature extraction model corresponding to the real-time gas signal through the model activator; In response to successful activation, perform signal feature extraction on the real-time gas signal through the sub-signal feature extraction model corresponding to the real-time gas signal to generate the gas signal features corresponding to the real-time gas signal in the gas signal feature group.

7. The method according to claim 6, wherein The tunneling section state prediction model includes: a fine-grained feature extractor and a multi-class predictor, and the multi-class predictor includes: a gas state predictor, a soil state predictor, and a cutter head state predictor; And determining the section state information corresponding to the tunneling section according to the image feature group, the humidity signal features, the gas signal feature group, and the pre-trained tunneling section state prediction model includes: Perform fine-grained feature extraction on the image feature group, the humidity signal features, and the gas signal feature group through the fine-grained feature extractor to obtain a fine-grained image feature group, fine-grained humidity signal features, and fine-grained gas signal features; Determine the soil state information included in the section state information according to the fine-grained image feature group and the soil state predictor; Determine the cutter head state information included in the section state information according to the fine-grained image feature group and the cutter head state predictor; Determine the gas state information included in the cross-section state information according to the fine-grained gas signal feature group and the gas state predictor.

8. An abnormal recognition device applied to a tunneling section, characterized in that, Including: An acquisition unit configured to acquire a real-time image group, a real-time humidity signal, and a real-time gas signal group, where the real-time image group includes: a first real-time image and a second real-time image, the first real-time image is acquired by a camera facing the cutter head opening of the shield machine from the side, and the second real-time image is acquired by a camera facing the screw conveyor; An image feature extraction unit configured to perform image feature extraction on the real-time image group to generate an image feature group; A signal feature extraction unit configured to perform signal feature extraction on the real-time humidity signal and the real-time gas signal group in parallel through a signal feature extraction model to generate a humidity signal feature and a gas signal feature group; A first determination unit configured to determine the cross-section state information corresponding to the tunneling cross-section according to the image feature group, the humidity signal feature, the gas signal feature group, and a pre-trained tunneling cross-section state prediction model, where the cross-section state information includes: gas state information, soil state information, and cutter head state information; A second determination unit configured to determine the abnormal type according to the cross-section state information and a pre-constructed abnormal decision tree; An initiation unit configured to initiate an abnormal prompt according to the abnormal type.

9. An electronic device, characterized in that, Including: One or more processors; A storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer-readable medium, characterized in that, Having a computer program stored thereon, where the computer program, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images