Pipe gallery scene remnant detection method and system based on visual neural network
By using visual neural network-based preprocessing and multi-layer convolutional network to identify remnants in utility tunnel scenes, the problem of low robustness and accuracy in existing technologies is solved, achieving real-time high-precision remnant detection, which is suitable for edge devices.
Patent Information
- Application Number
- CN202511177170.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-21
- Publication Date
- 2025-12-23
AI Technical Summary
Existing technologies for detecting debris in utility tunnel environments exhibit low robustness and accuracy under complex conditions such as changes in lighting and camera shake, and also have high computational complexity, making them difficult to deploy at the edge.
A visual neural network-based approach is adopted, including preprocessing, feature extraction, and feature classification. It identifies abandoned objects through a multi-layer convolutional network and combines human joint vectors to determine and alert on abandoned objects.
It improves the robustness and accuracy of debris detection in utility tunnel scenarios, achieving real-time, high-precision debris identification and early warning, and is suitable for edge devices.
Smart Images
Figure CN121190732A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application belongs to the technical field of computer vision and intelligent monitoring, and particularly relates to a pipeline gallery scene residual object detection method and system based on a visual neural network. BACKGROUND
[0002] The urban underground comprehensive pipeline gallery is an important space for ensuring the safe operation of urban infrastructure, and its internal environment is usually closed, narrow in space, unstable in lighting, and difficult for maintenance personnel to patrol and inspect. In such an environment, workers may leave tools, devices and other objects during work, and if these residual objects are not discovered in time, they may pose a safety hazard to future operation and maintenance, and even cause pipeline blockage or short circuit and other major risks.
[0003] Existing residual object detection methods mostly rely on static image analysis or traditional motion target segmentation algorithms, such as background modeling or frame difference method. However, such methods are prone to failure in complex working conditions such as light changes, camera shaking and dynamic backgrounds, and it is difficult to ensure robustness and accuracy. In recent years, although foreground segmentation methods based on deep learning have emerged, they mostly require large-scale labeled data for training, and perform poorly when facing scene migration. In addition, some models have high computational complexity and long inference time, making it difficult to deploy them on edge or embedded devices. SUMMARY
[0004] In view of the deficiencies of the prior art, a residual object detection method suitable for pipeline gallery monitoring actual environment, with universality, real-time and high recognition accuracy is needed. The present application discloses a pipeline gallery scene residual object detection method and system based on a visual neural network.
[0005] A pipeline gallery scene residual object detection method based on a visual neural network, comprising the following steps:
[0006] An RGB image of the pipeline gallery scene monitoring camera data is obtained, and the RGB image is preprocessed to obtain a pipeline histogram;
[0007] The pipeline histogram is input into a foreground object recognition network for preprocessing, feature extraction and feature classification to obtain a label prediction result;
[0008] The label prediction result is extracted to obtain a to-be-detected region of interest;
[0009] The to-be-detected region of interest is processed based on a multi-layer convolutional network to obtain a human joint point vector set;
[0010] Based on the human joint point vector set, residual objects are discriminated and alarmed according to a preset logic.
[0011] As an implementable manner, the RGB image is preprocessed to obtain a pipeline histogram, including the following steps:
[0012] The RGB image is preprocessed by Gaussian blur convolution to obtain a preprocessed image.
[0013] The preprocessed image is normalized to obtain a normalized image.
[0014] The value of each pixel is extracted from the normalized image and updated to the histogram of the background model storage unit to obtain a pipeline histogram.
[0015] As an implementable manner, the pipeline histogram is input into a foreground object recognition network to obtain a label prediction result through preprocessing, feature extraction and feature classification, including the following steps:
[0016] The pipeline histogram is preprocessed to obtain a pixel histogram.
[0017] The pixel histogram is feature-extracted using multiple convolution kernels to obtain a pixel-level feature tensor.
[0018] The pixel-level feature tensor is classified to obtain a label prediction result.
[0019] As an implementable manner, the pipeline histogram is preprocessed to obtain a pixel histogram, including the following steps:
[0020] From the time sequence image sequence, for each pixel position, a pixel histogram is calculated.
[0021] The pixel histogram is normalized.
[0022] The calculation formula of the pixel histogram is:
[0023]
[0024] Where h x,y (n) represents a pixel histogram related to the past T frames, I i (x,y) represents the pixel value corresponding to the (x,y) position of the i-th frame, H t (x,y) represents the pixel value of the (x,y) position of the current frame, and n represents the index interval number corresponding to the pixel value.
[0025] Where h x,y (n) represents a pixel histogram related to the past T frames, I i (x,y) represents the pixel value corresponding to the (x,y) position of the i-th frame, H t(x,y) represents the pixel value of the current frame (x,y) position, and n represents the index interval number position corresponding to the pixel value of the histogram.
[0026] As an implementable manner, the feature extraction of the pixel histogram using multiple convolution kernels to obtain a pixel-level feature tensor comprises the following steps:
[0027] The product distribution convolution processing and the sum distribution convolution processing are performed on each input pixel histogram to obtain a product probability density function and a sum probability density function;
[0028] The product probability density function and the sum probability density function are spliced and input into the convolution kernel for compression fusion, and then input into a subsequent linear rectifier activation layer and a multi-layer fully connected convolution network to obtain a pixel-level feature tensor;
[0029] The product distribution convolution processing formula is:
[0030]
[0031] Wherein, Z1 represents the product distribution convolution processing result, X represents the input distribution to be calculated, and W represents the convolution kernel distribution;
[0032] The calculation formula of the product probability density function is:
[0033]
[0034] Wherein, f z1 (z) represents the product probability density function, w represents the value of the integral variable, z represents the product distribution convolution processing result, f x represents the input distribution data, and f w represents the probability distribution of the product distribution convolution kernel learning.
[0035] The sum distribution convolution processing formula is:
[0036]
[0037] Wherein, Z2 represents the sum distribution convolution processing result, X represents the input distribution to be calculated, and W represents the convolution kernel distribution.
[0038] The calculation formula of the sum probability density function is:
[0039]
[0040] Wherein, f z2 (z) represents the sum probability density function, b represents the value of the integral variable, z represents the value of the sum variable, f x represents the input distribution data, and f BThis represents the probability distribution learned by the summation distribution convolution kernel.
[0041] As one possible implementation, the step of performing feature classification on the pixel-level feature tensor to obtain the foreground object recognition result includes the following steps:
[0042] The pixel-level feature tensor is processed by two fully connected layers to obtain the three-class classification probability of the pixel;
[0043] The three-class classification probabilities of pixels are processed by a normalization layer and a comparison layer to obtain the label prediction results;
[0044] The formula for calculating the label prediction result is:
[0045]
[0046] Where c represents the label value of each pixel, 0, 1, and 2 represent the background, foreground, and uncertain labels, respectively, and x and y represent the horizontal and vertical coordinates of the pixel. This represents the initial input pixel value. This indicates the label prediction result.
[0047] As one possible implementation, extracting the label prediction results to obtain the region of interest to be detected includes the following steps:
[0048] Perform a point-by-point traversal on all foreground pixels in the label prediction result to obtain unlabeled pixels with preset values;
[0049] Starting from the pixel, using either depth-first search or breadth-first search, all foreground pixels connected to the pixel are marked to form a connected region.
[0050] For each of the connected regions, calculate the minimum bounding rectangle;
[0051] The set of minimum bounding rectangles of all connected regions is taken as the region of interest to be detected.
[0052] As one possible implementation, the process of processing the region of interest to be detected using a multi-layer convolutional network to obtain a set of human joint vectors, wherein the multi-layer convolutional network employs a structure-optimized human pose estimation model, includes the following steps:
[0053] Multi-layer convolution processing is applied to the region of interest to be detected to extract semantic features;
[0054] Aggregating semantic features at different levels through multi-scale paths;
[0055] Regression predicts the coordinates of multiple key points on each human body, outputting a set of human joint vectors consisting of two-dimensional positions and their confidence scores.
[0056] As one possible implementation, the method of identifying and alerting about abandoned items based on a set of human joint point vectors and according to preset logic includes the following steps:
[0057] Determine whether a region of interest to be detected exists;
[0058] Based on the set of human joint point vectors, determine whether a human skeleton exists;
[0059] If the human skeleton fails to remain in place for the predetermined duration, the object is identified as abandoned property, and an alarm process is initiated.
[0060] A system for detecting abandoned objects in a utility tunnel scene based on a visual neural network includes: a data acquisition module, a label prediction result acquisition module, a region of interest extraction module, a human joint vector set acquisition module, and an abandoned object discrimination module.
[0061] The data acquisition module is used to acquire RGB images of monitoring camera data of the utility tunnel scene, and preprocess the RGB images to obtain a utility tunnel histogram;
[0062] The label prediction result acquisition module is used to input the histogram of the pipe gallery into the foreground object recognition network, perform preprocessing, feature extraction and feature classification to obtain the label prediction result;
[0063] The region of interest extraction module is used to extract the label prediction results to obtain the region of interest to be detected.
[0064] The human joint vector set acquisition module is used to process the region of interest to be detected based on a multi-layer convolutional network to obtain the human joint vector set.
[0065] The abandoned object identification module is used to identify and issue alarms for abandoned objects based on the set of human joint point vectors and according to preset logic.
[0066] This invention, by adopting the above technical solutions, has significant technical effects:
[0067] 1. The foreground object recognition network of the present invention is completed by three steps: preprocessing, feature extraction, and feature classifier. The preprocessing process converts the input monitoring image into a vector suitable for the operation of the neural network model. The feature extraction uses multiple convolutional kernels to extract feature information from the input data to obtain feature vectors. The feature classifier uses fully connected layers to extract features and classifies the feature vectors obtained in the above process, which greatly improves the model's discrimination ability and training stability.
[0068] 2. The present invention uses a human skeleton recognition network consisting of a preprocessing module and a multi-layer convolutional network. First, the output of the aforementioned foreground object recognition network is used as input to extract the detected regions of interest. Then, each region of interest is input into the multi-layer convolutional network to obtain the estimation result of the human skeleton. The determination of abandoned objects uses the output of the human skeleton recognition network as input and performs the identification and alarm of abandoned objects according to preset logic, which greatly improves the accuracy and precision of abandoned object recognition in the utility tunnel scene and realizes real-time early warning. Attached Figure Description
[0069] Figure 1 This is a flowchart illustrating the method for detecting abandoned objects in a utility tunnel scene based on a visual neural network as described in this invention.
[0070] Figure 2 This is an overall schematic diagram of the pipe gallery scene debris detection system based on visual neural networks described in this invention. Detailed Implementation
[0071] The present invention will be further described below with reference to the accompanying drawings and embodiments. The following embodiments are explanations of the present invention, but the present invention is not limited to the following embodiments.
[0072] Example 1:
[0073] A method for detecting abandoned objects in a utility tunnel scene based on a visual neural network, such as... Figure 1 As shown, it includes the following steps:
[0074] S100. Acquire RGB images of the monitoring camera data of the utility tunnel scene, and preprocess the RGB images to obtain the utility tunnel histogram;
[0075] S200. Input the histogram of the pipe gallery into the foreground object recognition network, perform preprocessing, feature extraction and feature classification to obtain the label prediction result;
[0076] S300. Extract the label prediction results to obtain the region of interest to be detected;
[0077] S400. The region of interest to be detected is processed based on a multi-layer convolutional network to obtain a set of human joint vectors;
[0078] S500: Based on the set of human joint point vectors, it identifies and alarms for abandoned items according to preset logic.
[0079] In this embodiment, step S100 involves acquiring RGB images of the monitoring camera data from the utility tunnel scene and preprocessing the RGB images to obtain a utility tunnel histogram, including the following steps:
[0080] S110: Perform Gaussian blur convolution preprocessing on the RGB (Red, Green, Blue) image to obtain the preprocessed image. Specifically, a 5*5 convolution kernel is used, and the weights at each position within the kernel are defined according to the standard normal distribution density function. The definition of this density function is:
[0081]
[0082] in Let be the variance of the normal distribution, empirically chosen as 0.707, and let i and j represent the indices of the convolution kernel matrix, with values ranging from 1 to 10. .
[0083] The effect of preprocessing is to remove noise and minor perturbations from the input image, thereby improving the stability of background modeling and the robustness of subsequent feature distribution calculation.
[0084] S120: Normalize the preprocessed image to obtain a normalized image. Each pixel value is converted to an integer value. Linear mapping to floating-point numbers Within the range. The effect of normalization is to standardize pixel values into floating-point numbers, improving the speed and accuracy of subsequent numerical calculations.
[0085] S130: Extract the value of each pixel from the normalized image and update it in the histogram of the background model storage unit to obtain the pipe gallery histogram. Specifically, for a size of Image For each pixel location, a counting unit of size 201 is maintained, representing a histogram distribution from -1 to 1 with a step size of 0.01. .Will a certain point in the middle belong The value is then mapped to the interval. Then, it is included in its corresponding histogram. In this method, i represents the x-coordinate of a pixel, j represents the y-coordinate, and k represents the pixel's color channel (0, 1, 2 correspond to the R, G, and B color channels, respectively). The effect of this step is to construct an independent temporal distribution model for each pixel on each color channel, reflecting its dynamic changes in historical images and providing high-resolution input for foreground pixel recognition. Compared to traditional background modeling methods, this method possesses stronger spatiotemporal modeling capabilities.
[0086] In this embodiment, step S200 inputs the histogram of the pipe gallery into the foreground object recognition network for preprocessing, feature extraction, and feature classification to obtain the label prediction result, including the following steps:
[0087] S210: Preprocess the histogram of the pipe gallery to obtain a pixel histogram, including the following steps:
[0088] Calculate the pixel histogram for each pixel location from the time-series image sequence.
[0089] The formula for calculating the pixel histogram is:
[0090]
[0091] Among them, h x,y (n) represents the pixel histogram related to the past T frames, I i (x,y) represents the pixel value at position (x,y) in the i-th frame, H t (x,y) represents the pixel value at position (x,y) in the current frame, and n represents the pixel value corresponding to the index interval number of the histogram.
[0092] The pixel histogram is then normalized. Assuming a total of N pixels have been statistically analyzed, the histogram is organized into a three-channel input, where represents the real number field. This step normalizes the pixel values to floating-point numbers with the same range, improving the speed and accuracy of subsequent numerical calculations.
[0093] S220: Extracting features from the pixel histogram using multiple convolutional kernels to obtain a pixel-level feature tensor, including the following steps:
[0094] For each input pixel histogram, perform multiplicative distributed convolution and summation distributed convolution to obtain the product probability density function and the summation probability density function.
[0095] The formula for multiplication integral distribution convolution is:
[0096]
[0097] Where Z1 represents the result of the multiplication distribution convolution, X represents the desired input distribution, and W represents the convolution kernel distribution;
[0098] The formula for calculating the product probability density function is:
[0099]
[0100] Among them, f z1 (z) represents the product probability density function, w represents the value of the integral variable, z represents the result of the product distribution convolution, and f x f represents the input distribution data. w This represents the probability distribution of learning from the multiplicative convolution kernel;
[0101] The formula for summation distribution convolution is:
[0102]
[0103] Where Z2 represents the result of the summation distribution convolution, X represents the desired input distribution, and W represents the convolution kernel distribution;
[0104] The formula for calculating the summation probability density function is:
[0105]
[0106] Among them, f z2 (z) represents the summation probability density function, b represents the value of the integration variable, z represents the value of the summation variable, and f x f represents the input distribution data. B This represents the probability distribution learned by the summation distribution convolution kernel.
[0107] The product probability density function and the summation probability density function are concatenated and input into the convolution kernel for compression and fusion. Then, a subsequent linear rectified activation layer and a multi-layer fully connected convolutional network are used to obtain a pixel-level feature tensor.
[0108] S230: Perform feature classification on the pixel-level feature tensor to obtain the label prediction result, including the following steps:
[0109] The pixel-level feature tensor is processed by two fully connected layers to obtain the three-class classification probability of the pixel;
[0110] The three-class classification probabilities of pixels are processed by a normalization layer and a comparison layer to obtain the label prediction results;
[0111] The formula for calculating the label prediction result is:
[0112]
[0113] Where 'c' represents the label value of each pixel, with 0, 1, and 2 representing background, foreground, and uncertain labels, respectively; 'x' and 'y' represent the horizontal and vertical coordinates of the pixel, 'x' represents the initial input pixel value, and 'y' represents the label prediction result. This step fuses the two different fusion distributions described above, compressing multi-channel distribution features, thereby improving the model's discriminative ability and training stability.
[0114] In this embodiment, the human skeleton recognition network consists of a preprocessing module and a multi-layer convolutional network. Step S300 extracts the label prediction results through the preprocessing module to obtain the region of interest to be detected. In this embodiment, the aforementioned input current frame corresponds to a region of size... A binary image, wherein a pixel value of 1 represents the foreground region and a pixel value of 0 represents the background region, includes the following steps:
[0115] S310: Perform a point-by-point traversal on all foreground pixels in the label prediction result to obtain unlabeled pixels with preset values;
[0116] S320: Starting from the pixel, use a depth-first search or breadth-first search method to mark all foreground pixels connected to the pixel to form a connected region;
[0117] S330: For each of the connected regions, calculate the minimum bounding rectangle;
[0118] S340: Use the set of smallest bounding rectangles of all connected regions as the region of interest to be detected.
[0119] In this embodiment, step S400 processes the region of interest to be detected based on a multi-layer convolutional network to obtain a set of human joint vectors. The multi-layer convolutional network adopts a human pose estimation model based on the YOLO-Pose architecture with optimized structure, and includes the following steps:
[0120] S410: Perform multi-layer convolution processing on the region of interest to be detected to extract semantic features;
[0121] S420: Aggregate semantic features at different levels through multi-scale paths. The multi-layer convolutional network in this step has undergone the following structural optimizations: multi-scale feature fusion based on upsampling and downsampling is used to solve the problem of reduced accuracy in multi-person pose estimation caused by frequent scale changes in human images; a lightweight inference structure based on heavy parameterization decouples the training and inference phases of the model, accelerating the inference speed.
[0122] S430: Regresses and predicts the coordinates of multiple key points on each human body, outputting a set of human joint vectors consisting of two-dimensional positions and their confidence scores. If no joint positions are output, it means that no staff member is present. The effect of the multi-layer convolutional network module is that it can determine whether there is a staff member in the region of interest, thereby assisting in the automatic identification of leftover objects. After introducing the YOLO-Pose architecture and optimizing its structure, this step has high real-time performance and strong robustness, and can accurately determine whether there is a staff member in the region under conditions of multiple people, complex actions, or occlusion.
[0123] In this embodiment, step S500, based on the human body joint point vector set, performs the identification and alarm of abandoned objects according to preset logic, including the following steps:
[0124] S510: Determine whether there is a region of interest to be detected. If it exists, it means that there may be leftovers.
[0125] S520: Based on the set of human joint vectors, determine whether a human skeleton exists. If it does not exist, it means that an object that has not been monitored by personnel has appeared.
[0126] S530: If a human skeleton is not present for a predetermined period of time, the object is identified as abandoned and an alarm is triggered. By logically determining whether a human skeleton exists and combining this with time-based analysis, intelligent identification and alarming for "objects not monitored by personnel" are achieved, while effectively reducing the false alarm rate.
[0127] The foreground object recognition network of this invention consists of three steps: preprocessing, feature extraction, and feature classifier. Preprocessing converts the input surveillance image into a vector suitable for neural network model computation. Feature extraction uses multiple convolutional kernels to extract feature information from the input data, obtaining feature vectors. The feature classifier uses fully connected layers to extract features and classify the feature vectors obtained in the above process. The human skeleton recognition network consists of a preprocessing module and a multi-layer convolutional network. First, the output of the aforementioned foreground object recognition network is used as input to extract the detected regions of interest. Then, each region of interest is input into the multi-layer convolutional network to obtain an estimation result of the human skeleton. The abandoned object detection uses the output of the human skeleton recognition network as input and performs abandoned object identification and alarm according to preset logic.
[0128] Example 2:
[0129] A system for detecting unauthorized objects in a utility tunnel scene based on a visual neural network, such as Figure 2 As shown, it includes: a data acquisition module 100, a label prediction result acquisition module 200, a region of interest extraction module 300, a human joint vector set acquisition module 400, and a debris discrimination module 500.
[0130] The data acquisition module 100 is used to acquire RGB images of monitoring camera data of the utility tunnel scene, and preprocess the RGB images to obtain a utility tunnel histogram;
[0131] The label prediction result acquisition module 200 is used to input the pipe gallery histogram into the foreground object recognition network, perform preprocessing, feature extraction and feature classification to obtain the label prediction result;
[0132] The region of interest extraction module 300 is used to extract the label prediction results to obtain the region of interest to be detected.
[0133] The human joint vector set acquisition module 400 is used to process the region of interest to be detected based on a multi-layer convolutional network to obtain a human joint vector set.
[0134] The abandoned object identification module 500 is used to identify and alarm for abandoned objects based on the human body joint point vector set and according to preset logic.
[0135] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0136] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0137] This invention is described with reference to flowchart illustrations and / or block diagrams of the method, terminal device (system), and computer program product according to the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0138] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0139] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0140] It should be noted that:
[0141] The phrase "an embodiment" or "an embodiment" used in this specification means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the invention. Therefore, the phrase "an embodiment" or "an embodiment" appearing in various places throughout the specification does not necessarily refer to the same embodiment.
[0142] Furthermore, it should be noted that the shapes and names of the components in the specific embodiments described in this specification may differ. All equivalent or simple variations made to the structure, features, and principles described in this patent concept are included within the protection scope of this patent. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, as long as they do not depart from the structure of this invention or exceed the scope defined in these claims, they should all fall within the protection scope of this invention.
Claims
1. A method for detecting abandoned objects in a utility tunnel scene based on a visual neural network, characterized in that, Includes the following steps: RGB images of the monitoring data from the utility tunnel scene are acquired, and the RGB images are preprocessed to obtain a histogram of the utility tunnel. The histogram of the pipe gallery is input into the foreground object recognition network for preprocessing, feature extraction, and feature classification to obtain the label prediction result; The label prediction results are extracted to obtain the region of interest to be detected; The region of interest to be detected is processed using a multi-layer convolutional network to obtain a set of human joint vectors. Based on the set of human joint vectors, the system identifies and alerts users to any abandoned items according to pre-defined logic.
2. The method for detecting abandoned objects in a utility tunnel scene based on a visual neural network according to claim 1, characterized in that, The preprocessing of the RGB image to obtain the pipe gallery histogram includes the following steps: The RGB image is preprocessed by Gaussian blur convolution to obtain the preprocessed image; The preprocessed image is then normalized to obtain a normalized image. The value of each pixel is extracted from the normalized image and updated in the histogram of the background model storage unit to obtain the pipe gallery histogram.
3. The method for detecting abandoned objects in a utility tunnel scene based on a visual neural network according to claim 1, characterized in that, The step of inputting the histogram of the utility tunnel into a foreground object recognition network for preprocessing, feature extraction, and feature classification to obtain label prediction results includes the following steps: The histogram of the utility tunnel is preprocessed to obtain a pixel histogram; Multiple convolutional kernels are used to extract features from the pixel histogram to obtain a pixel-level feature tensor. The pixel-level feature tensor is used for feature classification to obtain the label prediction result.
4. The method for detecting abandoned objects in a utility tunnel scene based on a visual neural network according to claim 3, characterized in that, The preprocessing of the pipe gallery histogram to obtain a pixel histogram includes the following steps: Calculate the pixel histogram for each pixel location from the time-series image sequence; The pixel histogram is normalized. The formula for calculating the pixel histogram is: ， Among them, h x,y (n) represents the pixel histogram related to the past T frames, I i (x,y) represents the pixel value at position (x,y) in the i-th frame, H t (x,y) represents the pixel value at position (x,y) in the current frame, and n represents the pixel value corresponding to the index interval number of the histogram.
5. The method for detecting abandoned objects in a utility tunnel scene based on a visual neural network according to claim 3, characterized in that, The step of using multiple convolutional kernels to extract features from the pixel histogram to obtain a pixel-level feature tensor includes the following steps: For each input pixel histogram, perform multiplicative distributed convolution and summative distributed convolution to obtain the product probability density function and the summative probability density function. The product probability density function and the summation probability density function are concatenated and input into the convolution kernel for compression and fusion. Then, a linear rectified activation layer and a multi-layer fully connected convolutional network are connected to obtain a pixel-level feature tensor. The formula for multiplication integral distribution convolution is: ， Where Z1 represents the result of the multiplication distribution convolution, X represents the desired input distribution, and W represents the convolution kernel distribution; The formula for calculating the product probability density function is: ， Among them, f z1 (z) represents the product probability density function, w represents the value of the integral variable, z represents the result of the product distribution convolution, and f x f represents the input distribution data. w This represents the probability distribution of learning from the multiplicative convolution kernel; The formula for summation distribution convolution is: ， Where Z2 represents the result of the summation distribution convolution, X represents the desired input distribution, and W represents the convolution kernel distribution; The formula for calculating the summation probability density function is: ， Among them, f z2 (z) represents the summation probability density function, b represents the value of the integration variable, z represents the value of the summation variable, and f x f represents the input distribution data. B This represents the probability distribution learned by the summation distribution convolution kernel.
6. The method for detecting abandoned objects in a utility tunnel scene based on a visual neural network according to claim 3, characterized in that, The step of performing feature classification on the pixel-level feature tensor to obtain the foreground object recognition result includes the following steps: The pixel-level feature tensor is processed by two fully connected layers to obtain the three-class classification probability of the pixel; The three-class classification probabilities of pixels are processed by a normalization layer and a comparison layer to obtain the label prediction results; The formula for calculating the label prediction result is: ， Where c represents the label value of each pixel, 0, 1, and 2 represent background, foreground, and uncertain labels, respectively, and x and y represent the horizontal and vertical coordinates of the pixel. This represents the initial input pixel value. This indicates the label prediction result.
7. The method for detecting abandoned objects in a utility tunnel scene based on a visual neural network according to claim 1, characterized in that, The step of extracting the label prediction results to obtain the region of interest to be detected includes the following steps: Perform a point-by-point traversal on all foreground pixels in the label prediction result to obtain unlabeled pixels with preset values; Starting from the pixel, using either depth-first search or breadth-first search, all foreground pixels connected to the pixel are marked to form a connected region. For each of the connected regions, calculate the minimum bounding rectangle; The set of minimum bounding rectangles of all connected regions is taken as the region of interest to be detected.
8. The method for detecting abandoned objects in a utility tunnel scene based on a visual neural network according to claim 1, characterized in that, The process of processing the region of interest to be detected using a multi-layer convolutional network to obtain a set of human joint point vectors, wherein the multi-layer convolutional network adopts a structure-optimized human pose estimation model, includes the following steps: Multi-layer convolution processing is applied to the region of interest to be detected to extract semantic features; Aggregating semantic features at different levels through multi-scale paths; Regression predicts the coordinates of multiple key points on each human body, outputting a set of human joint vectors consisting of two-dimensional positions and their confidence scores.
9. The method for detecting abandoned objects in a utility tunnel scene based on a visual neural network according to claim 8, characterized in that, The process of identifying and alerting about abandoned items based on a set of human joint point vectors and according to preset logic includes the following steps: Determine whether a region of interest to be detected exists; Based on the set of human joint point vectors, determine whether a human skeleton exists; If the human skeleton fails to remain in place for the predetermined duration, the object is identified as abandoned property, and an alarm process is initiated.
10. A system for detecting unauthorized objects in a utility tunnel scene based on a visual neural network, characterized in that, include: The module includes a data acquisition module, a label prediction result acquisition module, a region of interest extraction module, a human joint vector set acquisition module, and a legacy object discrimination module. The data acquisition module is used to acquire RGB images of monitoring camera data of the utility tunnel scene, and preprocess the RGB images to obtain a utility tunnel histogram; The label prediction result acquisition module is used to input the histogram of the pipe gallery into the foreground object recognition network, perform preprocessing, feature extraction and feature classification to obtain the label prediction result; The region of interest extraction module is used to extract the label prediction results to obtain the region of interest to be detected. The human joint vector set acquisition module is used to process the region of interest to be detected based on a multi-layer convolutional network to obtain the human joint vector set. The abandoned object identification module is used to identify and issue alarms for abandoned objects based on the set of human joint point vectors and according to preset logic.