Abnormal target detection method and device, edge computing equipment and storage medium

Abstract

The invention provides an abnormal target detection method and apparatus, an edge computing device and a storage medium. The method comprises the steps of constructing a comprehensive data set based on multi-source image data; training a target detection pre-training model through the comprehensive data set to obtain a target detection training model; and inputting field video monitoring data into the target detection training model, and outputting abnormal information when an abnormal target is detected. According to the technical scheme provided by the embodiment of the invention, the abnormal target is intelligently detected through the edge detection equipment provided with the target detection training model, the real-time monitoring of the working site can be realized without the need of the inspection personnel to arrive at the gas working site for inspection, the real-time performance is high, the labor cost is saved, the accident risk in the inspection work is reduced, and the working efficiency is improved. And the abnormal target detection efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, and in particular to an abnormal target detection method and device, an edge computing device, and a storage medium. BACKGROUND

[0002] In related technologies, the traditional gas inspection method is on-site inspection, which mainly relies on manual experience judgment and periodic inspection, and has problems such as low inspection efficiency, delayed response, and high missing rate, and is difficult to meet the dynamic monitoring needs of gas pipelines, underground pipe networks, and auxiliary facilities in complex scenarios. SUMMARY

[0003] In view of the above problems, the embodiments of the present application provide an abnormal target detection method, device, edge computing device, and storage medium to solve the problems of low inspection efficiency, delayed response, and high missing rate in the prior art on-site inspection relying on manual experience judgment and periodic inspection.

[0004] In a first aspect, the embodiments of the present application provide an abnormal target detection method, which comprises:

[0005] constructing a comprehensive data set based on multi-source image data;

[0006] training a target detection pre-training model through the comprehensive data set to obtain a target detection training model;

[0007] inputting on-site video monitoring data into the target detection training model and outputting abnormal information when an abnormal target is detected.

[0008] In a possible implementation manner, the multi-source image data comprises real scene data and enhanced data; before the comprehensive data set is constructed based on the multi-source image data, the method further comprises:

[0009] enhancing the real scene data through an image enhancement algorithm to obtain enhanced data.

[0010] In a possible implementation manner, the image enhancement algorithm comprises at least one of a Mosaic algorithm, a Random Affine affine transformation algorithm, a MixUp image fusion algorithm, a color adjustment algorithm, and a two-axis mirror transformation algorithm.

[0011] In a possible implementation manner, the multi-source image data comprises synthetic data; before the comprehensive data set is constructed based on the multi-source image data, the method further comprises:

[0012] constructing a simulated scene with target objects through a three-dimensional modeling software;

[0013] orthogonal projection is performed on the target object to generate a reference contour of the target object;

[0014] determining whether a foreground object is occluded to the target object;

[0015] if it is determined that no foreground object is occluded to the target object, determining the reference contour as a target contour;

[0016] if it is determined that a foreground object is occluded to the target object, generating a composite contour containing the foreground object and the target object, and determining a target contour from the composite contour;

[0017] labeling the target contour to obtain synthetic data.

[0018] In a possible implementation, the determining of the target contour from the composite contour comprises:

[0019] extracting a contour set from the composite contour by a neighbor boundary tracing algorithm, the contour set containing a plurality of original contours;

[0020] selecting a candidate contour from the plurality of original contours based on a preset area interval, an endpoint value of the area interval being set based on an area of the reference contour;

[0021] determining a target contour from the candidate contour based on a Hu moment of each candidate contour and a Hu moment of the reference contour, the target contour being a contour with the highest similarity to the reference contour.

[0022] In a possible implementation, the training of the target detection pre-training model based on the synthetic data set to obtain a target detection training model comprises:

[0023] dividing the synthetic data set into a first training subset, a validation subset and a second training subset;

[0024] training the target detection pre-training model based on the first training subset to obtain a target detection intermediate model;

[0025] evaluating a performance index of the target detection intermediate model based on the validation subset, and adjusting hyperparameters based on the performance index;

[0026] when the performance index of the target detection intermediate model reaches a preset threshold, determining the adjusted hyperparameters and a first training weight;

[0027] adjusting a confidence threshold to obtain a performance index change curve corresponding to different confidence thresholds, and determining an optimal confidence threshold;

[0028] Through the second training subset, the target detection intermediate model is trained based on the adjusted hyperparameters, the first training weight and the optimal confidence threshold, to obtain a target detection training model.

[0029] In a possible implementation, the performance indicators include accuracy, precision, recall and F1 score.

[0030] In a second aspect, an embodiment of the present application provides an abnormal target detection device, and the device comprises:

[0031] The construction module is configured to construct a comprehensive data set based on multi-source image data.

[0032] The training module is configured to train a target detection pre-training model through the comprehensive data set to obtain a target detection training model.

[0033] The detection module is configured to input live video monitoring data into the target detection training model and output abnormal information when an abnormal target is detected.

[0034] In a third aspect, an embodiment of the present application provides an edge computing device, and the edge computing device is deployed with the target detection training model of the first aspect.

[0035] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, and the computer readable storage medium comprises a stored program, wherein the program controls a device where the computer readable storage medium is located to perform the abnormal target detection method of the first aspect or any possible implementation of the first aspect when the program is running.

[0036] In the technical solution provided by the embodiment of the present application, the edge detection device deployed with the target detection training model intelligently detects abnormal targets, real-time monitoring of the operation site can be realized without the need of a patrol personnel to arrive at the gas operation site for patrol, human cost is saved, and the risk of accidents in the patrol work is reduced.

[0037] In the embodiment of the present application, the comprehensive data set constructed based on multi-source image data is used as training data for model training, which significantly improves the generalization ability of the model in complex and changeable scenes, enables the target detection model to be applied to various scenes, and improves the practicability and reliability. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 A flowchart of an abnormal target detection method provided by an embodiment of the present application.

[0039] Figure 2 A flowchart of generating a synthetic data in an abnormal target detection method provided by an embodiment of the present application.

[0040] Figure 3 A schematic diagram of determining a target contour from a composite contour in an abnormal target detection method provided by an embodiment of the present application.

[0041] Figure 4 A flowchart of a training method of a target detection training model in an abnormal target detection method provided by an embodiment of the present application.

[0042] Figure 5 A structural schematic diagram of an abnormal target detection device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0043] To make the objectives, technical solutions, and advantages of the present application clearer and more comprehensible, the present application is further described below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort belong to the scope of protection of the present application.

[0044] Figure 1 A flowchart of an abnormal target detection method provided by an embodiment of the present application, as shown in Figure 1 The method comprises the following steps.

[0045] Step 101: constructing a comprehensive data set based on multi-source image data.

[0046] In the embodiment of the present application, the multi-source image data comprises real scene data, and the real scene data is obtained through the Internet. A preset keyword is used to perform data retrieval and data collection in the Internet to obtain retrieval image data. For example, the keyword is “manhole cover image” or “gas pipeline image”. A Natural Language Processing (NLP) model is used to perform semantic understanding on the retrieval image data, and relevant image data is selected from the retrieval image data. Resolution and clarity filtering is performed to select real scene data from the relevant image data.

[0047] Specifically, the NLP model is used to perform semantic understanding on the text description in the image metadata of the retrieval image data, and irrelevant images are filtered out to obtain relevant image data. Resolution and clarity filtering is performed to select high-quality images that meet the resolution and clarity requirements from the relevant image data as real scene data, and low-quality images that are blurred or out of focus are removed to improve the quality of the training data. The resolution and clarity filtering is performed as follows. A preset resolution condition is used to select images that meet the resolution requirement. An edge sharpness detection algorithm is used to select images that meet the clarity requirement. For example, the resolution condition is that the minimum side length of the image data is greater than or equal to 512 pixels.

[0048] In the embodiment of the present application, the real scene data includes scene data under different shooting devices, different light intensities and different weather conditions. For example, the real scene data includes image data or video data under different environmental characteristics such as indoor, outdoor, daytime and night. By introducing scene data covering different environmental characteristics, the data coverage range of the comprehensive data set is more extensive, so that the target detection training model trained based on the comprehensive data set has better generalization ability and robustness.

[0049] In the embodiment of the present application, the real scene data is divided into unoccluded scene data and occluded scene data, and the occluded scene data is divided into two categories of complete occlusion and partial occlusion to meet the application requirements of the target detection training model in the occluded scene. For example, the occluded scene data includes common typical occluded scenes such as the manhole cover being occluded by the earthmoving vehicle in the construction scene and the gas pipeline being covered by the vegetation. By introducing the occluded scene data, the target detection training model can cope with complex situations with occluded relationship, and the adaptability of the model to unknown scenes is improved.

[0050] In the embodiment of the present application, the multi-source image data further includes enhanced data. The real scene data is enhanced by an image enhancement algorithm to obtain the enhanced data. The image enhancement algorithm includes at least one of a Mosaic algorithm, a Random Affine affine transformation algorithm, a MixUp image fusion algorithm, a color adjustment algorithm and a two-axis mirror transformation algorithm. In actual application, only one image enhancement algorithm can be used, or multiple image enhancement algorithms can be used in combination to realize data enhancement, and the present application is not limited thereto.

[0051] Specifically, the real scene data includes a plurality of base images, and the augmented data includes first augmented data, second augmented data, third augmented data, fourth augmented data and fifth augmented data. The first augmented data is obtained by cutting, scaling and region rearranging four randomly selected base images and then splicing them through a Mosaic algorithm. The second augmented data is obtained by performing at least one of random rotation, translation, scaling and skewing on a base image through a Random Affine affine transformation algorithm. The translation amount of the translation operation in the horizontal direction and the vertical direction is not more than 50% of the size of the base image, the scaling ratio of the scaling operation is between 0.5 times and 2 times, and the skewing angle of the skewing operation is within ± 30 degrees. The third augmented data is obtained by adjusting the transparency ratio of two base images and performing pixel-level superposition through a MixUp image fusion algorithm. The fourth augmented data is obtained by performing at least one of hue adjustment, saturation adjustment and transparency adjustment on a base image through a color adjustment algorithm. The fifth augmented data is obtained by randomly performing up-down flipping and left-right flipping of a base image through a double-axis mirror transformation algorithm. The probability of up-down flipping and left-right flipping is 50%.

[0052] In the embodiment of the present application, the comprehensive data set is constructed based on multi-source image data, and the model is trained through the comprehensive data set, which can significantly improve the adaptability of the target detection training model to different scenes and changes and enhance the generalization ability of the model.

[0053] In the embodiment of the present application, the real scene data is artificially labeled based on the minimum bounding box principle. The label frame generated based on the minimum bounding box principle is the smallest rectangle or cuboid that can completely enclose a group of points or objects. In the labeling process, a label frame strictly adhering to the contour of the target object is generated to ensure the adhesion degree of the label frame to the contour of the target object. Samples with insufficient resolution resulting in blurring, samples with low contrast due to lens defocus, and samples with an occlusion degree exceeding a preset occlusion threshold are marked as unreliable samples. Through the three control mechanisms of label frame geometric constraint, image quality constraint and occlusion degree constraint, the pollution of low-quality labeled samples to the training data set is effectively eliminated, and the data quality of the training data set is improved.

[0054] In the embodiment of the present application, the multi-source image data further includes synthetic data. When the cost of finding or building a scene is high, or it is difficult to collect related data in the scene, a simulated scene can be constructed through a three-dimensional modeling software to realize data collection in the simulated scene.

[0055] Figure 2 The flowchart for generating synthetic data in an abnormal target detection method provided by the embodiment of the present application is shown in Figure 2 The method comprises the following steps.

[0056] Step 1011, constructing a simulation scene with the target object by a three-dimensional modeling software.

[0057] In the embodiment of the present application, the target object is an abnormal target to be detected by the target detection training model. The basic geometric body (such as a cube, a sphere, a cylinder, etc.) is taken as an initial modeling unit, and the point-line-surface relationship of the model is dynamically modified through vertex editing, edge subdivision and surface adjustment operations to ensure that the geometric structure of the model meets the requirements of Euclidean geometry principle and surface continuity. In the modeling process, the non-manifold structure, triangular face degeneration and grid density of high curvature area are detected and corrected, so as to generate a model meeting the physical rendering standard of good topological structure, thereby constructing a simulation scene.

[0058] Further, the material is loaded through the material library calling function, the normal map, roughness map and metalness map are combined to realize the simulation of optical properties. At the same time, the multi-channel high dynamic range imaging (High Dynamic Range Imaging, abbreviated as HDR) map ambient occlusion (Ambient Occlusion, abbreviated as AO) mapping is configured to enhance the global lighting effect of the scene. In the scene data generation stage, the camera parameters are adjusted to dynamically adjust the perspective focal length, depth of field and composition parameters, and the meteorological simulation engine is combined to render the lighting data under different weather conditions in real time, including fog effect, raindrop refraction, snow reflection and day-night light intensity change. The camera is set around the orbit curve of the target object. The OpenImage Denoise noise reduction technology (sampling number 256+) of the Cycles rendering engine is used to improve the image quality, and finally the synthesis scene pictures containing the top view, side view and oblique view are generated.

[0059] Step 1012, orthogonally projecting the target object to generate a reference contour of the target object.

[0060] In this step, the target object is projected at multiple angles through multi-angle lighting simulation based on the HDR environment map, and an image containing complete texture features is generated through ray tracing rendering. The orthogonally projects the target object to generate a reference contour containing only the geometric contour of the target object. The HDR environment map can simulate the lighting conditions in the real world, making the scene more realistic.

[0061] Step 1013, judging whether there is a foreground object to block the target object, if it is judged that there is no foreground object to block the target object, step 1014 is executed; if it is judged that there is a foreground object to block the target object, step 1015 is executed.

[0062] Step 1014, determining the reference contour as the target contour.

[0063] Step 1015, generating a composite contour containing the foreground object and the target object, and determining the target contour from the composite contour.

[0064] In the embodiment of the present application, the reference contour and the composite contour are subjected to binaryzation processing to generate a single-channel grayscale image. The pixel value is strictly limited to 0 (background) or 1 (foreground) to eliminate the interference of light noise. For some images that may be interfered by noise, smoothing processing is performed first. For example, median filtering or other methods are used to remove some isolated noise points, so as to avoid false contours caused by noise or damage to the continuity of the contours, and make the subsequent contour extraction more accurate.

[0065] Specifically, the target contour is determined from the composite contour, including: extracting a contour set from the composite contour by a neighborhood boundary tracing algorithm, the contour set containing a plurality of original contours; selecting a candidate contour from the plurality of original contours based on a preset area interval, the end point value of the area interval being set based on the area of the reference contour; determining the target contour from the candidate contour based on the Hu moment of each candidate contour and the Hu moment of the reference contour, the target contour being the contour with the highest similarity to the reference contour. The neighborhood boundary tracing algorithm is an eight-neighborhood boundary tracing algorithm, which extracts the contour set from the composite contour based on the eight-neighborhood relationship of the pixel points in the image.

[0066] For example, the upper limit value of the area interval is the area of the reference contour, and the lower limit value of the area interval is 10% of the area of the reference contour. At this time, the original contour with an area less than 10% of the area of the reference contour or an area greater than the area of the reference contour is filtered; and the original contour with an area greater than 10% of the area of the reference contour and an area less than the area of the reference contour is taken as the candidate contour.

[0067] Specifically, the similarity comparison between the candidate contour and the reference contour is performed based on the Hu moment. The geometric moments of the contours are normalized and combined to form seven characteristic values, i.e. the Hu moment. The Hu moment has translation, rotation and scaling invariance. The similarity comparison between the contours is performed by comparing the similarity characteristic values between the contours to determine the target contour. For example, the similarity characteristic value is the inverse of the logarithmic transformed value of the Hu moment. At this time, the greater the similarity characteristic value, the smaller the similarity between the contours; the smaller the similarity characteristic value, the greater the similarity between the contours.

[0068] Figure 3 A schematic diagram of determining the target contour from the composite contour in an abnormal target detection method provided by the embodiment of the present application is shown in Figure 3As shown, the target object includes two circular pieces, and a cylinder between the two circular pieces. The target object is orthogonally projected to obtain a reference contour. Above and below the target object, there are foreground objects that occlude the target object, at this time, a composite contour containing the foreground objects and the target object is generated. A contour set is extracted from the composite contour, and the contour set includes three contours, contour 1, contour 2 and contour 3. The similarity between the contours is compared by comparing the similarity characteristic values between the contours. The similarity characteristic value of contour 1 is 0.378, the similarity characteristic value of contour 2 is 0.298, and the similarity characteristic value of contour 3 is 0.494. The similarity characteristic value of contour 2 is the smallest, and the similarity with the reference contour is the largest. Therefore, contour 2 is determined as the target contour.

[0069] Step 1016, labeling the target contour to obtain synthetic data.

[0070] In the embodiment of the present application, by labeling the target object with occlusion relationship, rich occlusion scene data and accurate labeled box are provided, so that the target detection training model can cope with complex situations with occlusion relationship, and the adaptability of the model to unknown scenes is improved.

[0071] In the embodiment of the present application, the simulation scene is constructed by modeling software, a large amount of standardized and diversified synthetic data can be generated in a short time, the training data of the model is more abundant, and the ability of the model to cope with unknown scenes is further improved.

[0072] Step 102, training the target detection pre-training model by using the comprehensive data set to obtain a target detection training model.

[0073] In this step, the target detection pre-training model is constructed based on the YOLOv11-m algorithm, and the target detection pre-training model includes a backbone network, a neck network and a head network. The backbone network is responsible for efficiently extracting the basic features of the input image. Through multi-level convolution operation, feature maps of different scales are extracted to capture the semantic information of the input image. Local detail information such as edges and textures is extracted through a shallow convolution layer, and high-resolution spatial structures are retained. Low-level features are gradually aggregated through stacked convolution layers such as C3K2 modules to form more abstract local patterns. Through deep network such as SPPF module, multi-scale information is fused to extract semantic-level features such as object categories and context relationships, which are used for final classification or detection tasks. The neck network connects the shallow features (high resolution) and the deep features (low resolution) through a feature pyramid network or a path aggregation network, realizes the combination of deep semantic information and shallow detail information, and enhances the small target detection capability. The head network generates target detection results based on the fused features input by the neck network.

[0074] Figure 4 A flowchart of a training method of a target detection training model in an abnormal target detection method provided by an embodiment of the present application is shown in FIG. 12, which includes the following steps. Figure 4

[0075] In step 1021, the comprehensive data set is divided into a first training subset, a validation subset and a second training subset.

[0076] In this step, the first training subset includes 60% of the real scene data, all the augmented data and all the synthetic data; the validation subset includes 10% of the real scene data; and the second training subset includes 30% of the real scene data. The above data allocation ratio is only an example, and other ratios can be used for data allocation of the comprehensive data set in actual application, which is not limited in the embodiment of the present application.

[0077] In the example of the present application, before step 1021, it further includes: pre-processing the multi-source image data in the comprehensive data set. The pre-processing process is: after scaling the multi-source image data to a target size, normalizing the pixel value. The multi-source image data includes real scene data, augmented data and synthetic data.

[0078] Specifically, according to the size of the multi-source image data and the target size, the horizontal scaling ratio and the vertical scaling ratio are calculated; the target scaling ratio is determined from the horizontal scaling ratio and the vertical scaling ratio, and the multi-source image data is scaled according to the target scaling ratio; according to the difference between the size of the scaled image and the target size, the two short sides of the scaled image are padded to make the size of the padded image the same as the target size. For example, the size of the multi-source image data is 320*160, and the target size is 640*640. At this time, the horizontal scaling ratio is 2, and the vertical scaling ratio is 4. The multi-source image data is scaled by 2 to obtain a scaled image (size 640*320). The short side of the scaled image is padded, i.e. the two sides of 320 are padded (usually black or gray padding), to obtain a padded image (size 640*640).

[0079] In the embodiment of the present application, by uniformly scaling the size of the multi-source image data, the image size meets the model requirements while maintaining the original aspect ratio, avoiding deformation caused by direct scaling. By normalizing the pixel value, the training stability and convergence speed of the target detection training model are improved.

[0080] In step 1022, the target detection pre-training model is trained by the first training subset to obtain a target detection intermediate model.

[0081] ​In this step, the model is initialized by the backbone network weight pre-trained based on the MSCOCO dataset. There are a large amount of data in the MSCOCO dataset, and therefore the pre-trained weight has a certain generalization ability. When using the pre-trained weight as the initial weight of the model, the model is more likely to learn new features, and can find the correct parameter direction more quickly, thereby accelerating the model training process, shortening the model training time, and improving the efficiency of research or application.

[0082] In step 1023, the performance indicators of the target detection intermediate model are evaluated by verifying the subset, and the hyperparameters are adjusted based on the performance indicators.

[0083] In this step, the performance indicators include accuracy, precision, recall, and F1 score. In terms of hyperparameters, the Stochastic Gradient Descent (SGD) optimizer (momentum 0.937, weight decay 0.0005) is used, the initial learning rate is set to 0.01 and combined with the Cosine decay scheduler, and 3 iteration cycles of linear warm-up are used to stabilize the training process. The batch size is set to 32, the input resolution is set to 640x640, and the training rounds are set to 200. The model convergence is optimized by the early stopping mechanism and the loss function weight distribution (Box loss 7.5, classification loss 0.5). The early stopping mechanism refers to terminating the iteration when the model performance does not improve for a plurality of iteration cycles (e.g., 50 consecutive iteration cycles), thereby effectively preventing overfitting. In the embodiments of the present application, the above parameter settings are only examples, and other hyperparameters can be used based on the model training requirements, and the embodiments of the present application do not limit this.

[0084] In step 1024, when the performance indicators of the target detection intermediate model reach the preset threshold, the adjusted hyperparameters and the first training weight are determined.

[0085] In step 1025, the performance indicator change curve corresponding to different confidence threshold values is obtained by adjusting the confidence threshold value, and the optimal confidence threshold value is determined.

[0086] In this step, the performance indicator change curve includes the accuracy change curve, the precision change curve, the recall change curve, and the F1 score change curve. In actual applications, the performance indicator change curve can be determined based on the model performance requirements. In the embodiments of the present application, the F1 score change curve is taken as an example to describe the performance indicator change curve. By adjusting the confidence threshold value, the F1 score change curve corresponding to different confidence threshold values is obtained, and the confidence threshold value with the highest F1 score is taken as the optimal confidence threshold value. The higher the F1 score, the more robust the model.

[0087] Step 1026, training the target detection intermediate model based on the adjusted hyperparameters, the first training weight, and the optimal confidence threshold through the second training subset to obtain a target detection training model.

[0088] In this step, the second training subset is semi-supervised and automatically labeled based on the adjusted hyperparameters, the first training weight, and the optimal confidence threshold, and the labeling errors are corrected through manual review. The second training subset is used to continue training based on the first training weight, which improves the accuracy and detection capability of the target detection training model.

[0089] Step 103, inputting the live video monitoring data into the target detection training model, and outputting abnormal information when an abnormal target is detected.

[0090] In this step, the abnormal information includes target frame coordinates, warning types, abnormal target types, confidence, and timestamps. When the target detection training model detects an abnormal target, it generates a target frame to locate the abnormal target; the target frame is superimposed on the live video monitoring data to visually warn the abnormal target; and abnormal information is generated for each target frame, thereby realizing real-time visual presentation of data. Optionally, after outputting the abnormal information, the live video monitoring data integrated with the abnormal information is written to a local storage medium to realize persistent archiving of data.

[0091] In the embodiment of the present application, based on the analysis of accident experience and the characteristics of gas space scenes, a plurality of abnormal target types are systematically constructed. The abnormal target types include large engineering equipment abnormalities (such as bulldozers, excavators, road rollers, earthmoving vehicles, etc.), construction tools and sign abnormalities (such as construction fences, water horse isolation devices, construction warning signs, spades, roadblocks, engineering protective fences, and pickaxes), personnel and safety protection tool abnormalities (such as safety helmets, reflective vests, and safety belts), gas valve well abnormalities (such as manhole covers and well eyes), and pressure regulating station and box body surrounding abnormalities. Among them, the abnormal detection of the gas valve well includes the integrity detection of the manhole cover (such as whether the manhole cover is collapsed, damaged, or missing), the position feature detection of the well eye, and the surrounding environment detection of the manhole cover (such as whether there are accumulations around the manhole cover). The abnormal detection of the pressure regulating station and the box body surrounding includes the integrity detection of the cathodic protection device, the visibility detection of the safety warning sign, and the integrity detection of the fence structure. In the embodiment of the present application, the abnormal target types cover various hazards in the gas operation scene, and through the target detection training model, multiple types of abnormal targets can be recognized at the same time, realizing comprehensive and efficient detection of abnormal targets.

[0092] In the embodiment of the present application, the early warning types include gas third-party construction early warning, incorrect wearing of protective equipment early warning, gas valve well early warning and general early warning. Among them, if the target detection training model detects large engineering machinery abnormalities and construction tool and sign abnormalities, the gas third-party construction early warning is generated. On the basis of the gas third-party construction early warning, if a pedestrian target is detected, a pedestrian target frame is generated; when no safety helmet, safety vest and other safety protection tools are identified in the pedestrian target frame, the incorrect wearing of protective equipment early warning is generated. If a gas valve well anomaly is detected, a gas valve well anomaly alarm is generated; if a pressure regulating station and box body surrounding anomaly is detected, a general early warning is generated.

[0093] Specifically, when detecting a pedestrian target, the pixel number of the pedestrian target frame is detected, and the pedestrian target frame with a pixel number less than a preset pixel threshold and the pedestrian target frame with an occlusion degree greater than a preset occlusion threshold are removed. For example, the pixel threshold is 30000. When the pixel number is less than the pixel threshold, it indicates that the pixel number of the pedestrian target frame is small, and the feature information that can be reflected is limited, and the model is difficult to extract sufficient details, affecting the accuracy of target detection.

[0094] In the technical scheme provided by the embodiment of the present application, the edge detection device with the target detection training model is deployed to intelligently detect abnormal targets, without the need for inspection personnel to arrive at the gas operation site for inspection, so that real-time monitoring of the operation site can be realized, the real-time performance is strong, and the human cost is saved, the accident risk in the inspection work is reduced, and the abnormal target detection efficiency is improved.

[0095] In the embodiment of the present application, the comprehensive data set constructed by the multi-source image data is used as training data for model training, which significantly improves the generalization ability of the model in complex and variable scenes, so that the target detection model can be applied to various scenes, and the practicality and reliability are improved.

[0096] Figure 5 The structure diagram of an abnormal target detection device provided by the embodiment of the present application is shown in Figure 5 As shown in the figure, the abnormal target detection device includes a construction module 11, a training module 12 and a detection module 13. The construction module 11 is used to construct a comprehensive data set based on multi-source image data; the training module 12 is used to train a target detection pre-training model through the comprehensive data set to obtain a target detection training model; and the detection module 13 is used to input the on-site video monitoring data into the target detection training model and output abnormal information when an abnormal target is detected.

[0097] In the embodiment of the present application, the multi-source image data includes real scene data and enhanced data, and the abnormal target detection device further includes an enhancement module 14, which is used to perform data enhancement on the real scene data through an image enhancement algorithm to obtain enhanced data.

[0098] In the embodiment of the present application, the image enhancement algorithm comprises at least one of a Mosaic algorithm, a Random Affine affine transformation algorithm, a MixUp image fusion algorithm, a color adjustment algorithm, and a two-axis mirror transformation algorithm.

[0099] In the embodiment of the present application, the abnormal target detection device further comprises a modeling module 15, a projection module 16, a judgment module 17, a first determination module 18, a second determination module 19, and a labeling module 20. The modeling module 15 is configured to construct a simulation scene with a target object by using a three-dimensional modeling software. The projection module 16 is configured to perform an orthogonal projection on the target object to generate a reference contour of the target object. The judgment module 17 is configured to determine whether there is a foreground object that causes an occlusion to the target object. If the judgment module 17 determines that there is no foreground object that causes an occlusion to the target object, the first determination module 18 is triggered to perform a step of determining the reference contour as a target contour. If the judgment module 17 determines that there is a foreground object that causes an occlusion to the target object, the second determination module 19 is triggered to perform a step of generating a composite contour containing the foreground object and the target object, and determining a target contour from the composite contour. The labeling module 20 is configured to label the target contour to obtain synthetic data.

[0100] In the embodiment of the present application, the second determination module 19 is specifically configured to extract a contour set from the composite contour by using a neighborhood boundary tracing algorithm, the contour set containing a plurality of original contours; select a candidate contour from the plurality of original contours based on a preset area interval, an endpoint value of the area interval being set based on an area of the reference contour; and determine a target contour from the candidate contour based on a Hu moment of each candidate contour and a Hu moment of the reference contour, the target contour being a contour with the highest similarity to the reference contour.

[0101] In the embodiment of the present application, the training module 12 is specifically configured to divide the comprehensive data set into a first training subset, a validation subset, and a second training subset; train the target detection pre-training model through the first training subset to obtain a target detection intermediate model; evaluate a performance index of the target detection intermediate model through the validation subset, and adjust hyperparameters based on the performance index; determine adjusted hyperparameters and a first training weight when the performance index of the target detection intermediate model reaches a preset threshold; obtain a performance index change curve corresponding to different confidence threshold values by adjusting the confidence threshold value, and determine an optimal confidence threshold value; and train the target detection intermediate model based on the adjusted hyperparameters, the first training weight, and the optimal confidence threshold value through the second training subset to obtain a target detection training model.

[0102] The edge detection device with the target detection training model deployed intelligently detects abnormal targets, real-time monitoring of the operation site can be realized without the inspection personnel arriving at the gas operation site for inspection, real-time performance is strong, and manpower cost is saved, accident risks in the inspection work are reduced, and abnormal target detection efficiency is improved.

[0103] In the embodiment of the present application, the comprehensive data set constructed by the multi-source image data is used as training data for model training, which significantly improves the generalization ability of the model in complex and variable scenes, enables the target detection model to be applied to various scenes, and improves the practicability and reliability.

[0104] The embodiment of the present application provides an edge computing device, and the edge computing device is deployed with a target detection training model.

[0105] The edge detection device with the target detection training model deployed intelligently detects abnormal targets, real-time monitoring of the operation site can be realized without the inspection personnel arriving at the gas operation site for inspection, real-time performance is strong, and manpower cost is saved, accident risks in the inspection work are reduced, and abnormal target detection efficiency is improved.

[0106] The embodiment of the present application provides a computer readable storage medium, which comprises a stored program, wherein when the program runs, the computer readable storage medium controls the device where the computer readable storage medium is located to execute each step of the embodiment of the abnormal target detection method described above, and the specific description can be referred to the embodiment of the abnormal target detection method described above.

[0107] The above is only a preferred specific embodiment of the present application, but the protection scope of the present application is not limited to this, any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An abnormal target detection method, characterized in that, The method includes: Construct a comprehensive dataset based on multi-source image data; The object detection pre-training model is trained using the comprehensive dataset to obtain the object detection training model; The on-site video surveillance data is input into the target detection training model, and abnormal information is output when an abnormal target is detected.

2. The method according to claim 1, characterized in that, The multi-source image data includes real-scene data and augmented data; Before constructing the comprehensive dataset based on multi-source image data, the following steps are also included: The real-world scene data is augmented using an image enhancement algorithm to obtain enhanced data.

3. The method according to claim 2, characterized in that, The image enhancement algorithm includes at least one of the following: Mosaic algorithm, Random Affine affine transformation algorithm, MixUp image fusion algorithm, color adjustment algorithm, and dual-axis mirror transformation algorithm.

4. The method according to claim 1, characterized in that, The multi-source image data includes synthetic data; before constructing the comprehensive dataset based on the multi-source image data, the following steps are also included: Using 3D modeling software, a simulated scene with the target object is constructed; The target object is orthogonally projected to generate the reference contour of the target object; Determine whether a foreground object is obstructing the target object; If it is determined that no foreground object occludes the target object, then the reference contour is determined as the target contour. If it is determined that a foreground object is occluding the target object, a composite contour containing the foreground object and the target object is generated, and the target contour is determined from the composite contour. The target contour is labeled to obtain synthetic data.

5. The method according to claim 4, characterized in that, Determining the target contour from the composite contour includes: A contour set is extracted from the composite contour using a neighborhood boundary tracing algorithm. The contour set contains multiple original contours. Candidate contours are selected from the plurality of original contours based on a preset area range, wherein the endpoint values ​​of the area range are set based on the area of ​​the reference contour. Based on the Hu moments of each candidate contour and the Hu moments of the reference contour, a target contour is determined from the candidate contours, and the target contour is the contour with the highest similarity to the reference contour.

6. The method according to claim 1, characterized in that, The step of training the object detection pre-training model using the comprehensive dataset to obtain the object detection training model includes: The comprehensive dataset is divided into a first training subset, a validation subset, and a second training subset; The object detection pre-trained model is trained using the first training subset to obtain an intermediate object detection model; The performance metrics of the intermediate object detection model are evaluated by validating a subset, and hyperparameters are adjusted based on the performance metrics. When the performance metrics of the intermediate object detection model reach a preset threshold, the adjusted hyperparameters and the first training weights are determined. By adjusting the confidence threshold, the performance index change curves corresponding to different confidence thresholds are obtained, and the optimal confidence threshold is determined. The target detection intermediate model is trained using the second training subset based on the adjusted hyperparameters, the first training weights, and the optimal confidence threshold to obtain the target detection training model.

7. The method according to claim 6, characterized in that, The performance metrics include accuracy, precision, recall, and F1 score.

8. An abnormal target detection device, characterized in that, The device includes: The building module is used to construct a comprehensive dataset based on multi-source image data; The training module is used to train the object detection pre-trained model using the comprehensive dataset to obtain the object detection training model; The detection module is used to input on-site video surveillance data into the target detection training model and output abnormal information when an abnormal target is detected.

9. An edge computing device, characterized in that, The edge computing device is equipped with the object detection training model as described in claim 1.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform the abnormal target detection method as described in any one of claims 1-7.

Images