Feature enhancement and multi-scale feature-based ponding condition identification method, electronic equipment and medium

Abstract

The invention discloses a feature enhancement and multi-scale feature-based ponding condition identification method, electronic equipment and a medium. The method comprises the following steps: obtaining a target image; a multi-scale feature fusion network model is established, a multi-scale feature map is obtained according to the target image, and the multi-scale feature fusion network model comprises four feature extraction branches; performing object classification according to the multi-scale feature map to obtain an original image; performing data enhancement on the original image to obtain enhanced data; and according to the enhanced data, identifying the ponding condition of the target image based on the edge end and the server end. According to the invention, the data is trained through a network structure, and the architecture that the edge end and the server end are combined to carry out primary identification and secondary identification is introduced, so that the accuracy and practicability of ponding grade identification are improved.

Description

Technical Field

[0001] The present invention relates to the field of image data processing technology and deep learning, and more specifically, to a method, electronic device and medium for identifying water accumulation based on feature enhancement and multi-scale features. Background Art

[0002] With the acceleration of urbanization, the problem of waterlogging caused by heavy rain has become an important challenge faced by many cities. Timely and accurate assessment of the extent of urban waterlogging is crucial for disaster prevention and mitigation. In recent years, the rapid development of computer vision and deep learning technologies has promoted the innovation of urban waterlogging monitoring technology. Traditional monitoring methods mainly rely on the layout of water level meters and sensors, which are expensive to deploy and difficult to cover wide areas. In order to solve these problems, researchers have begun to explore waterlogging detection methods based on computer vision and deep learning algorithms. For example, by utilizing surveillance videos of urban roads combined with deep learning models, the depth and distribution of waterlogging can be automatically identified from video images in real time, providing a more efficient and low-cost solution.

[0003] Existing studies have shown that waterlogging detection methods based on computer vision have significant advantages. For example, two recent studies used YOLOv3 and YOLOv4 deep learning models to identify water depth and achieved good results. However, they still have certain accuracy issues when identifying waterlogging levels and lack practicality.

[0004] Therefore, it is necessary to develop a method, electronic equipment and medium for identifying water accumulation based on feature enhancement and multi-scale features.

[0005] The information disclosed in the background technology section of the present invention is only intended to deepen the understanding of the general background technology of the present invention, and should not be regarded as acknowledging or suggesting in any form that the information constitutes the prior art already known to those skilled in the art. Summary of the invention

[0006] The present invention proposes a method, electronic device and medium for identifying waterlogging based on feature enhancement and multi-scale features, which can train data through a network structure and introduce an architecture for joint primary and secondary identification by the edge and server ends, thereby improving the accuracy and practicality of waterlogging level identification.

[0007] In a first aspect, an embodiment of the present disclosure provides a method for identifying waterlogging based on feature enhancement and multi-scale features, including:

[0008] Obtaining a target image;

[0009] Establishing a multi-scale feature fusion network model, and obtaining a multi-scale feature map according to the target image, wherein the multi-scale feature fusion network model includes four feature extraction branches;

[0010] Classify objects according to the multi-scale feature map to obtain an original image;

[0011] Performing data enhancement on the original image to obtain enhanced data;

[0012] According to the enhanced data, water accumulation of the target image is identified based on the edge end and the server end.

[0013] Preferably, the convolution operations of the four feature extraction branches are:

[0014]

[0015] Among them, F i represents the fusion feature map of the input detection head, O i represents the output of the i-th branch.

[0016] Preferably, classifying objects according to the multi-scale feature map includes:

[0017] Generate a priori frame according to the multi-scale feature map;

[0018] Locate the bounding box of the object by learning and predicting the offset of each prior box;

[0019] The classifier calculates the probability of different categories of each prior box to achieve object classification.

[0020] Preferably, the data enhancement includes piecewise linear transformation and histogram equalization.

[0021] Preferably, the piecewise linear transformation is:

[0022]

[0023] Where r represents the pixel value of the original image, s represents the pixel value of the transformed image, and a, b, c, d, e, and f are segmented stretching parameters.

[0024] Preferably, the histogram equalization comprises:

[0025] The histogram of the original image is calculated, the cumulative distribution function of the original image is calculated by histogram accumulation, and then a new grayscale value is obtained by interpolation calculation.

[0026] Preferably, according to the enhanced data, identifying the water accumulation condition of the target image based on the edge end and the server end includes:

[0027] The edge end is identified based on the enhanced data. If the edge end identifies water accumulation, a secondary identification is performed through the service end to identify the water accumulation level and obtain the water accumulation situation.

[0028] Preferably, if water accumulation is identified at the edge end, no secondary identification is performed.

[0029] In a second aspect, an embodiment of the present disclosure further provides an electronic device, the electronic device comprising:

[0030] A memory storing executable instructions;

[0031] A processor runs the executable instructions in the memory to implement the method for identifying water accumulation based on feature enhancement and multi-scale features.

[0032] In a third aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the method for identifying water accumulation based on feature enhancement and multi-scale features is implemented.

[0033] The methods and apparatus of the present invention have other features and advantages that will be apparent from, or will be described in detail in, the accompanying drawings and subsequent detailed descriptions incorporated herein, which together serve to explain the specific principles of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The above and other objects, features and advantages of the present invention will become more apparent through a more detailed description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, wherein like reference numerals generally represent like components throughout the exemplary embodiments of the present invention.

[0035] Figure 1 A flowchart showing the steps of a method for identifying waterlogging based on feature enhancement and multi-scale features according to an embodiment of the present invention is shown.

[0036] Figure 2 A schematic diagram of a multi-scale feature fusion network structure according to an embodiment of the present invention is shown.

[0037] Figure 3 A schematic diagram showing a primary edge recognition result in which no water accumulation is found according to an embodiment of the present invention.

[0038] Figure 4 A schematic diagram showing a secondary service-side identification result of finding water accumulation according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0039] The preferred embodiments of the present invention will be described in more detail below. Although the preferred embodiments of the present invention are described below, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0040] To facilitate understanding of the solutions and effects of the embodiments of the present invention, three specific application examples are given below. Those skilled in the art should understand that the examples are only for facilitating understanding of the present invention, and any specific details thereof are not intended to limit the present invention in any way.

[0041] Example 1

[0042] Figure 1 A flowchart showing the steps of a method for identifying waterlogging based on feature enhancement and multi-scale features according to an embodiment of the present invention is shown.

[0043] like Figure 1 As shown, the waterlogging recognition method based on feature enhancement and multi-scale features includes:

[0044] Step 101, obtaining a target image;

[0045] Step 102, establishing a multi-scale feature fusion network model, and obtaining a multi-scale feature map according to the target image, wherein the multi-scale feature fusion network model includes four feature extraction branches;

[0046] Step 103, classifying objects according to the multi-scale feature map to obtain an original image;

[0047] Step 104, performing data enhancement on the original image to obtain enhanced data;

[0048] Step 105 , based on the enhanced data, identifying the water accumulation condition of the target image based on the edge end and the server end.

[0049] In one example, the convolution operations of the four feature extraction branches are:

[0050]

[0051] Among them, F i represents the fusion feature map of the input detection head, O i represents the output of the i-th branch.

[0052] In one example, classifying an object according to a multi-scale feature map includes:

[0053] Generate a priori boxes based on multi-scale feature maps;

[0054] Locate the bounding box of the object by learning and predicting the offset of each prior box;

[0055] The classifier calculates the probability of different categories of each prior box to achieve object classification.

[0056] In one example, data augmentation includes piecewise linear transformation and histogram equalization.

[0057] In one example, the piecewise linear transformation is:

[0058]

[0059] Where r represents the pixel value of the original image, s represents the pixel value of the transformed image, and a, b, c, d, e, and f are segmented stretching parameters.

[0060] In one example, histogram equalization includes:

[0061] The histogram of the original image is calculated, the cumulative distribution function of the original image is calculated by histogram accumulation, and then a new grayscale value is obtained by interpolation calculation.

[0062] In one example, based on the enhanced data, the water accumulation condition of the target image identified by the edge end and the server end includes:

[0063] Edge recognition is performed based on enhanced data. If water accumulation is detected at the edge, secondary recognition is performed through the server to identify the level of water accumulation and obtain the water accumulation situation.

[0064] In one example, if water accumulation is detected at the edge, no secondary identification is performed.

[0065] Figure 2 A schematic diagram of a multi-scale feature fusion network structure according to an embodiment of the present invention is shown.

[0066] Specifically, a multi-scale feature fusion network structure is established, such as Figure 2 As shown in the figure, considering that different sizes of objects to be identified may require features of different scales for effective identification, multiple parallel branches are used for feature extraction, and each branch is responsible for processing feature information of different scales. This design enables the network to have stronger adaptability when processing objects of different sizes.

[0067] The multi-scale feature fusion network structure contains four feature extraction branches, corresponding to four different scale information: the first branch directly aggregates the input features and the results sampled from the second branch; the second branch aggregates the output features and the output of the third branch after the convolution operation; the third branch also aggregates the output features and the output of the fourth branch after the convolution operation; the fourth branch obtains the output after the convolution operation, so that four multi-scale features can be obtained. The specific convolution operation is shown in the following formula:

[0068]

[0069] Among them, F i represents the fusion feature map of the input detection head, O i Represents the output of the i-th branch.

[0070] The model generates a series of prior boxes based on the scales extracted from the multi-scale feature map. These prior boxes cover different positions and scales in the input image and provide detection anchors for objects of different sizes and shapes. Next, the bounding box regression adjusts the position of each prior box by learning and predicting its offset, thereby accurately locating the bounding box of the object. Finally, the category prediction uses a classifier to calculate the probability of different categories for each prior box, classify the object, and obtain the original image. By utilizing feature maps at different levels and predefined prior boxes, the detection module is able to detect objects at different scales.

[0071] Data enhancement is performed on the original image to obtain enhanced data. Data enhancement includes piecewise linear transformation and histogram equalization.

[0072] The piecewise linear transformation divides the grayscale value of the image into multiple intervals and applies different linear transformations to each interval, thereby enhancing the contrast of different parts of the image. This method can highlight the grayscale range of interest (such as details in a specific brightness interval) according to actual needs, while suppressing grayscale areas of no interest. The key is to allow personalized adjustments to different grayscale intervals of the image, and each interval uses a different linear function for grayscale value mapping. In this way, the details of a specific grayscale interval can be amplified or enhanced, while other areas that do not need to be paid attention to can remain unchanged or compressed. This method is particularly suitable for applications that need to highlight certain details of the image (such as a specific brightness or contrast range), and can effectively improve the visual effect of the image.

[0073] By flexibly setting the transformation parameters of each interval, the piecewise linear transformation can optimize the image processing effect according to specific needs, especially when it is necessary to emphasize specific areas without interfering with other parts. The general formula of the piecewise linear transformation is as follows:

[0074]

[0075] Where r represents the pixel value of the original image, s represents the pixel value of the transformed image, and a, b, c, d, e, and f are segmented stretching parameters.

[0076] Histogram equalization adjusts the grayscale distribution of an image to make the histogram of the image more uniform, thereby expanding the dynamic range of grayscale values ​​and enhancing the overall contrast of the image. Specifically, histogram equalization is a nonlinear stretching method that stretches the grayscale that accounts for a large proportion of the image, while compressing the grayscale that accounts for a small proportion and redistributing the image pixel values. Its essence is to analyze the histogram of the image and transform the grayscale of the image, thereby improving the visual effect of the image, especially when the grayscale distribution is uneven, it can significantly improve the visibility of details.

[0077] The basic steps to achieve histogram equalization through the histogram of the original image are as follows:

[0078] (1) Calculate the histogram of the original image;

[0079] (2) Calculate the cumulative distribution function (CDF) of the original image by histogram accumulation;

[0080] (3) Based on the cumulative distribution function, the new gray value is obtained by interpolation calculation:

[0081]

[0082] In the formula, r k and k Respectively represent the corresponding pixels of each gray level k of the original image src and the new image dst, p r Represents the probability density function of the gray value r, n j Represents the number of pixels with gray value j, L is the gray level, and N is the total number of pixels.

[0083] Other data enhancement methods used in the present invention are shown in Table 1.

[0084] Table 1

[0085]

[0086] Figure 3 A schematic diagram showing a primary edge recognition result in which no water accumulation is found according to an embodiment of the present invention.

[0087] Figure 4 A schematic diagram showing a secondary service-side identification result of finding water accumulation according to an embodiment of the present invention is shown.

[0088] The secondary recognition process combines edge recognition and server recognition. The front-end interface is built using PySide6 and connected to the back-end deep learning model. Figure 3As shown in the figure, when the model receives data without water accumulation, the edge will determine that there is no water accumulation and display a green prompt on the front end, marking "no water accumulation". At this time, there is no need for server-side recognition, thus achieving efficient secondary recognition. If the uploaded data contains water accumulation, the edge will first identify the water accumulation, but the specific level of the water accumulation has not yet been determined. At this time, the image will be transmitted to the server for further fine recognition. The server-side model will deeply analyze the image and identify the contrast objects in the image, such as vehicles and wheels. These objects, as reference objects, help to more accurately determine the depth of the water accumulation. The server conducts a comprehensive analysis based on these contrast objects and selects the more serious water accumulation level as the class division. Finally, the server-side recognition result will be displayed above the image in red font, indicating the level of water accumulation. In this example, the system determines that the water accumulation level is "moderate water accumulation" and marks it in a conspicuous red font to help users quickly understand the specific situation of the water accumulation in the image.

[0089] The benefit of secondary recognition is that it can improve the efficiency and accuracy of the system. First, edge recognition can quickly filter out areas without water accumulation, avoiding unnecessary server-side calculations, thereby saving computing resources and time. Second, for images containing water accumulation, the edge only performs preliminary water accumulation recognition, while more accurate water accumulation level classification is completed by the server. Figure 4 As shown. Through this hierarchical processing, the server can use more computing resources for in-depth analysis and combine comparison objects (such as vehicles, wheels, etc.) to improve the accuracy of water level judgment. This secondary recognition method can not only improve the overall processing speed, but also ensure high accuracy of water detection, especially in complex or ambiguous scenes. Ultimately, users can obtain water level information more quickly and accurately, improving the practicality and reliability of the system.

[0090] Example 2

[0091] The present disclosure provides an electronic device, which includes: a memory storing executable instructions; a processor running the executable instructions in the memory to implement the above-mentioned method for identifying water accumulation based on feature enhancement and multi-scale features.

[0092] An electronic device according to an embodiment of the present disclosure includes a memory and a processor.

[0093] The memory is used to store non-temporary computer-readable instructions. Specifically, the memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, etc.

[0094] The processor may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions. In one embodiment of the present disclosure, the processor is used to run the computer-readable instructions stored in the memory.

[0095] Those skilled in the art should be able to understand that in order to solve the technical problem of how to obtain a good user experience, the present embodiment may also include well-known structures such as a communication bus and an interface, and these well-known structures should also be included in the protection scope of the present disclosure.

[0096] For detailed description of this embodiment, reference may be made to the corresponding descriptions in the aforementioned embodiments, which will not be repeated here.

[0097] Example 3

[0098] An embodiment of the present disclosure provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the method for identifying water accumulation based on feature enhancement and multi-scale features is implemented.

[0099] According to the computer-readable storage medium of the embodiment of the present disclosure, non-transitory computer-readable instructions are stored thereon. When the non-transitory computer-readable instructions are executed by a processor, all or part of the steps of the above-mentioned methods of each embodiment of the present disclosure are executed.

[0100] The above-mentioned computer-readable storage media include, but are not limited to: optical storage media (e.g., CD-ROM and DVD), magneto-optical storage media (e.g., MO), magnetic storage media (e.g., magnetic tape or mobile hard disk), media with built-in rewritable non-volatile memory (e.g., memory card) and media with built-in ROM (e.g., ROM box).

[0101] Those skilled in the art should understand that the purpose of the above description of the embodiments of the present invention is only to exemplarily illustrate the beneficial effects of the embodiments of the present invention, and is not intended to limit the embodiments of the present invention to any given examples.

[0102] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments.

Claims

1. A method for identifying waterlogging based on feature enhancement and multi-scale features, characterized in that: include: Obtaining a target image; Establishing a multi-scale feature fusion network model, and obtaining a multi-scale feature map according to the target image, wherein the multi-scale feature fusion network model includes four feature extraction branches; Classify objects according to the multi-scale feature map to obtain an original image; Performing data enhancement on the original image to obtain enhanced data; According to the enhanced data, water accumulation of the target image is identified based on the edge end and the server end.

2. The method for identifying waterlogging based on feature enhancement and multi-scale features according to claim 1, wherein: The convolution operations of the four feature extraction branches are: Among them, F i represents the fusion feature map of the input detection head, O i represents the output of the i-th branch.

3. The method for identifying waterlogging based on feature enhancement and multi-scale features according to claim 1, wherein: Classifying an object according to the multi-scale feature map includes: Generate a priori frame according to the multi-scale feature map; Locate the bounding box of the object by learning and predicting the offset of each prior box; The classifier calculates the probability of different categories of each prior box to achieve object classification.

4. The method for identifying waterlogging based on feature enhancement and multi-scale features according to claim 1, wherein: The data enhancement includes piecewise linear transformation and histogram equalization.

5. The method for identifying waterlogging based on feature enhancement and multi-scale features according to claim 4, wherein: The piecewise linear transformation is: Where r represents the pixel value of the original image, s represents the pixel value of the transformed image, and a, b, c, d, e, and f are segmented stretching parameters.

6. The method for identifying waterlogging based on feature enhancement and multi-scale features according to claim 4, wherein: The histogram equalization comprises: The histogram of the original image is calculated, the cumulative distribution function of the original image is calculated by histogram accumulation, and then a new grayscale value is obtained by interpolation calculation.

7. The method for identifying waterlogging based on feature enhancement and multi-scale features according to claim 1, wherein: According to the enhanced data, identifying the water accumulation condition of the target image based on the edge end and the server end includes: The edge end is identified based on the enhanced data. If the edge end identifies water accumulation, a secondary identification is performed through the service end to identify the water accumulation level and obtain the water accumulation situation.

8. The method for identifying waterlogging based on feature enhancement and multi-scale features according to claim 7, wherein: If water accumulation is identified at the edge, no secondary identification is performed.

9. An electronic device, characterized in that: The electronic device comprises: A memory storing executable instructions; A processor, wherein the processor runs the executable instructions in the memory to implement the method for identifying water accumulation based on feature enhancement and multi-scale features as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for identifying waterlogging based on feature enhancement and multi-scale features described in any one of claims 1 to 8 is implemented.

Images