Image intelligent segmentation method and system based on intelligent security

Abstract

The present invention relates to the field of image segmentation technology, and discloses an intelligent image segmentation method and system based on intelligent security, including: collecting an image to be segmented in an intelligent security area, performing image detail enhancement on the image to be segmented to obtain an enhanced image; calculating the inter-class variance of the enhanced image, performing preliminary background separation, positive and negative structural element sliding and pixel growth area on the enhanced image to obtain a target background separation image; collecting multi-scale features of the target background separation image, performing feature fusion to obtain fusion features, performing initial image segmentation on the target background separation image to obtain an initial segmented image; performing boundary optimization, region segmentation, semantic enhancement and secondary image segmentation on the segmentation boundary of the initial segmented image to obtain a secondary segmented image, and when the image segmentation level meets the preset segmentation requirements, the secondary segmented image is used as the final intelligent segmented image. The present invention can improve the accuracy of image segmentation of an intelligent security system in the case of complex scenes.

Description

Technical Field

[0001] The invention relates to an intelligent image segmentation method and system based on intelligent security, belonging to the technical field of image segmentation. Background Art

[0002] Image segmentation is a key technology in the field of intelligent security. It plays an extremely important supporting role in tasks such as target recognition, behavior analysis, and security monitoring. Building an intelligent image segmentation method based on intelligent security can help the security system accurately extract key information from the image, such as target objects such as people and vehicles, thereby greatly improving the accuracy and efficiency of security monitoring. With this intelligent segmentation method, security personnel can quickly detect abnormal situations and take corresponding countermeasures in time to effectively prevent the occurrence of safety accidents and enhance the safety and prevention capabilities of specific areas.

[0003] At present, traditional image segmentation algorithms are usually used in intelligent security image segmentation, such as threshold-based segmentation methods, edge detection algorithms, etc. However, these traditional methods have obvious limitations when facing complex and changeable security scenes. Since images in actual security scenes often have many problems such as large lighting changes, severe target occlusion, and complex backgrounds, it is difficult for traditional algorithms to adaptively adjust segmentation strategies, resulting in inaccurate segmentation of target objects when processing these images, and it is easy to cause mis-segmentation or incomplete segmentation, which cannot meet the high-precision requirements of intelligent security systems for high-precision image segmentation.

[0004] Therefore, there is an urgent need for a solution that can improve the accuracy of image segmentation of intelligent security systems in complex scenarios. Summary of the invention

[0005] The present invention provides an image intelligent segmentation method and system based on intelligent security, the main purpose of which is to improve the accuracy of security authentication while reducing the workload of path detection.

[0006] To achieve the above object, the present invention provides an image intelligent segmentation method based on intelligent security, comprising:

[0007] Collecting an image to be segmented in the intelligent security area, identifying original pixel values ​​of the image to be segmented, and enhancing image details of the image to be segmented based on the original pixel values ​​to obtain an enhanced image;

[0008] Calculating the inter-class variance of the enhanced image, using the inter-class variance to find the foreground target pixels of the enhanced image, performing preliminary background separation on the enhanced image based on the foreground target pixels to obtain a preliminary separation image, performing positive and negative structural element sliding on the preliminary separation image to obtain a refined separation image, constructing a pixel growth region of the refined separation image, and using the pixel growth region to perform precise background separation on the refined separation image to obtain a target background separation image;

[0009] Collecting image features of the target background separation image at multi-scale resolutions to obtain multi-scale features, performing feature fusion on the multi-scale features to obtain fusion features, and performing initial image segmentation on the target background separation image using the fusion features to obtain an initial segmented image;

[0010] Calculate the inter-class divergence matrix of the initial segmented image, use the inter-class divergence matrix to optimize the segmentation boundary of the initial segmented image to obtain an optimized segmented image, perform region segmentation on the optimized segmented image to obtain a region segmented image, identify image key points in the region segmented image, use the image key points to perform target recognition on the region segmented image to obtain an image target, use the fusion features to perform image semantic enhancement on the image target to obtain enhanced target information, perform secondary image segmentation on the initial segmented image based on the enhanced target information to obtain a secondary segmented image, calculate the image segmentation level of the secondary segmented image, and when the image segmentation level meets the preset segmentation requirements, use the secondary segmented image as the final intelligent segmented image.

[0011] Optionally, the performing image detail enhancement on the image to be segmented based on the original pixel values ​​to obtain an enhanced image includes:

[0012] The pixel mean of the original pixel value is calculated using the following formula:

[0013] ;

[0014] in, represents the mean pixel, Represents a pixel point of the image to be segmented, and H represents the pixel point The number of rows in the field, L represents the number of pixels The number of columns in the field, Pixels of the image to be segmented The pixel value of the image to be segmented is x, and y represents the horizontal coordinate of a pixel in the image to be segmented, and y represents the vertical coordinate of a pixel in the image to be segmented;

[0015] Based on the pixel mean, the image to be segmented is subjected to mean filtering to obtain a filtered image, and the filtered image is sharpened to obtain an enhanced image.

[0016] Optionally, the calculating the inter-class variance of the enhanced image includes:

[0017] Counting the grayscale histogram of the enhanced image;

[0018] Calculating the average grayscale value of the enhanced image based on the grayscale histogram, and calculating the background pixel probability of the enhanced image;

[0019] Based on the background class pixel probability, calculating the background class average grayscale value of the enhanced image;

[0020] Calculating the probability of foreground pixels of the enhanced image;

[0021] Based on the foreground pixel probability, calculating the foreground average grayscale value of the enhanced image;

[0022] Based on the average gray value, the average gray value of the background class and the average gray value of the foreground class, the inter-class variance of the enhanced image is calculated using the following formula:

[0023] ;

[0024] in, represents the between-class variance, Background class pixel probability, represents the probability of foreground class pixels, represents the average gray value of the background class, represents the average gray value of the foreground class, Represents the average gray value.

[0025] Optionally, performing preliminary background separation on the enhanced image based on the foreground target pixels to obtain a preliminary separated image includes:

[0026] Based on the foreground target pixels, setting a background separation size of the enhanced image;

[0027] Using the background separation size, constructing an all-zero matrix of the enhanced image;

[0028] Using the all-zero matrix, constructing a background model of the enhanced image;

[0029] The background model is used to perform preliminary background separation on the enhanced image to obtain a preliminary separated image.

[0030] Optionally, performing positive and negative structural element sliding on the preliminary separated image to obtain a refined separated image includes:

[0031] Querying the image size of the preliminary separated image;

[0032] constructing an element sliding structure of the preliminary separated image based on the image size;

[0033] The element sliding structure is used to perform image erosion on the preliminary separated image, and the element sliding structure is used to perform image dilation on the preliminary separated image to obtain a refined separated image.

[0034] Optionally, collecting image features of the target background separation image at multi-scale resolutions to obtain multi-scale features includes:

[0035] Inputting the target background separated image into a preconfigured neural network;

[0036] Using the neural network to perform horizontal pooling and vertical pooling on the target background separation image to obtain a cascade feature map;

[0037] Performing multi-layer convolution processing on the cascaded feature map to obtain a multi-layer convolution feature map;

[0038] Performing regularization processing on the multi-layer convolution feature map to obtain a multi-layer feature map;

[0039] The image features of the multi-layer feature map are extracted to obtain multi-scale features.

[0040] Optionally, performing initial image segmentation on the target background separated image by using the fusion feature to obtain an initial segmented image includes:

[0041] Converting the target background separated image into a grayscale image;

[0042] identifying separation key points of the grayscale image using the fusion features;

[0043] Performing non-maximum suppression on the grayscale image with the separation key point as the center to obtain a suppressed image;

[0044] An image edge of the suppressed image is detected, and the suppressed image is segmented based on the image edge to obtain an initial segmented image.

[0045] Optionally, the using the fusion feature to perform image semantic enhancement on the image target to obtain enhanced target information includes:

[0046] Normalizing the fused features to obtain normalized features;

[0047] Constructing feature semantics of each sub-feature in the normalized feature to obtain a semantic feature;

[0048] Extracting image features of the image target and converting the image features into a feature matrix;

[0049] Mapping the semantic features into the feature matrix to perform association matching on the semantic features and the image features to obtain matching features;

[0050] Performing semantic knowledge supplementation on the matching features to obtain semantic knowledge supplementation features;

[0051] After the semantic knowledge supplementary features are used as output information, target information is generated for the semantic knowledge supplementary features to obtain enhanced target information.

[0052] Optionally, the calculating the image segmentation level of the secondary segmented image includes: calculating the mean intersection and union ratio of the secondary segmented image by using the following method:

[0053] ;

[0054] in, represents the even intersection and ratio, It represents the number of pixels in the secondary segmentation image whose true value is class a but is predicted to be class b. Indicates the number of categories of the secondary segmented image, represents the number of pixels predicted correctly in the secondary segmentation image, represents the number of pixels predicted incorrectly in the secondary segmentation image,

[0055] The segmentation level of the secondary segmented image is evaluated according to the mean intersection union ratio.

[0056] In order to solve the above problems, the present invention also provides an image intelligent segmentation system based on intelligent security, the system comprising:

[0057] An image enhancement module is used to collect the image to be segmented in the intelligent security area, identify the original pixel value of the image to be segmented, and enhance the image details of the image to be segmented based on the original pixel value to obtain an enhanced image;

[0058] An image background separation module is used to calculate the inter-class variance of the enhanced image, use the inter-class variance to find the foreground target pixels of the enhanced image, perform preliminary background separation on the enhanced image based on the foreground target pixels to obtain a preliminary separation image, perform positive and negative structural element sliding on the preliminary separation image to obtain a refined separation image, construct a pixel growth region of the refined separation image, and use the pixel growth region to perform precise background separation on the refined separation image to obtain a target background separation image;

[0059] An image initial segmentation module is used to collect image features of the target background separation image at multi-scale resolutions to obtain multi-scale features, perform feature fusion on the multi-scale features to obtain fusion features, and perform image initial segmentation on the target background separation image using the fusion features to obtain an initial segmented image;

[0060] The final image segmentation module is used to calculate the inter-class divergence matrix of the initial segmented image, use the inter-class divergence matrix to optimize the segmentation boundary of the initial segmented image to obtain an optimized segmented image, perform region segmentation on the optimized segmented image to obtain a region segmented image, identify image key points in the region segmented image, use the image key points to perform target recognition on the region segmented image to obtain image targets, use the fusion features to perform image semantic enhancement on the image targets to obtain enhanced target information, perform secondary image segmentation on the initial segmented image based on the enhanced target information to obtain a secondary segmented image, calculate the image segmentation level of the secondary segmented image, and when the image segmentation level meets the preset segmentation requirements, use the secondary segmented image as the final intelligent segmented image.

[0061] Compared with the problems described in the background technology, the embodiment of the present invention collects the image to be segmented in the intelligent security area, identifies its original pixel value (including color, brightness and other information), and enhances the image details according to the original pixel value. Furthermore, the present invention calculates the inter-class variance of the enhanced image (an indicator to measure the degree of grayscale difference between the foreground and background), finds the foreground target pixel based on it, performs preliminary background separation, constructs a full zero matrix and background model, and obtains a preliminary separated image. Then, through the sliding refinement separation of positive and negative structural elements, a pixel growth area is constructed and used to accurately separate the background to obtain a target background separation image, and gradually improves the precision and accuracy of foreground and background separation; further, the present invention collects the image features of the target background separation image at multi-scale resolutions, inputs it into a preconfigured neural network, extracts multi-scale features after horizontal pooling, vertical pooling, multi-layer convolution processing, and regularization processing, and then performs feature fusion to obtain fused features, and then uses the fused features to convert the target background separation image into a grayscale image, identify the separation key points, and perform non-maximum suppression, edge The detection and segmentation steps obtain the initial segmented image, which is convenient for the preliminary control of the security scene; further, the present invention calculates the inter-class divergence matrix of the initial segmented image to optimize the segmentation boundary to obtain the optimized segmented image, then performs regional segmentation on it, identifies the image key points in the regional segmented image, uses the key points to perform target recognition to obtain the image target, and uses the fusion features to semantically enhance the image target. Through normalization, construction of semantic features, association matching, semantic knowledge supplementation and target information generation steps to obtain enhanced target information, based on which the initial segmented image is subjected to secondary image segmentation to further improve the segmentation accuracy. Therefore, the image intelligent segmentation method and system based on intelligent security provided by the embodiment of the present invention can improve the image segmentation accuracy of the intelligent security system in the case of complex scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 A schematic diagram of a flow chart of an intelligent image segmentation method based on intelligent security provided by an embodiment of the present invention;

[0063] Figure 2 A schematic diagram of modules for implementing the intelligent image segmentation method based on intelligent security provided in one embodiment of the present invention.

[0064] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0065] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0066] The embodiment of the present application provides an image intelligent segmentation method based on smart security. The execution subject of the image intelligent segmentation method based on smart security includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the method provided by the embodiment of the present application. In other words, the image intelligent segmentation method based on smart security can be executed by software or hardware installed on a terminal device or a server device. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc.

[0067] Embodiment 1:

[0068] Reference Figure 1 FIG. 1 is a flow chart of an intelligent image segmentation method based on intelligent security provided by an embodiment of the present invention. In this embodiment, the intelligent image segmentation method based on intelligent security includes:

[0069] S1. Collect an image to be segmented in an intelligent security area, identify original pixel values ​​of the image to be segmented, and enhance image details of the image to be segmented based on the original pixel values ​​to obtain an enhanced image.

[0070] The embodiment of the present invention can obtain image data that needs to be subsequently processed from the intelligent security area by collecting the image to be segmented in the intelligent security area. The intelligent security area refers to an area that uses intelligent technical means to monitor and manage security in a specific range of space, such as residential communities, commercial places, and industrial plants. Various visual information in the security area, such as the activities of people and vehicles, the status of facilities, etc., plays a key role in ensuring regional safety and detecting abnormal situations.

[0071] Optionally, the image to be segmented can be obtained by continuously acquiring images of the monitored area using a high-resolution image sensor.

[0072] Furthermore, the embodiment of the present invention can understand the basic data units constituting the image by identifying the original pixel values ​​of the image to be segmented, and then formulate corresponding processing strategies and judge the quality of the image based on these values.

[0073] The original pixel value refers to the color, brightness and other information of the image at each position.

[0074] Furthermore, the embodiment of the present invention enhances the image details of the image to be segmented based on the original pixel values ​​to obtain an enhanced image, which can reduce misjudgments caused by poor image quality, improve the ability of the entire security monitoring system to capture and analyze various situations in the scene, and better serve the purpose of ensuring regional security.

[0075] As an embodiment of the present invention, the step of performing image detail enhancement on the image to be segmented based on the original pixel value to obtain an enhanced image includes: calculating the pixel mean of the original pixel value using the following formula:

[0076] ;

[0077] in, represents the mean pixel, Represents a pixel point of the image to be segmented, and H represents the pixel point The number of rows in the field, L represents the number of pixels The number of columns in the field, Pixels of the image to be segmented The pixel value of the image to be segmented is x, and y represents the horizontal coordinate of a pixel in the image to be segmented, and y represents the vertical coordinate of a pixel in the image to be segmented;

[0078] Based on the pixel mean, the image to be segmented is subjected to mean filtering to obtain a filtered image, and the filtered image is sharpened to obtain an enhanced image.

[0079] Optionally, performing mean filtering on the image to be segmented to obtain a filtered image may be achieved by using a high-pass filter.

[0080] S2. Calculate the inter-class variance of the enhanced image, use the inter-class variance to find the foreground target pixels of the enhanced image, perform preliminary background separation on the enhanced image based on the foreground target pixels to obtain a preliminary separation image, slide the positive and negative structural elements on the preliminary separation image to obtain a refined separation image, construct a pixel growth region of the refined separation image, use the pixel growth region to accurately separate the background of the refined separation image to obtain a target background separation image.

[0081] The embodiment of the present invention can provide a key data basis for subsequently accurately distinguishing the foreground and the background by calculating the inter-class variance of the enhanced image.

[0082] The inter-class variance refers to a statistical indicator that measures the degree of grayscale difference between different categories (usually simply understood as foreground and background) in an image.

[0083] As an embodiment of the present invention, the calculation of the inter-class variance of the enhanced image includes: counting the grayscale histogram of the enhanced image, calculating the average grayscale value of the enhanced image based on the grayscale histogram, calculating the background class pixel probability of the enhanced image, calculating the background class average grayscale value of the enhanced image based on the background class pixel probability, calculating the foreground class pixel probability of the enhanced image, calculating the foreground class average grayscale value of the enhanced image based on the foreground class pixel probability, and calculating the inter-class variance of the enhanced image based on the average grayscale value, the background class average grayscale value and the foreground class average grayscale value using the following formula:

[0084] ;

[0085] in, represents the between-class variance, Background class pixel probability, represents the probability of foreground class pixels, represents the average gray value of the background class, represents the average gray value of the foreground class, Represents the average gray value.

[0086] Optionally, the grayscale histogram can be obtained by scanning the enhanced image and counting the number of pixels appearing at each grayscale level. The average grayscale value can be calculated based on the grayscale histogram in combination with the grayscale mean square formula and data such as the number of pixels at each grayscale level and the grayscale value. The background pixel probability can be distinguished from the background pixels in the image according to a preset background pixel threshold, and then the proportion of the background pixels in the image is calculated, wherein the preset background pixel threshold can be set in combination with actual application analysis. The foreground pixel probability calculation method is the same as the background pixel probability principle, so it will not be elaborated. The background average grayscale value can be obtained by calculating the average grayscale values ​​of these pixels based on the determined background pixels and their grayscale conditions. The foreground average grayscale value calculation principle is the same as the background average grayscale value principle.

[0087] The embodiment of the present invention can screen out those pixels with a high probability belonging to the foreground target from a large number of pixels in the enhanced image by using the inter-class variance to find the foreground target pixels of the enhanced image, so as to achieve separation of the image target and the background.

[0088] Optionally, the foreground target pixel can be obtained by performing foreground and background separation and differentiation using the calculation result of the inter-class variance.

[0089] The embodiment of the present invention performs preliminary background separation on the enhanced image based on the foreground target pixels to obtain a preliminary separated image, which can quickly simplify the image content and reduce the amount of data and complexity of subsequent processing.

[0090] As an embodiment of the present invention, the method of performing preliminary background separation on the enhanced image based on the foreground target pixels to obtain a preliminary separated image includes: setting a background separation size of the enhanced image based on the foreground target pixels, constructing an all-zero matrix of the enhanced image using the background separation size, constructing a background model of the enhanced image using the all-zero matrix, and performing preliminary background separation on the enhanced image using the background model to obtain a preliminary separated image.

[0091] The all-zero matrix refers to a matrix in which all elements are 0, and the background model refers to a mathematical and structured representation of image background features.

[0092] Optionally, the background separation size can be determined based on relevant characteristics of the foreground target pixels (such as distribution range, etc.), and the all-zero matrix can be obtained by setting the background separation size as the specification (number of rows, number of columns) and creating a matrix with all elements being zero. The background model can be made into a background model by giving appropriate values ​​to its corresponding elements with the help of the all-zero matrix according to specific algorithm rules or model construction methods (median filtering, Gaussian noise).

[0093] The embodiment of the present invention obtains a refined separation image by sliding the positive and negative structural elements of the preliminary separation image, which can improve the refinement and accuracy of the separation of the foreground and the background.

[0094] As an embodiment of the present invention, the method of sliding the positive and negative structural elements of the preliminary separated image to obtain a refined separated image includes: querying the image size of the preliminary separated image, constructing an element sliding structure of the preliminary separated image based on the image size, performing image erosion on the preliminary separated image using the element sliding structure, and performing image dilation on the preliminary separated image using the element sliding structure to obtain a refined separated image.

[0095] The element sliding structure refers to a structural element in morphological image processing, which is a template of a custom shape (such as a common rectangle, circle, cross, etc.) and size.

[0096] Optionally, the element sliding structure can be created according to the obtained image size and the preset structural element shape (such as rectangle, circle, etc.) and size requirements. The image erosion of the preliminary separated image can utilize the constructed element sliding structure to slide and traverse the preliminary separated image in a certain order (such as from left to right, from top to bottom), and judge and update the image pixels according to the rules of the erosion algorithm, and remove the boundary pixels that do not meet the conditions to achieve the image erosion effect. The image expansion of the preliminary separated image can construct the element sliding structure to slide on the corroded image, and supplement the corresponding pixels according to the requirements of the expansion algorithm, so that the target area of ​​the image is restored to a certain extent and the boundary is refined, etc., so as to finally obtain a refined separated image.

[0097] Furthermore, the embodiment of the present invention can help the user determine the complete area range of the foreground object by constructing the pixel growth area of ​​the refined separation image.

[0098] The pixel growth region refers to a pixel set range formed by continuously "growing" and expanding outwards from some initial "seed" pixels in the image according to a specific similarity rule or growth criterion.

[0099] Optionally, the construction process of the pixel growth region is as follows: first, select some initial seed pixels in the refined separation image, and these seed pixels can be selected based on the prior features of the target object (such as pixels of specific color, grayscale value, etc.). Then, set the growth criterion, for example, stipulate that if the adjacent pixels and the seed pixels meet certain threshold conditions in terms of grayscale difference, texture characteristics, etc., they are regarded as pixels that can be included in the region. Finally, starting from the seed pixel, traverse its neighboring pixels, and continuously include the neighboring pixels that meet the conditions according to the growth criterion, continue to expand, and gradually construct a pixel growth region.

[0100] The embodiment of the present invention utilizes the pixel growth area to accurately separate the background of the refined separation image to obtain a target background separation image, which can further improve the accuracy of background separation, so that the final target background separation image can restore the actual distribution of foreground objects and backgrounds in the real scene to the greatest extent.

[0101] Optionally, the process of using the pixel growth region to accurately separate the background of the refined separation image to obtain the target background separation image is as follows: first, the pixels in the pixel growth region in the refined separation image are marked as foreground pixels, and then the remaining pixels in the image except the pixel growth region are identified as background pixels. Finally, these pixels identified as background are removed or distinguished in a specific way (such as setting them to a uniform color, etc.), so as to obtain the target background separation image.

[0102] S3. Collect image features of the target background separation image at multi-scale resolutions to obtain multi-scale features, perform feature fusion on the multi-scale features to obtain fused features, and use the fused features to perform initial image segmentation on the target background separation image to obtain an initial segmented image.

[0103] The embodiment of the present invention acquires the image features of the target background separation image at multi-scale resolutions to obtain multi-scale features, and can extract representative features from images of different scale resolutions.

[0104] The multi-scale features refer to the edge features, texture features, shape features and other elements of the image that reflect the essential attributes of the target in the image.

[0105] As an embodiment of the present invention, the image features of the target background separation image at multi-scale resolutions are collected to obtain multi-scale features, including: inputting the target background separation image into a preconfigured neural network, using the neural network to perform horizontal pooling and vertical pooling on the target background separation image to obtain a cascaded feature map, performing multi-layer convolution processing on the cascaded feature map to obtain a multi-layer convolution feature map, regularizing the multi-layer convolution feature map to obtain a multi-layer feature map, extracting image features of the multi-layer feature map to obtain multi-scale features.

[0106] Optionally, the use of the neural network to perform horizontal pooling and vertical pooling on the target background separation image to obtain a cascaded feature map can use the pooling layer in the neural network to reduce the dimension of the image data through pooling operations in the horizontal direction (such as maximum pooling, average pooling, etc.), obtain horizontal features, and also perform corresponding pooling operations in the vertical direction. After integrating the features of the two directions, the cascaded feature map is obtained. The multi-layer convolution processing of the cascaded feature map to obtain a multi-layer convolution feature map can use multiple convolution layers configured in the neural network, let the cascaded feature map pass through these convolution layers in turn, use the convolution kernel and the feature map to perform convolution operations, and continuously extract and strengthen more complex and deep image feature operations. The multi-layer feature map can use the batch normalization method to normalize the feature values ​​of each layer of the data in the multi-layer convolution feature map, so that the data distribution is more in line with expectations and overfitting is reduced.

[0107] The embodiment of the present invention performs feature fusion on the multi-scale features to obtain fused features that can make up for the shortcomings of single-scale features in describing image content, so that the fused features can characterize the target and background in the image from multiple dimensions. For example, large-scale features guide the overall segmentation direction, and small-scale features help refine the segmentation boundaries, working together to improve the ability to accurately describe the image content.

[0108] Optionally, the performing feature fusion on the multi-scale features to obtain fused features may be achieved by fusing them through a multi-scale fusion function.

[0109] Furthermore, the embodiment of the present invention performs initial image segmentation on the target background separation image by utilizing the fusion feature. The initial segmented image obtained can allow the security monitoring system to obtain preliminary information of key targets in the image at the first time, which is convenient for subsequent further analysis of the target's behavior, status, etc., such as determining whether there are suspicious persons entering a specific area, whether the vehicle is parked in a specified location, etc., thereby achieving preliminary control of the security scene.

[0110] As an embodiment of the present invention, the target background separation image is initially segmented using the fusion feature to obtain an initial segmented image, including: converting the target background separation image into a grayscale image, identifying the separation key points of the grayscale image using the fusion feature, performing non-maximum suppression on the grayscale image with the separation key points as the center to obtain a suppressed image, detecting the image edge of the suppressed image, and segmenting the suppressed image based on the image edge to obtain an initial segmented image.

[0111] The separation key points refer to the pixel points in the image that play a key role in effectively segmenting the target from the background or different regions of interest.

[0112] The optional grayscale image can be obtained by converting the target background separation image using a dual-channel method. The separation key points can be obtained by utilizing the image feature information contained in the fusion features, through a feature matching algorithm and a weighting method, by determining those pixels in the grayscale image that have an important indicative role in image segmentation. For example, typical points in the boundary area between the target and the background in the image can be used as separation key points. The grayscale image is subjected to non-maximum suppression with the separation key points as the center, and the suppressed image is obtained. The characteristic intensities (such as gradient amplitude, etc.) of the pixels in the neighborhood are compared according to the set neighborhood range, and those pixels that are not local maximum values ​​are suppressed, and only the pixels with the largest characteristic intensity are retained.

[0113] S4. Calculate the inter-class divergence matrix of the initial segmented image, use the inter-class divergence matrix to optimize the segmentation boundary of the initial segmented image to obtain an optimized segmented image, perform region segmentation on the optimized segmented image to obtain a region segmented image, identify image key points in the region segmented image, use the image key points to perform target recognition on the region segmented image to obtain image targets, use the fusion features to perform image semantic enhancement on the image targets to obtain enhanced target information, perform secondary image segmentation on the initial segmented image based on the enhanced target information to obtain a secondary segmented image, calculate the image segmentation level of the secondary segmented image, and when the image segmentation level meets the preset segmentation requirements, use the secondary segmented image as the final intelligent segmented image.

[0114] The embodiment of the present invention can provide a key quantitative basis for subsequent operations such as boundary optimization by calculating the inter-class divergence matrix of the initial segmented image.

[0115] The inter-class scatter matrix refers to an important concept used to measure the degree of discreteness between different categories of data in the fields of pattern recognition and image processing.

[0116] Optionally, the process of calculating the inter-class divergence matrix of the initial segmented image is as follows: first, determine the categories in the initial segmented image, count the number of pixels in each category and calculate their probabilities in the image, then determine the mean vector of pixels in each category in the feature space, and then calculate the overall mean vector of all pixels, and finally, obtain it based on the probabilities, the mean vectors of each category and the overall mean vector.

[0117] Furthermore, the embodiment of the present invention optimizes the segmentation boundaries of the initial segmented image by utilizing the inter-class divergence matrix to obtain an optimized segmented image, and can adjust and refine those segmentation boundaries in the initial segmented image that are not precise, have jagged edges, or are blurred, so that the boundaries are more closely aligned with the true contours of the target objects, reduce misjudged areas, and enhance the distinction between areas of different categories.

[0118] Optionally, the process of using the inter-class scatter matrix to optimize the segmentation boundary of the initial segmented image to obtain the optimized segmented image is: using the inter-class scatter matrix to analyze the inter-category discrete degree information of the initial segmented image to distinguish the importance of each pixel at the boundary of the initial segmented image, and adjusting the category or boundary position of the pixel points whose segmentation boundary is not accurate enough according to the feature differences reflected by the inter-class scatter matrix.

[0119] The embodiment of the present invention performs region segmentation on the optimized segmented image to obtain a region segmented image, which can divide the image into smaller sub-regions with more semantic consistency, and perform a finer decomposition of the image content to mine more detailed information hidden in the image.

[0120] Optionally, the optimized segmented image is subjected to regional segmentation to obtain a regional segmented image, and based on the existing segmentation boundaries in the optimized segmented image and the differences in features of different regions, regional blocks that can be further subdivided are identified and then segmented using a threshold-based segmentation method.

[0121] Furthermore, the embodiment of the present invention can screen out representative pixel points that can reflect the key features and structural information of the target object from the numerous pixels of the region segmentation image by identifying the image key points in the region segmentation image. These key points are usually located at the turning points of the contour of the target object, the places where local features are prominent, etc., such as the pixel points corresponding to the joints of a person, the corners of a vehicle, etc., and they are extracted as important reference elements for subsequent analysis.

[0122] Optionally, the image key points can be determined based on the features of different regions in the region segmentation image, such as texture, grayscale change, shape contour, etc., to determine the feature type for extracting the key points, and then extracted using Harris corner detection combined with the feature type.

[0123] The embodiment of the present invention uses the image key points to perform target recognition on the region segmented image, and the image target can be accurately classified and recognized for the target object in the image, which is one of the core functions of the intelligent security system. Only by accurately knowing what the target is can we further analyze whether its behavior is abnormal, whether it poses a threat to the security area, etc. based on its category characteristics, thereby providing an accurate basis for taking corresponding security measures and issuing warnings, and ensuring effective monitoring and safety management of the situation in the security area.

[0124] Optionally, the target recognition is performed on the region segmented image using the image key points, and the image target can be identified by combining the image key points with a target recognition algorithm in a deep learning model.

[0125] The embodiment of the present invention uses the fusion features to perform image semantic enhancement on the image target to obtain enhanced target information, which can further improve the quality and depth of the feature description of the target object, so that in complex intelligent security scenarios, the enhanced target information can more accurately help users understand the essential attributes, status, etc. of the target object. For example, in the presence of occlusion or complex background interference, the key features of the target object can still be better grasped, providing more powerful feature support for subsequent more sophisticated image segmentation, behavior analysis and other operations, thereby reducing the occurrence of misjudgments.

[0126] As an embodiment of the present invention, the method of using the fusion feature to perform image semantic enhancement on the image target to obtain enhanced target information includes: normalizing the fusion feature to obtain a normalized feature, constructing feature semantics of each sub-feature in the normalized feature to obtain a semantic feature, extracting image features of the image target, converting the image features into a feature matrix, mapping the semantic features into the feature matrix to associate and match the semantic features with the image features to obtain a matching feature, supplementing the matching feature with semantic knowledge to obtain a semantic knowledge supplement feature, using the semantic knowledge supplement feature as output information, and then generating target information for the semantic knowledge supplement feature to obtain enhanced target information.

[0127] Optionally, the normalization of the fused features to obtain normalized features can be implemented using a normalization function, and the semantic features can be obtained by analyzing the image semantic meaning represented by each sub-feature in the normalized features based on a predefined semantic knowledge base or semantic mapping rules learned through a trained model, assigning corresponding semantic labels to them, etc., and combining these semantically annotated sub-features. The association matching of the semantic features and the image features can be matched using a cosine similarity function, and the semantic knowledge supplementation of the matching features to obtain semantic knowledge supplement features can be obtained by supplementing and improving the relatively lacking semantic parts in the matching features based on the richer semantic knowledge contained in the semantic features and related knowledge graphs, rules, etc.

[0128] In the embodiment of the present invention, by performing secondary image segmentation on the initial segmented image based on the enhanced target information, the secondary segmented image obtained can further refine the segmentation boundary between the target object and the background, thereby improving the precision and accuracy of segmentation.

[0129] Optionally, the initial segmented image is subjected to secondary image segmentation based on the enhanced target information, and the secondary segmented image can determine more accurate category attribution and boundary range judgment basis of each region in the initial segmented image according to the rich semantics and feature content in the enhanced target information, and then use the image segmentation algorithm (such as based on threshold, region growing, etc.) to re-segment the initial segmented image according to the newly determined basis.

[0130] The embodiment of the present invention can determine whether the current segmentation result has reached the expected quality requirement by calculating the image segmentation level of the secondary segmented image.

[0131] As an embodiment of the present invention, the calculating the image segmentation level of the secondary segmented image includes: calculating the mean intersection and union ratio of the secondary segmented image by:

[0132] ;

[0133] in, represents the even intersection and ratio, It represents the number of pixels in the secondary segmentation image whose true value is class a but is predicted to be class b. Indicates the number of categories of the secondary segmented image, represents the number of pixels predicted correctly in the secondary segmentation image, represents the number of pixels predicted incorrectly in the secondary segmentation image,

[0134] The segmentation level of the secondary segmented image is evaluated according to the mean intersection union ratio.

[0135] Optionally, the evaluating of the segmentation level of the secondary segmented image according to the mean intersection and union ratio may be determined by comparing the mean intersection and union ratio with a preset threshold.

[0136] Furthermore, the embodiments of the present invention can obtain an accurately segmented intelligent security image by using the secondary segmented image as the final intelligent segmented image when the image segmentation level meets the preset segmentation requirement, thereby improving the security level of the intelligent security area.

[0137] It should be explained that the preset segmentation requirement refers to setting the segmentation standard according to the user's needs, which can be set to 0.8, and the specific setting needs to be combined with the actual application scenario.

[0138] Embodiment 2:

[0139] like Figure 2 The figure shows a functional module diagram of an image intelligent segmentation system based on intelligent security according to the present invention.

[0140] The image intelligent segmentation system 200 based on intelligent security of the present invention can be installed in an electronic device. According to the functions to be implemented, the image intelligent segmentation system based on intelligent security can include an image enhancement module 201, an image background separation module 202, an image initial segmentation module 203 and an image final segmentation module 204. The module of the present invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0141] In the embodiment of the present invention, the functions of each module / unit are as follows:

[0142] The image enhancement module 201 is used to collect the image to be segmented in the intelligent security area, identify the original pixel value of the image to be segmented, and enhance the image details of the image to be segmented based on the original pixel value to obtain an enhanced image;

[0143] The image background separation module 202 is used to calculate the inter-class variance of the enhanced image, use the inter-class variance to find the foreground target pixels of the enhanced image, perform preliminary background separation on the enhanced image based on the foreground target pixels to obtain a preliminary separation image, perform positive and negative structural element sliding on the preliminary separation image to obtain a refined separation image, construct a pixel growth region of the refined separation image, and use the pixel growth region to perform precise background separation on the refined separation image to obtain a target background separation image;

[0144] The image initial segmentation module 203 is used to collect image features of the target background separation image at multi-scale resolutions to obtain multi-scale features, perform feature fusion on the multi-scale features to obtain fusion features, and perform image initial segmentation on the target background separation image using the fusion features to obtain an initial segmented image;

[0145] The image final segmentation module 204 is used to calculate the inter-class divergence matrix of the initial segmented image, use the inter-class divergence matrix to optimize the segmentation boundary of the initial segmented image to obtain an optimized segmented image, perform region segmentation on the optimized segmented image to obtain a region segmented image, identify image key points in the region segmented image, use the image key points to perform target recognition on the region segmented image to obtain image targets, use the fusion features to perform image semantic enhancement on the image targets to obtain enhanced target information, perform secondary image segmentation on the initial segmented image based on the enhanced target information to obtain a secondary segmented image, calculate the image segmentation level of the secondary segmented image, and when the image segmentation level meets the preset segmentation requirements, use the secondary segmented image as the final intelligent segmented image.

[0146] In detail, each module in the intelligent security-based image intelligent segmentation system 200 in the embodiment of the present invention is used in the same manner as described above. Figure 1 The same technical means are used as the image intelligent segmentation method based on intelligent security described in , and can produce the same technical effects, so I will not go into details here.

[0147] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0148] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. An intelligent image segmentation method based on intelligent security, characterized in that: The method comprises: Collecting an image to be segmented in the intelligent security area, identifying original pixel values ​​of the image to be segmented, and enhancing image details of the image to be segmented based on the original pixel values ​​to obtain an enhanced image; Calculating the inter-class variance of the enhanced image, using the inter-class variance to find the foreground target pixels of the enhanced image, performing preliminary background separation on the enhanced image based on the foreground target pixels to obtain a preliminary separation image, performing positive and negative structural element sliding on the preliminary separation image to obtain a refined separation image, constructing a pixel growth region of the refined separation image, and using the pixel growth region to perform precise background separation on the refined separation image to obtain a target background separation image; Collecting image features of the target background separation image at multi-scale resolutions to obtain multi-scale features, performing feature fusion on the multi-scale features to obtain fusion features, and performing initial image segmentation on the target background separation image using the fusion features to obtain an initial segmented image; Calculate the inter-class divergence matrix of the initial segmented image, use the inter-class divergence matrix to optimize the segmentation boundary of the initial segmented image to obtain an optimized segmented image, perform region segmentation on the optimized segmented image to obtain a region segmented image, identify image key points in the region segmented image, use the image key points to perform target recognition on the region segmented image to obtain an image target, use the fusion features to perform image semantic enhancement on the image target to obtain enhanced target information, perform secondary image segmentation on the initial segmented image based on the enhanced target information to obtain a secondary segmented image, calculate the image segmentation level of the secondary segmented image, and when the image segmentation level meets the preset segmentation requirements, use the secondary segmented image as the final intelligent segmented image.

2. The method for intelligent image segmentation based on intelligent security as claimed in claim 1, characterized in that: The step of performing image detail enhancement on the image to be segmented based on the original pixel values ​​to obtain an enhanced image comprises: The pixel mean of the original pixel value is calculated using the following formula: ； in, represents the mean pixel, Represents a pixel point of the image to be segmented, and H represents the pixel point The number of rows in the field, L represents the number of pixels The number of columns in the field, Pixels of the image to be segmented The pixel value of the image to be segmented is x, and y represents the horizontal coordinate of a pixel in the image to be segmented, and y represents the vertical coordinate of a pixel in the image to be segmented; Based on the pixel mean, the image to be segmented is subjected to mean filtering to obtain a filtered image, and the filtered image is sharpened to obtain an enhanced image.

3. The method for intelligent image segmentation based on intelligent security as claimed in claim 1, characterized in that: The calculating the inter-class variance of the enhanced image comprises: Counting the grayscale histogram of the enhanced image; Calculating the average grayscale value of the enhanced image based on the grayscale histogram, and calculating the background pixel probability of the enhanced image; Based on the background class pixel probability, calculating the background class average grayscale value of the enhanced image; Calculating the probability of foreground pixels of the enhanced image; Based on the foreground pixel probability, calculating the foreground average grayscale value of the enhanced image; Based on the average gray value, the average gray value of the background class and the average gray value of the foreground class, the inter-class variance of the enhanced image is calculated using the following formula: ； in, represents the between-class variance, Background class pixel probability, represents the probability of foreground class pixels, represents the average gray value of the background class, represents the average gray value of the foreground class, Represents the average gray value.

4. The method for intelligent image segmentation based on intelligent security as claimed in claim 1, characterized in that: The step of performing preliminary background separation on the enhanced image based on the foreground target pixels to obtain a preliminary separated image includes: Based on the foreground target pixels, setting a background separation size of the enhanced image; Using the background separation size, constructing an all-zero matrix of the enhanced image; Using the all-zero matrix, constructing a background model of the enhanced image; The background model is used to perform preliminary background separation on the enhanced image to obtain a preliminary separated image.

5. The method for intelligent image segmentation based on intelligent security as claimed in claim 1, characterized in that: The performing positive and negative structural element sliding on the preliminary separated image to obtain a refined separated image includes: Querying the image size of the preliminary separated image; constructing an element sliding structure of the preliminary separated image based on the image size; The element sliding structure is used to perform image erosion on the preliminary separated image, and the element sliding structure is used to perform image dilation on the preliminary separated image to obtain a refined separated image.

6. The method for intelligent image segmentation based on intelligent security as claimed in claim 1, characterized in that: The collecting image features of the target background separation image at multi-scale resolutions to obtain multi-scale features includes: Inputting the target background separated image into a preconfigured neural network; Using the neural network to perform horizontal pooling and vertical pooling on the target background separation image to obtain a cascade feature map; Performing multi-layer convolution processing on the cascaded feature map to obtain a multi-layer convolution feature map; Performing regularization processing on the multi-layer convolution feature map to obtain a multi-layer feature map; The image features of the multi-layer feature map are extracted to obtain multi-scale features.

7. The method for intelligent image segmentation based on intelligent security as claimed in claim 1, characterized in that: The method of performing initial image segmentation on the target background separated image by using the fusion feature to obtain an initial segmented image includes: Converting the target background separated image into a grayscale image; identifying separation key points of the grayscale image using the fusion features; Performing non-maximum suppression on the grayscale image with the separation key point as the center to obtain a suppressed image; An image edge of the suppressed image is detected, and the suppressed image is segmented based on the image edge to obtain an initial segmented image.

8. The method for intelligent image segmentation based on intelligent security as claimed in claim 1, characterized in that: The using the fusion feature to perform image semantic enhancement on the image target to obtain enhanced target information includes: Normalizing the fused features to obtain normalized features; Constructing feature semantics of each sub-feature in the normalized feature to obtain a semantic feature; Extracting image features of the image target and converting the image features into a feature matrix; Mapping the semantic features into the feature matrix to perform association matching on the semantic features and the image features to obtain matching features; Performing semantic knowledge supplementation on the matching features to obtain semantic knowledge supplementation features; After the semantic knowledge supplementary features are used as output information, target information is generated for the semantic knowledge supplementary features to obtain enhanced target information.

9. The method for intelligent image segmentation based on intelligent security as claimed in claim 1, characterized in that: The calculating the image segmentation level of the secondary segmented image comprises: The mean intersection and union ratio of the secondary segmented image is calculated as follows: ； in, represents the even intersection and ratio, It represents the number of pixels in the secondary segmentation image whose true value is class a but is predicted to be class b. Indicates the number of categories of the secondary segmented image, represents the number of pixels predicted correctly in the secondary segmentation image, represents the number of pixels predicted incorrectly in the secondary segmentation image, The segmentation level of the secondary segmented image is evaluated according to the mean intersection union ratio.

10. An intelligent image segmentation system based on intelligent security, characterized in that: The system comprises: An image enhancement module is used to collect the image to be segmented in the intelligent security area, identify the original pixel value of the image to be segmented, and enhance the image details of the image to be segmented based on the original pixel value to obtain an enhanced image; An image background separation module is used to calculate the inter-class variance of the enhanced image, use the inter-class variance to find the foreground target pixels of the enhanced image, perform preliminary background separation on the enhanced image based on the foreground target pixels to obtain a preliminary separation image, perform positive and negative structural element sliding on the preliminary separation image to obtain a refined separation image, construct a pixel growth region of the refined separation image, and use the pixel growth region to perform precise background separation on the refined separation image to obtain a target background separation image; An image initial segmentation module is used to collect image features of the target background separation image at multi-scale resolutions to obtain multi-scale features, perform feature fusion on the multi-scale features to obtain fusion features, and perform image initial segmentation on the target background separation image using the fusion features to obtain an initial segmented image; The final image segmentation module is used to calculate the inter-class divergence matrix of the initial segmented image, use the inter-class divergence matrix to optimize the segmentation boundary of the initial segmented image to obtain an optimized segmented image, perform region segmentation on the optimized segmented image to obtain a region segmented image, identify image key points in the region segmented image, use the image key points to perform target recognition on the region segmented image to obtain image targets, use the fusion features to perform image semantic enhancement on the image targets to obtain enhanced target information, perform secondary image segmentation on the initial segmented image based on the enhanced target information to obtain a secondary segmented image, calculate the image segmentation level of the secondary segmented image, and when the image segmentation level meets the preset segmentation requirements, use the secondary segmented image as the final intelligent segmented image.

Images