A Deep Learning-Based Intelligent Image Information Analysis System

Abstract

This invention discloses a deep learning-based intelligent image information analysis system, belonging to the field of image analysis technology. It solves the technical problem of difficulty in accurately acquiring image information for each grid in an image, making it difficult to adapt to high-precision target detection and affecting the accuracy of target detection. By moving the detection window within the grid, the size and proportion of the moving detection window can be flexibly adjusted to adapt to target objects of different sizes and shapes, facilitating rapid extraction of image features within the grid. Simultaneously, this flexibility enables the model to better identify and locate various types of objects, thereby improving detection accuracy and efficiency. Furthermore, pre-classifying the grid and determining which grids may contain target objects is particularly important for applications requiring precise boundary information. By collecting a dataset of target objects related to the image to be analyzed, the system ensures that the training data is highly relevant to the actual application scenario.

Description

Technical Field

[0001] This invention belongs to the field of image analysis, specifically a deep learning-based intelligent image information analysis system. Background Technology

[0002] With the improvement of computing power and the availability of large amounts of labeled data, deep learning technology has become a core technology in the field of image processing. The continuous advancement of computer hardware, especially the widespread adoption of acceleration devices such as GPUs, has significantly improved the training and inference speed of deep learning models, providing strong support for the realization of intelligent image information analysis systems. Modern deep learning models such as ResNet, Inception, and Transformer have further boosted the performance of tasks such as image recognition, object detection, and image segmentation. Intelligent image information analysis systems that utilize deep learning technology to automatically process, analyze, and interpret image data can identify objects and other key information in images. Deep learning models, especially convolutional neural networks (CNNs), are widely used due to their efficiency in image recognition and classification tasks.

[0003] Most image information analysis systems directly detect target objects in images, making it difficult to accurately obtain image information for each grid in the image. This makes them unsuitable for high-precision target detection. When training the model, it is difficult to consider the influence of the background image on target recognition, which affects the accuracy of target detection. Summary of the Invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes an image information intelligent analysis system based on deep learning to solve the technical problem that it is difficult to accurately obtain image information of each grid in an image, and it is difficult to adapt to the detection of highly detailed targets, thus affecting the accuracy of target detection.

[0005] To address the aforementioned problems, a first aspect of the present invention provides an intelligent image information analysis system based on deep learning, comprising:

[0006] Image processing module: Collects image data to be analyzed through a smart terminal, preprocesses the image, performs edge detection on the preprocessed image, marks the detected area, and divides the marked area into a grid.

[0007] Feature extraction module: Detects image information by moving the detection window within the grid of the labeled area, extracts features from the labeled area, and generates feature vectors;

[0008] Image detection module: Detects whether there is a target object in the grid, classifies the grid without the target object as the reference grid, classifies the grid with the boundary line of the target object as the boundary grid, and classifies the grid inside the boundary line of the target object as the internal grid;

[0009] Dataset processing module: Collects target object image datasets related to the image data to be analyzed. Through the image detection module, it divides the target object image dataset into reference grids, boundary grids, and internal grids, and labels the target object category and grid type.

[0010] Target recognition module: A convolutional neural network model is trained using labeled reference grids, boundary grids, and internal grids to identify target object categories.

[0011] Optionally, in one example of the above aspects, it also includes:

[0012] Model optimization module: At preset time intervals, the module obtains the target object category results identified by the model, retrieves several images with the highest similarity to the identified target object from the target object image dataset, sends them to experts for target object category diagnosis, conducts multiple rounds of anonymous consultation, collects the experts' diagnostic results, and optimizes the trained convolutional neural network model based on the diagnostic results.

[0013] Optionally, in one example of the above aspects, the image processing module acquires image data to be analyzed through a smart terminal, preprocesses the image, performs edge detection on the preprocessed image, labels the detected areas, and divides the labeled areas into grids, including the following steps:

[0014] Image data to be analyzed is collected via a smart terminal;

[0015] Image data is obtained from smart terminals via API interfaces;

[0016] Clean and denoise the image, and resize it to a uniform size;

[0017] The Canny edge detection algorithm is used to detect edges in the preprocessed image and generate an edge map.

[0018] Create an initially empty mask, and draw the background area on the initial mask based on the extracted contours;

[0019] The cv2.findContours() function is used to extract contour data from the edge detection results, draw contours on the background mask, and mark their bounding rectangles to distinguish between the background and foreground regions. The foreground region is then marked as the detection region.

[0020] Determine the number of rows and columns of the grid, or the size of each grid cell, and divide each labeled area into grid cells.

[0021] Optionally, in one example of the above aspects, the feature extraction module detects image information in the grid of the labeled region by moving the detection window, and extracts features from the labeled region to generate a feature vector, including the following steps:

[0022] Set a moving detection window in the grid within the labeled area, and define the size of the detection window and the sliding window mechanism. Set the window movement step size to between 1 / 2 and 1 times the side length of the detection window size, and move the detection window step by step to extract image patches at each location.

[0023] The image patch extracted from the window is subjected to color histogram features and texture features, and these features are used as the feature vector of the image patch extracted from the window.

[0024] Optionally, in one example of the above aspects, the extraction of color histogram features and texture features includes the following steps:

[0025] Obtain the image patch extracted from the window, convert the RGB image extracted from the window to an HSV image using the rgb2hsv function in MATLAB, and calculate the histograms of the hue channel, saturation channel, and brightness channel respectively using the cv2.calcHist() function in HSV space, and then normalize the histograms.

[0026] The histograms of each channel are merged to form a comprehensive color feature vector, which is used as the color histogram feature of the pixel.

[0027] The image patch extracted from the window is converted into a grayscale image, the gray-level co-occurrence matrix is ​​calculated, and the contrast, energy, entropy and uniformity of the image patch are extracted from the gray-level co-occurrence matrix as the texture features of the image patch.

[0028] For each pixel in the image patch extracted from the window, compare its grayscale value with that of its neighboring pixels, generate a binary code, and calculate the LBP histogram of the entire image as a texture descriptor.

[0029] The extracted texture features and texture descriptions are combined to form a texture feature vector.

[0030] Optionally, in one example of the above aspects, the image detection module detects whether a target object exists in the grid, classifies grids without target objects as reference grids, classifies grids with target object boundaries as boundary grids, and classifies grids inside the target object boundaries as internal grids, including the following steps:

[0031] Extract historical data of a large number of target object images, extract color histogram features and texture features of the historical data through the feature extraction module, and label the feature vectors with target objects;

[0032] A machine learning model is trained using feature vectors to identify whether an image patch is a target object.

[0033] The feature vector of the image patch extracted from the window is fed into the trained machine learning model to identify whether the image patch extracted from the window is the target object;

[0034] If all image patches extracted from the window in the grid are not identified as target objects, the corresponding grid is determined to be a grid where no target object exists and is classified as a reference grid.

[0035] If all the image patches extracted from the window in the grid are identified as the target object, then the corresponding grid is determined to be a grid inside the boundary line of the target object and is classified as an internal grid.

[0036] If a portion of the image patch extracted from the window in the grid is identified as a target object, while a portion is not, then the corresponding grid is determined to be a grid with a target object boundary line and is classified as a boundary grid.

[0037] Optionally, in one example of the above aspects, the target recognition module trains a convolutional neural network model using labeled reference grids, boundary grids, and internal grids to identify target object categories, including the following steps:

[0038] Obtain the machine learning model trained by the image detection module, identify whether the image patch extracted from the window is the target object, and obtain the quantity data of the reference grid, boundary grid and internal grid;

[0039] Calculate the initial reference weights of the reference grid, boundary grid, and internal grid based on the number of reference grids, boundary grids, and internal grids.

[0040] The convolutional neural network model was trained using the labeled reference grid, boundary grid, and internal grid respectively.

[0041] The output layer of the trained convolutional neural network model is connected to a weighted fusion layer. The weighted fusion layer is set according to the reference weights of the reference grid, the boundary grid, and the internal grid. The weighted fusion layer fuses the convolutional neural network models trained on the reference grid, the boundary grid, and the internal grid respectively.

[0042] The fused CNN convolutional neural network model is used to identify the category of the target object.

[0043] Optionally, in one example of the above aspects, the target recognition module calculates the initial reference weights of the reference grid, boundary grid, and internal grid based on the quantity data of the reference grid, boundary grid, and internal grid, including the following steps:

[0044] The statistical data includes the number of reference grids, boundary grids, and internal grids, as well as the total number of grids;

[0045] The initial reference weights of the reference grid, internal grid, and boundary grid are calculated sequentially.

[0046] The reference weight Cr of the reference grid is calculated using the following formula:

[0047]

[0048] The reference weight Ci of the internal mesh is calculated using the following formula:

[0049]

[0050] The reference weight Cb of the boundary grid is calculated using the following formula:

[0051] Cb = 1-Cr-Ci

[0052] Where G is the total number of grids and Gi is the number of internal grids.

[0053] Optionally, in one example of the above aspects, the model optimization module obtains the target object category results identified by the model at preset time intervals, and retrieves several image data with the highest similarity to the identified target object from the target object image dataset as reference information, including the following steps:

[0054] At preset time intervals, the target object category results identified by the fusion model are obtained, as well as the target object category identification results of the convolutional neural network models trained on the reference grid, boundary grid, and internal grid in the weighted fusion layer of the fusion model.

[0055] Retrieve target object image data from target object image datasets in fields related to the image data to be analyzed, and extract color histogram features and texture features through the object feature extraction module;

[0056] The similarity between the color histogram features of the target object image data and the color histogram features of the identified target object grid is calculated using the histogram intersection method. Where hi and gi represent the values ​​of the two color histograms in the i-th color interval, N represents the dimension of the color histogram, i.e. the number of color intervals, and the result is normalized.

[0057] The contrast, energy, entropy, and uniformity parameters in the texture features of the target object image data and the texture features of the target object mesh are calculated as feature vectors. The cosine similarity between the two feature vectors is calculated as the texture feature similarity, and the results are normalized.

[0058] The color histogram feature similarity and texture feature similarity are weighted and averaged to select the image data with the highest similarity to the target object, which are then used as reference information.

[0059] Optionally, in one example of the above aspects, collecting expert diagnostic results and optimizing the trained convolutional neural network model based on these results includes the following steps:

[0060] Collect expert diagnostic results, statistically analyze the correct proportion of target object category recognition results of the fusion model, and the correct proportion of target object category recognition results of the convolutional neural network models trained on the reference grid, boundary grid and internal grid in the weighted fusion layer.

[0061] If the accuracy of the convolutional neural network model trained on the reference grid is less than 80%, and less than the accuracy of the object category recognition results of the convolutional neural network models trained on the boundary grid and the inner grid respectively, the reference weight of the convolutional neural network model trained on the reference grid is reduced by 1% until the reference weight is reduced to 8%, at which point the reference weight will not be reduced further; otherwise, the reference weight of the convolutional neural network model trained on the reference grid will not be changed.

[0062] In this process, after the reference weights of the convolutional neural network model trained on the reference grid are changed, the reference weights of the convolutional neural network models trained on the boundary grid and the inner grid are recalculated by the target recognition module.

[0063] Compared with the prior art, the beneficial effects of the present invention are:

[0064] This invention moves the detection window within a grid, allowing for flexible adjustment of its size and proportion to accommodate target objects of different sizes and shapes. This facilitates rapid extraction of image features within the grid. Furthermore, this flexibility enables the model to better identify and locate various types of objects, thereby improving detection accuracy and efficiency. In addition, pre-classifying the grid and determining which grids may contain target objects is particularly important for applications requiring precise boundary information, such as medical image analysis and fine object segmentation.

[0065] This invention collects a dataset of target objects related to the images to be analyzed, ensuring that the training data is highly relevant to the actual application scenario, which facilitates the improvement of the model's generalization ability and accuracy. At the same time, the reference grid, boundary grid, and internal grid in the image dataset are classified and labeled to provide richer information and help the model learn how to accurately identify target objects. Attached Figure Description

[0066] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0067] Figure 1 This is a schematic diagram of the system framework of the present invention. Detailed Implementation

[0068] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0069] Please see Figure 1 The first aspect of this invention provides an intelligent image information analysis system based on deep learning, comprising:

[0070] Image processing module: Collects image data to be analyzed through a smart terminal, preprocesses the image, performs edge detection on the preprocessed image, marks the detected area, and divides the marked area into a grid.

[0071] Feature extraction module: Detects image information by moving the detection window within the grid of the labeled area, extracts features from the labeled area, and generates feature vectors;

[0072] Image detection module: Detects whether there is a target object in the grid, classifies the grid without the target object as the reference grid, classifies the grid with the boundary line of the target object as the boundary grid, and classifies the grid inside the boundary line of the target object as the internal grid;

[0073] Dataset processing module: Collects target object image datasets related to the image data to be analyzed. Through the image detection module, it divides the target object image dataset into reference grids, boundary grids, and internal grids, and labels the target object category and grid type.

[0074] Target recognition module: A convolutional neural network model is trained using labeled reference grids, boundary grids, and internal grids to identify target object categories.

[0075] Specifically, in this embodiment, the feature extraction module detects image information in the grid of the labeled area by moving the detection window, and extracts features from the labeled area to generate feature vectors; the image detection module detects whether there is a target object in the grid, classifies the grid without a target object as a reference grid, classifies the grid with a target object boundary line as a boundary grid, and classifies the grid inside the target object boundary line as an internal grid.

[0076] By moving the detection window within the grid, its size and scale can be flexibly adjusted to accommodate target objects of different sizes and shapes, facilitating rapid extraction of image features within the grid. This flexibility also enables the model to better identify and locate various object types. Classifying the grid into reference grids, boundary grids, and internal grids helps reduce false detection and false positive rates. For example, by identifying and ignoring reference grids where no target object exists, detection accuracy and efficiency are improved. Furthermore, pre-classifying the grid and determining which grids might contain target objects optimizes computational resource usage. Detailed grid classification, such as boundary grids and internal grids, improves the accuracy of target object boundary recognition. This is particularly important for applications requiring precise boundary information, such as medical image analysis and fine object segmentation.

[0077] The image detection module detects whether there is a target object in the grid, classifies the grid without a target object as a reference grid, classifies the grid with a target object boundary line as a boundary grid, and classifies the grid inside the target object boundary line as an internal grid.

[0078] Classifying the mesh into boundary meshes and interior meshes helps to more accurately locate and identify the boundaries of target objects. This is especially important for applications requiring high-precision target recognition and localization, such as autonomous vehicles and robot navigation. Furthermore, distinguishing between the interior and boundary regions of a target object during object segmentation improves segmentation accuracy. This is particularly useful for multi-object detection and segmentation in complex scenes. Detailed mesh classification facilitates more accurate determination of which regions contain target objects and which do not, helping to reduce false positives. Adjusting the detection window size or using multi-scale detection strategies allows for the effective detection of target objects of different sizes and shapes. This flexibility enables later-trained models to adapt to various complex real-world application scenarios.

[0079] The dataset processing module collects target object image datasets related to the image data to be analyzed. The image detection module divides the target object image dataset into reference grids, boundary grids, and internal grids, and labels the target object category and grid type. The target recognition module trains a convolutional neural network model using the labeled reference grids, boundary grids, and internal grids to identify the target object category.

[0080] By collecting a dataset of target objects related to the images to be analyzed, we can ensure that the training data is highly relevant to real-world application scenarios, thus improving the model's generalization ability and accuracy. Classifying and labeling the reference grid, boundary grid, and internal grid in the image dataset provides richer information, helping the model learn how to accurately identify target objects. Clear grid classification and labeling help the model distinguish target objects from the background, reducing false positives and false negatives, such as failing to identify the true target.

[0081] Meanwhile, labeling the target object category and mesh type can support the model to perform multi-task learning, that is, to learn to classify target objects and identify their spatial locations at the same time, which can further improve the model's performance.

[0082] In one embodiment of the present invention, it further includes:

[0083] Model optimization module: At preset time intervals, the module obtains the target object category results identified by the model, retrieves several images with the highest similarity to the identified target object from the target object image dataset, sends them to experts for target object category diagnosis, conducts multiple rounds of anonymous consultation, collects the experts' diagnostic results, and optimizes the trained convolutional neural network model based on the diagnostic results.

[0084] In one embodiment of the present invention, the image processing module collects image data to be analyzed through a smart terminal, preprocesses the image, performs edge detection on the preprocessed image, marks the detected areas, and divides the marked areas into grids, including the following steps:

[0085] Image data to be analyzed is collected via a smart terminal;

[0086] Image data is obtained from smart terminals through APIs (such as OpenCV, PIL, etc.).

[0087] Clean and denoise the image, and resize it to a uniform size;

[0088] The Canny edge detection algorithm is used to detect edges in the preprocessed image and generate an edge map.

[0089] Create an initially empty mask, and draw the background area on the initial mask based on the extracted contours;

[0090] The cv2.findContours() function is used to extract contour data from the edge detection results, draw contours on the background mask, and mark their bounding rectangles to distinguish between the background and foreground regions. The foreground region is then marked as the detection region.

[0091] Determine the number of rows and columns of the grid, or the size of each grid cell, for example, 64x64 pixels, and divide each labeled area into grids.

[0092] In one embodiment of the present invention, the feature extraction module detects image information in the grid of the labeled region by moving the detection window, and extracts features from the labeled region to generate a feature vector, including the following steps:

[0093] Set up a moving detection window in the grid within the labeled area and define the size of the detection window, for example, 10x10 pixels and a sliding window mechanism. Set the window movement step size to between 1 / 2 and 1 times the side length of the detection window size. Move the detection window step by step and extract image patches at each location.

[0094] The image patch extracted from the window is subjected to color histogram features and texture features, and these features are used as the feature vector of the image patch extracted from the window.

[0095] In one embodiment of the present invention, the extraction of color histogram features and texture features includes the following steps:

[0096] Obtain the image patch extracted from the window, convert the RGB image extracted from the window to an HSV image using the rgb2hsv function in MATLAB, and calculate the histograms of the hue channel, saturation channel, and brightness channel respectively using the cv2.calcHist() function in HSV space, and then normalize the histograms.

[0097] The histograms of each channel are merged to form a comprehensive color feature vector, which is used as the color histogram feature of the pixel.

[0098] The image patch extracted from the window is converted into a grayscale image, the gray-level co-occurrence matrix is ​​calculated, and the contrast, energy, entropy and uniformity of the image patch are extracted from the gray-level co-occurrence matrix as the texture features of the image patch.

[0099] For each pixel in the image patch extracted from the window, compare its grayscale value with that of its neighboring pixels, generate a binary code, and calculate the LBP histogram of the entire image as a texture descriptor.

[0100] The extracted texture features and texture descriptions are combined to form a texture feature vector.

[0101] Specifically, select the relative position when calculating GLCM, for example, the distance is 1, and the direction can be 0°, 45°, 90° and 135°.

[0102] For each direction, initialize the GLCM matrix with a size of 256x256, assuming a grayscale value range of 0 to 255, to count the frequency of pixel pairs.

[0103] For each pixel, check its neighboring pixels in the specified direction and at a specified distance.

[0104] The grayscale value of the current pixel and the grayscale values ​​of its neighboring pixels are used as indices to increment the count in GLCM.

[0105] Once the GLCM gray-level co-occurrence matrix is ​​calculated, texture features are extracted, including contrast, energy, entropy, and uniformity.

[0106] In one embodiment of the present invention, the image detection module detects whether a target object exists in the grid, classifies the grid without a target object as a reference grid, classifies the grid with a target object boundary line as a boundary grid, and classifies the grid inside the target object boundary line as an internal grid, including the following steps:

[0107] Extract historical data of a large number of target object images, extract color histogram features and texture features of the historical data through the feature extraction module, and label the feature vectors with target objects;

[0108] A machine learning model is trained using feature vectors to identify whether an image patch is a target object.

[0109] The feature vector of the image patch extracted from the window is fed into the trained machine learning model to identify whether the image patch extracted from the window is the target object;

[0110] If all image patches extracted from the window in the grid are not identified as target objects, the corresponding grid is determined to be a grid where no target object exists and is classified as a reference grid.

[0111] If all the image patches extracted from the window in the grid are identified as the target object, then the corresponding grid is determined to be a grid inside the boundary line of the target object and is classified as an internal grid.

[0112] If a portion of the image patch extracted from the window in the grid is identified as a target object, while a portion is not, then the corresponding grid is determined to be a grid with a target object boundary line and is classified as a boundary grid.

[0113] In one embodiment of the present invention, the target recognition module trains a convolutional neural network model using labeled reference grids, boundary grids, and internal grids to identify target object categories, including the following steps:

[0114] Obtain the machine learning model trained by the image detection module, identify whether the image patch extracted from the window is the target object, and obtain the quantity data of the reference grid, boundary grid and internal grid;

[0115] Calculate the initial reference weights of the reference grid, boundary grid, and internal grid based on the number of reference grids, boundary grids, and internal grids.

[0116] The convolutional neural network model was trained using the labeled reference grid, boundary grid, and internal grid respectively.

[0117] The output layer of the trained convolutional neural network model is connected to a weighted fusion layer. The weighted fusion layer is set according to the reference weights of the reference grid, the boundary grid, and the internal grid. The weighted fusion layer fuses the convolutional neural network models trained on the reference grid, the boundary grid, and the internal grid respectively.

[0118] The fused CNN convolutional neural network model is used to identify the category of the target object.

[0119] In one embodiment of the present invention, the target recognition module calculates the initial reference weights of the reference grid, boundary grid, and internal grid based on the quantity data of the reference grid, boundary grid, and internal grid, including the following steps:

[0120] The statistical data includes the number of reference grids, boundary grids, and internal grids, as well as the total number of grids;

[0121] The initial reference weights of the reference grid, internal grid, and boundary grid are calculated sequentially.

[0122] The reference weight Cr of the reference grid is calculated using the following formula:

[0123]

[0124] The reference weight Ci of the internal mesh is calculated using the following formula:

[0125]

[0126] The reference weight Cb of the boundary grid is calculated using the following formula:

[0127] Cb = 1-Cr-Ci

[0128] Where G is the total number of grids and Gi is the number of internal grids.

[0129] In one embodiment of the present invention, the model optimization module acquires the target object category result identified by the model at preset time intervals, and retrieves a number of image data with the highest similarity to the identified target object from the target object image dataset as reference information, including the following steps:

[0130] At preset time intervals, the target object category results identified by the fusion model are obtained, as well as the target object category identification results of the convolutional neural network models trained on the reference grid, boundary grid, and internal grid in the weighted fusion layer of the fusion model.

[0131] Retrieve target object image data from target object image datasets in fields related to the image data to be analyzed, and extract color histogram features and texture features through the object feature extraction module;

[0132] The similarity between the color histogram features of the target object image data and the color histogram features of the identified target object grid is calculated using the histogram intersection method. Where hi and gi represent the values ​​of the two color histograms in the i-th color interval, N represents the dimension of the color histogram, i.e. the number of color intervals, and the result is normalized.

[0133] The contrast, energy, entropy, and uniformity parameters in the texture features of the target object image data and the texture features of the target object mesh are calculated as feature vectors. The cosine similarity between the two feature vectors is calculated as the texture feature similarity, and the results are normalized.

[0134] The color histogram feature similarity and texture feature similarity are weighted and averaged to select the image data with the highest similarity to the target object, which are then used as reference information.

[0135] In one embodiment of the present invention, collecting expert diagnostic results and optimizing the trained convolutional neural network model based on the diagnostic results includes the following steps:

[0136] Collect expert diagnostic results, statistically analyze the correct proportion of target object category recognition results of the fusion model, and the correct proportion of target object category recognition results of the convolutional neural network models trained on the reference grid, boundary grid and internal grid in the weighted fusion layer.

[0137] If the accuracy of the convolutional neural network model trained on the reference grid is less than 80%, and less than the accuracy of the object category recognition results of the convolutional neural network models trained on the boundary grid and the inner grid respectively, the reference weight of the convolutional neural network model trained on the reference grid is reduced by 1% until the reference weight is reduced to 8%, at which point the reference weight will not be reduced further; otherwise, the reference weight of the convolutional neural network model trained on the reference grid will not be changed.

[0138] In this process, after the reference weights of the convolutional neural network model trained on the reference grid are changed, the reference weights of the convolutional neural network models trained on the boundary grid and the inner grid are recalculated by the target recognition module.

[0139] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. A deep learning-based intelligent image information analysis system, characterized in that, include: Image processing module: Collects image data to be analyzed through a smart terminal, preprocesses the image, performs edge detection on the preprocessed image, marks the detected area, and divides the marked area into a grid. Feature extraction module: Detects image information by moving the detection window within the grid of the labeled area, extracts features from the labeled area, and generates feature vectors; Image detection module: Detects whether there is a target object in the grid, classifies the grid without the target object as the reference grid, classifies the grid with the boundary line of the target object as the boundary grid, and classifies the grid inside the boundary line of the target object as the internal grid; Dataset processing module: Collects target object image datasets related to the image data to be analyzed. Through the image detection module, it divides the target object image dataset into reference grids, boundary grids, and internal grids, and labels the target object category and grid type. Target recognition module: Trains a convolutional neural network model using labeled reference grids, boundary grids, and internal grids to identify target object categories; The target recognition module trains a convolutional neural network model using labeled reference grids, boundary grids, and internal grids to identify target object categories, including the following steps: Obtain the machine learning model trained by the image detection module, identify whether the image patch extracted from the window is the target object, and obtain the quantity data of the reference grid, boundary grid and internal grid; Calculate the initial reference weights of the reference grid, boundary grid, and internal grid based on the number of reference grids, boundary grids, and internal grids. The convolutional neural network model was trained using the labeled reference grid, boundary grid, and internal grid respectively. The output layer of the trained convolutional neural network model is connected to a weighted fusion layer. The weighted fusion layer is set according to the reference weights of the reference grid, the boundary grid, and the internal grid. The weighted fusion layer fuses the convolutional neural network models trained on the reference grid, the boundary grid, and the internal grid respectively. The fused CNN convolutional neural network model is used to identify the category of the target object. The target recognition module calculates the initial reference weights of the reference grid, boundary grid, and internal grid based on the quantity data of the reference grid, boundary grid, and internal grid, including the following steps: The statistical data includes the number of reference grids, boundary grids, and internal grids, as well as the total number of grids; The initial reference weights of the reference grid, internal grid, and boundary grid are calculated sequentially. The reference weight Cr of the reference grid is calculated using the following formula: ； The reference weight Ci of the internal mesh is calculated using the following formula: ； The reference weight Cb of the boundary grid is calculated using the following formula: ； Where G is the total number of grids and Gi is the number of internal grids.

2. The image information intelligent analysis system based on deep learning according to claim 1, characterized in that, Also includes: Model optimization module: At preset time intervals, the module obtains the target object category results identified by the model, retrieves several images with the highest similarity to the identified target object from the target object image dataset, sends them to experts for target object category diagnosis, conducts multiple rounds of anonymous consultation, collects the experts' diagnostic results, and optimizes the trained convolutional neural network model based on the diagnostic results.

3. The image information intelligent analysis system based on deep learning according to claim 1, characterized in that, The image processing module collects image data to be analyzed through a smart terminal, preprocesses the image, performs edge detection on the preprocessed image, labels the detected areas, and divides the labeled areas into grids, including the following steps: Image data to be analyzed is collected via a smart terminal; Image data is obtained from smart terminals via API interfaces; Clean and denoise the image, and resize it to a uniform size; The Canny edge detection algorithm is used to detect edges in the preprocessed image and generate an edge map. Create an initially empty mask, and draw the background area on the initial mask based on the extracted contours; The cv2.findContours() function is used to extract contour data from the edge detection results, draw contours on the background mask, and mark their bounding rectangles to distinguish between the background and foreground regions. The foreground region is then marked as the detection region. Determine the number of rows and columns of the grid, or the size of each grid cell, and divide each labeled area into grid cells.

4. The image information intelligent analysis system based on deep learning according to claim 1, characterized in that, The feature extraction module detects image information within the grid of the labeled region by moving the detection window, extracts features from the labeled region, and generates feature vectors, including the following steps: Set a moving detection window in the grid within the labeled area, and define the size of the detection window and the sliding window mechanism. Set the window movement step size to between 1 / 2 and 1 times the side length of the detection window size, and move the detection window step by step to extract image patches at each location. The image patch extracted from the window is subjected to color histogram features and texture features, and these features are used as the feature vector of the image patch extracted from the window.

5. The image information intelligent analysis system based on deep learning according to claim 4, characterized in that, Extracting color histogram features and texture features includes the following steps: Obtain the image patch extracted from the window, convert the RGB image extracted from the window to an HSV image using the rgb2hsv function in MATLAB, and calculate the histograms of the hue channel, saturation channel, and brightness channel respectively using the cv2.calcHist() function in HSV space, and then normalize the histograms. The histograms of each channel are merged to form a comprehensive color feature vector, which is used as the color histogram feature of the pixel. The image patch extracted from the window is converted into a grayscale image, the gray-level co-occurrence matrix is ​​calculated, and the contrast, energy, entropy and uniformity of the image patch are extracted from the gray-level co-occurrence matrix as the texture features of the image patch. For each pixel in the image patch extracted from the window, compare its grayscale value with that of its neighboring pixels, generate a binary code, and calculate the LBP histogram of the entire image as a texture descriptor. The extracted texture features and texture descriptions are combined to form a texture feature vector.

6. The image information intelligent analysis system based on deep learning according to claim 1, characterized in that, The image detection module detects whether a target object exists in the grid, classifies grids without target objects as reference grids, classifies grids with target object boundaries as boundary grids, and classifies grids inside target object boundaries as internal grids, including the following steps: Extract historical data of a large number of target object images, extract color histogram features and texture features of the historical data through the feature extraction module, and label the feature vectors with target objects; A machine learning model is trained using feature vectors to identify whether an image patch is a target object. The feature vector of the image patch extracted from the window is fed into the trained machine learning model to identify whether the image patch extracted from the window is the target object; If all image patches extracted from the window in the grid are not identified as target objects, the corresponding grid is determined to be a grid where no target object exists and is classified as a reference grid. If all the image patches extracted from the window in the grid are identified as the target object, then the corresponding grid is determined to be a grid inside the boundary line of the target object and is classified as an internal grid. If a portion of the image patch extracted from the window in the grid is identified as a target object, while a portion is not, then the corresponding grid is determined to be a grid with a target object boundary line and is classified as a boundary grid.

7. The image information intelligent analysis system based on deep learning according to claim 2, characterized in that, The model optimization module acquires the target object category results identified by the model at preset time intervals, and retrieves several image data with the highest similarity to the identified target object from the target object image dataset as reference information, including the following steps: At preset time intervals, the target object category results identified by the fusion model are obtained, as well as the target object category identification results of the convolutional neural network models trained on the reference grid, boundary grid, and internal grid in the weighted fusion layer of the fusion model. Retrieve target object image data from target object image datasets in fields related to the image data to be analyzed, and extract color histogram features and texture features through the object feature extraction module; The similarity between the color histogram features of the target object image data and the color histogram features of the identified target object grid is calculated using the histogram intersection method. , where hi and gi represent the values ​​of the two color histograms in the i-th color interval, N represents the dimension of the color histogram, and the results are normalized. The contrast, energy, entropy, and uniformity parameters in the texture features of the target object image data and the texture features of the target object mesh are calculated as feature vectors. The cosine similarity between the two feature vectors is calculated as the texture feature similarity, and the results are normalized. The color histogram feature similarity and texture feature similarity are weighted and averaged to select the image data with the highest similarity to the target object, which are then used as reference information.

8. The image information intelligent analysis system based on deep learning according to claim 2, characterized in that, Collect expert diagnostic results and optimize the trained convolutional neural network model based on these results, including the following steps: Collect expert diagnostic results, statistically analyze the correct proportion of target object category recognition results of the fusion model, and the correct proportion of target object category recognition results of the convolutional neural network models trained on the reference grid, boundary grid and internal grid in the weighted fusion layer. If the accuracy of the convolutional neural network model trained on the reference grid is less than 80%, and less than the accuracy of the object category recognition results of the convolutional neural network models trained on the boundary grid and the inner grid respectively, the reference weight of the convolutional neural network model trained on the reference grid is reduced by 1% until the reference weight is reduced to 8%, at which point the reference weight will no longer be reduced; otherwise, the reference weight of the convolutional neural network model trained on the reference grid will not be changed. In this process, after the reference weights of the convolutional neural network model trained on the reference grid are changed, the reference weights of the convolutional neural network models trained on the boundary grid and the inner grid are recalculated by the target recognition module.

Images