A method and device for segmenting target object image region based on image recognition

Abstract

The present invention discloses a method and device for segmenting target object image regions based on image recognition, the method comprising: obtaining collected aerial images of pine forests; obtaining a processed image of at least two mixed regions based on the collected images after cutting; training a first target recognition model with the processed images as training samples; constructing an initial second target recognition model based on the trained first feature extraction module and the untrained second region detection module and training the second target recognition model with the collected images as training samples; performing pine wilt disease region boundary segmentation based on the image region and the neighborhood region image of the preliminary position of the pine wilt disease region identified by the second target recognition model, and obtaining the pine wilt disease region boundary. The present invention improves the recognition accuracy of the pine wilt disease tree region and reduces the phenomenon of misidentification or missed recognition of the pine wilt disease tree region.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and device for segmenting a target object image region based on image recognition. Background Art

[0002] Pine wilt disease is a highly destructive forest disease caused by pine wood nematodes, which can cause serious damage to the ecological balance. In order to effectively protect the healthy growth of trees, it is necessary to conduct timely and accurate detection of pine wilt disease on trees, so as to accurately treat trees with pine wilt disease. To this end, it is necessary to use AI technology to identify and treat pine wilt disease.

[0003] The prior art has the following deficiencies:

[0004] The existing pine wilt disease identification system is based on visual recognition technology, but the collected images are outdoor environment images, which reduces the accuracy of pine wilt disease identification based on the collected outdoor pine forest images. It is easy to miss and misidentify pine wilt disease trees, resulting in a decrease in the recognition accuracy of pine wilt disease and an inability to obtain accurate pine wilt disease tree areas, so that there are errors in the results of pine wilt disease tree area identification, which is not conducive to the forestry department's management and protection operations on pine wilt disease trees. Summary of the invention

[0005] In view of the problems existing in the above-mentioned prior art, the present invention provides a method and device for segmenting target object image regions based on image recognition. The technical solution is as follows:

[0006] In a first aspect, a method for segmenting a target object image region based on image recognition is provided, comprising the following steps:

[0007] Access to collected aerial images of pine forests;

[0008] Based on the collected image, a processed image of at least two areas, namely, a mixed healthy tree area, a pine wood nematode disease area, and a background area, is obtained by cutting;

[0009] Using the processed images as training samples, training a first target recognition model for identifying the type and position of a target to be identified in each processed image, wherein the target to be identified is one of a healthy tree area, a pine wood nematode disease area, and a background area, and the first target recognition model includes a first feature extraction module and a first area recognition module;

[0010] Building an initial second target recognition model based on the trained first feature extraction module and the untrained second region detection module, wherein the second region detection module and the first region recognition module process the input image differently;

[0011] The collected image is used as a training sample to input an initial second target recognition model, and network parameter training is performed on a second region detection module in the initial second target recognition model to obtain a trained second target recognition model; the second target recognition model is used to identify the preliminary positions of healthy tree areas, pine wood nematode disease areas, and background areas in the collected image;

[0012] Based on the regional image of the preliminary position of the pine wilt disease area and the neighboring regional image, the boundary of the pine wilt disease area is segmented to obtain the boundary of the pine wilt disease area.

[0013] In some embodiments, the method further comprises:

[0014] The area of ​​the pine wilt disease area is determined based on the identified boundaries of the pine wilt disease area, and the degree of damage caused by the pine wilt disease is determined based on the ratio of the area of ​​the pine wilt disease area to the area of ​​healthy trees.

[0015] In some embodiments, in the target recognition results, the pine wood nematode disease area includes three categories, early diseased pine tree images, mid-stage diseased pine tree images, and late-stage diseased pine tree images; among which the early diseased pine tree images are most similar to normal healthy pine tree images.

[0016] In some implementations, constructing an initial second target recognition model based on a trained first feature extraction module and an untrained second region detection module includes:

[0017] The second area detection module includes preset healthy tree area scanning feature maps, pine wood nematode disease area scanning feature maps, and background area scanning feature maps, which are used to perform scanning operations with the feature maps output by the first feature extraction module, and output scanning results, which include the distribution positions of healthy tree areas, pine wood nematode disease areas, and background areas.

[0018] In some embodiments, the method for the second object recognition model to identify preliminary locations of healthy tree areas, pine wood nematode disease areas, and background areas in a captured image includes:

[0019] Inputting the collected image into the first feature extraction module to extract features and obtain a multi-scale feature map;

[0020] Based on the multi-scale feature map input into the second region detection module, image region detection is performed on each scale feature map respectively to determine the number and position distribution data of healthy tree areas, pine wood nematode disease areas, and background areas in the collected image;

[0021] The image recognition accuracy is determined based on the error between the number of identified different types of regions and the number of different types of regions in the image prior knowledge, and the error between the position distribution of different types of regions and the position distribution of different types of regions in the image prior knowledge;

[0022] Iteratively training the parameters of the second region detection module based on the image recognition accuracy until the image recognition accuracy reaches a preset threshold;

[0023] The trained second target recognition model is input into the collected image to be recognized to obtain the healthy tree area, pine wood nematode disease area, background area and corresponding positions in the collected image.

[0024] In some implementations, the inputting the second region detection module based on the multi-scale feature map to perform image region detection on each scale feature map respectively includes:

[0025] For each scale feature map, a healthy tree area scanning feature map, a pine wood nematode area scanning feature map, and a background area scanning feature map are respectively used to perform scanning operations with the scale feature map to obtain a healthy tree area distribution map, a pine wood nematode area distribution map, and a background area distribution map corresponding to the scale feature map;

[0026] The distribution maps of healthy tree areas, pine wilt disease areas, and background areas identified by multiple scale feature maps are fused to obtain the distribution information of healthy tree areas, pine wilt disease areas, and background areas corresponding to the collected images.

[0027] In some embodiments, the number of channels of the healthy tree area scanning feature map, the pine wilt disease area scanning feature map, and the background area scanning feature map is the same as the number of channels of the scale feature map, and the parameters of each pixel of the healthy tree area scanning feature map, the pine wilt disease area scanning feature map, and the background area scanning feature map are obtained through training, the pine wilt disease area scanning feature map represents the complete pixel feature distribution information of the pine wilt disease area, the healthy tree area scanning feature map represents the complete pixel feature distribution information of the healthy tree area, and the background area scanning feature map represents the complete pixel feature distribution information of the healthy tree area;

[0028] The parameters of each pixel of the trained healthy tree area scanning feature map, pine wood nematode disease area scanning feature map, and background area scanning feature map make the pixel value belonging to the healthy tree area on the feature map after the scanning operation of the healthy tree area scanning feature map and the scale feature map is within a first preset range, the pixel value belonging to the pine wood nematode disease area on the feature map after the scanning operation of the pine wood nematode disease area scanning feature map and the scale feature map is within a second preset range, and the pixel value belonging to the background area on the feature map after the scanning operation of the background area scanning feature map and the scale feature map is within a third preset range, and the first preset range, the second preset range, and the third preset range are the same or different.

[0029] In a second aspect, a device for segmenting a target object image region based on image recognition is provided, comprising:

[0030] An image acquisition unit, used for acquiring the collected aerial images of the pine forest;

[0031] An image processing unit, used for obtaining processed images of at least two areas including a mixed healthy tree area, a pine wood nematode disease area, and a background area through cutting based on the collected image;

[0032] A first model training unit is used to train a first target recognition model using the processed image as a training sample, and is used to recognize the type and position of the target to be recognized in each processed image, wherein the target to be recognized is one of a healthy tree area, a pine wood nematode disease area, and a background area, and the first target recognition model includes a first feature extraction module and a first area recognition module;

[0033] A second model building unit is used to build an initial second target recognition model based on the trained first feature extraction module and the untrained second region detection module, wherein the second region detection module and the first region recognition module have different processing processes for the input image;

[0034] A second model training unit is used to input an initial second target recognition model using the collected image as a training sample, and to perform network parameter training on a second region detection module in the initial second target recognition model to obtain a trained second target recognition model; the second target recognition model is used to identify the preliminary positions of healthy tree areas, pine wood nematode disease areas, and background areas in the collected image;

[0035] The region segmentation unit is used to segment the boundary of the pine wilt disease area based on the regional image of the preliminary position of the pine wilt disease area and the neighboring regional image to obtain the boundary of the pine wilt disease area.

[0036] According to a third aspect, an electronic device is provided, the electronic device comprising:

[0037] processor;

[0038] a memory for storing processor-executable instructions;

[0039] The processor implements the target object image region segmentation method as described in the first aspect above by running the executable instructions.

[0040] In a fourth aspect, a computer-readable storage medium is provided, on which computer instructions are stored, and when the instructions are executed by a processor, the steps of the target object image region segmentation method as described in the first aspect are implemented.

[0041] The present invention provides a method and device for target object image region segmentation based on image recognition, which has the following beneficial effects: the present invention performs preliminary pine wilt disease area position identification based on a first target recognition model and a second target recognition model, and the identification result is mainly used to determine the approximate position range of all pine wilt disease areas in the image, and further performs pine wilt disease area boundary segmentation based on the image area and the neighborhood area image of the preliminary position of the pine wilt disease area to obtain the pine wilt disease area boundary, which is convenient for further determining the area of ​​the pine wilt disease area in the pine forest, and avoiding the false detection and missed detection problems caused by directly identifying the boundary of the pine wilt disease area. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a flowchart of a method for segmenting a target object image region based on image recognition provided in an embodiment of the present application;

[0043] Figure 2 It is a schematic flow chart of a method for the second target recognition model in an embodiment of the present application to recognize preliminary positions of healthy tree areas, pine wood nematode disease areas, and background areas in a captured image;

[0044] Figure 3 It is a structural schematic diagram of a target object image region segmentation device based on image recognition in an embodiment of the present application. DETAILED DESCRIPTION

[0045] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0046] See also Figure 1 , a target object image region segmentation method based on image recognition provided by an embodiment of the present application comprises the following steps:

[0047] Step 1, obtaining the collected aerial images of pine forests;

[0048] Step 2, obtaining processed images of at least two areas including a mixed healthy tree area, a pine wood nematode disease area, and a background area through cutting based on the collected image;

[0049] Step 3, using the processed image as a training sample, training a first target recognition model for identifying the type and position of the target to be identified in each processed image, the first target recognition model comprising a first feature extraction module and a first region recognition module, the target to be identified is one of a healthy tree region, a pine wilt disease region, and a background region, so that the first target recognition model has the ability to perform different encodings on different regions based on image features of the healthy tree region, image features of the pine wilt disease region, and image features of the background region, and the encoding has the ability to distinguish and identify the healthy tree region, the pine wilt disease region, and the background region, the size of the feature map for identifying different regions obtained after the first target recognition model extracts image features from the processed image is determined based on the minimum size of each region in the training sample, and the reduction multiple of the original size of the processed image and the size of the feature map is less than the minimum size of each region in the training sample;

[0050] Step 4, constructing an initial second target recognition model based on the trained first feature extraction module and the untrained second region detection module, wherein the second region detection module has a different processing process for the input image from the first region recognition module, and is a second region detection module obtained by improving the first region recognition module of the first target recognition model;

[0051] Step 5, using the collected image as a training sample to input an initial second target recognition model, and performing network parameter training on a second region detection module in the initial second target recognition model to obtain a trained second target recognition model; the second target recognition model is used to identify the initial positions of healthy tree areas, pine wood nematode disease areas, and background areas in the collected image;

[0052] Step 6: segment the boundary of the pine wilt disease area based on the image area of ​​the preliminary position of the pine wilt disease area and the neighboring area image to obtain the boundary of the pine wilt disease area.

[0053] In an embodiment of the present application, the collected image is cut into processed images containing at least two regions, preferably, the collected image is cut into processed images containing two of the regions, and a first target recognition model is trained. The first target recognition model can use a conventional target recognition algorithm, and the first target recognition model has a unit for extracting features from an input image and a unit for identifying and detecting the extracted feature map. The first target recognition model learns how to encode the two regions separately according to the image features of the regions based on a single processed image and makes the encoding results of the two regions have the ability to significantly distinguish the two regions. The first target recognition model learns how to distinguish and encode healthy tree regions, pine wilt disease regions, and background regions through multiple processed images containing two different regions. Then, the first feature extraction module trained by the first target model is used to construct an initial second target recognition model, and the module whose model parameters are not trained in the second target recognition model is trained using samples to obtain a trained second target recognition model, which can respectively identify the initial positions of healthy tree regions, pine wilt disease regions, and background regions in the collected image.

[0054] The first target recognition model learns how to distinguish and encode healthy tree areas, pine wood nematode disease areas, and background areas through processing multiple images containing two different areas. That is, the first feature extraction module of the trained first target recognition model has the ability to distinguish and encode three different areas respectively, and the first area recognition module of the trained first target recognition model has the ability to recognize the type of area to be recognized in the image.

[0055] The trained first target model can be widely used in models of various other tasks, for example, in a task model for single identification of healthy tree areas in an entire aerial image containing a variety of different areas, and in a task model for single identification of pine wood nematode disease areas in an entire aerial image containing a variety of different areas. It is only necessary to change the labeled data and re-produce the training samples to fine-tune the original first target model.

[0056] In the embodiment of the present application, in order to improve the efficiency of model detection and avoid missing detections in different areas, the ability of the first feature extraction module trained in the first target model to distinguish and encode healthy tree areas, pine wood nematode disease areas, and background areas is utilized to improve the first area recognition module at the back end of the first target recognition model, so that the improved first area recognition module, i.e., the second area detection module, has the ability to detect multiple different areas at the same time. During training, only the parameters of the second area detection module need to be trained, which effectively reduces the training time of the second target recognition model. Moreover, the parameters that need to be trained in the second area detection module provided in the embodiment of the present application are significantly less than the parameters of the synchronous multi-target recognition model in the prior art (described in detail later).

[0057] In the embodiment of the present application, a staged learning process is simulated. First, the image contains the region type and the image is simply processed to preliminarily learn the features of different regions. In the first target recognition model, there is a certain prior knowledge of the healthy tree region, the pine wood nematode disease region, and the background region. When further tasks are required, the local part of the first target recognition model can be improved as needed, and the improved local part can be trained using appropriate training samples (carrying labeled data suitable for the task). For the embodiment of the present application, not only the versatility of the first feature extraction module of the first target recognition model is improved, but also the training time of the second target recognition model is reduced.

[0058] It can be understood that the "processed image" in the above steps 2 and 3 is obtained by cutting the local image based on the aerial image of step 1, and the cut local image only contains image pixels of two areas. The "captured image" in the above steps 4 and 5 is based on retaining the information of all different areas in the aerial image of step 1. Of course, the "captured image" in steps 4 and 5 can be obtained based on the aerial image of step 1 through some image preprocessing, such as denoising, correction, etc.

[0059] In the embodiment of the present application, preliminary pine wilt disease area position identification is first performed based on the first target recognition model and the second target recognition model. The identification result is mainly used to determine the approximate position range of all pine wilt disease areas in the image. Further, the pine wilt disease area boundary segmentation is performed based on the regional image of the preliminary position of the pine wilt disease area and the neighborhood regional image to obtain the pine wilt disease area boundary, which is convenient for further determining the area of ​​the pine wilt disease area in the pine forest and avoiding the false detection and missed detection problems caused by directly identifying the boundary of the pine wilt disease area.

[0060] In one embodiment, the target object image region segmentation method based on image recognition also includes: determining the area of ​​the pine wilt disease area based on the identified pine wilt disease area boundary, and determining the degree of damage caused by the pine wilt disease based on the ratio of the pine wilt disease area to the area of ​​healthy trees.

[0061] In the embodiment of the present application, after the boundary of the pine wilt disease area is determined, the area of ​​the pine wilt disease area is calculated, and the degree of damage caused by the pine wilt disease can be further analyzed.

[0062] In one embodiment, in the target recognition results, the pine wood nematode disease area includes three categories, early diseased pine tree images, mid-stage diseased pine tree images, and late-stage diseased pine tree images; among which the early diseased pine tree images are most similar to normal healthy pine tree images.

[0063] In one embodiment, the above step 4, constructing an initial second target recognition model based on the trained first feature extraction module and the untrained second region detection module, includes:

[0064] The second area detection module includes preset healthy tree area scanning feature maps, pine wood nematode disease area scanning feature maps, and background area scanning feature maps, which are used to perform scanning operations with the feature maps output by the first feature extraction module, and output scanning results, which include the distribution positions of healthy tree areas, pine wood nematode disease areas, and background areas.

[0065] It should be noted that, in the embodiment of the present application, the most important parts of the second area detection module are the scanning feature map of the healthy tree area, the scanning feature map of the pine wood nematode disease area, and the scanning feature map of the background area, but the second area detection module also includes other data processing units, such as a unit for further processing the output results of the feature extraction module, which is used to facilitate subsequent scanning operations with the scanning feature map in the second area detection module, such as a data post-processing unit after the scanning operation on the scanning feature map, to facilitate the subsequent output of classification results and statistical results, etc.

[0066] In one embodiment, see Figure 2 In step 5, the method of the second target recognition model identifying the preliminary positions of the healthy tree area, the pine wood nematode disease area, and the background area in the collected image includes:

[0067] Step 421, input the training sample into the first feature extraction module to extract features and obtain a multi-scale feature map;

[0068] Step 422, based on the multi-scale feature map input to the second region detection module, image region detection is performed on each scale feature map respectively, and the number and position distribution data of healthy tree regions, pine wood nematode disease regions, and background regions in the collected image are determined;

[0069] Step 423, determining the image recognition accuracy based on the error between the number of identified different types of regions and the number of different types of regions in the image prior knowledge, and the error between the position distribution of different types of regions and the position distribution of different types of regions in the image prior knowledge;

[0070] Step 424, iteratively training the parameters of the second region detection module based on the image recognition accuracy until the image recognition accuracy reaches a preset threshold, and the training of the second region detection module is completed;

[0071] Step 425, input the trained second target recognition model (including the trained first feature extraction module and the trained second area detection module) for the collected image to be recognized, and obtain the healthy tree area, pine wood nematode disease area, background area and corresponding positions in the collected image.

[0072] In the embodiment of the present application, the image recognition accuracy is determined based on the error between the number of identified different types of regions and the number of different types of regions in the image prior knowledge, and the error between the position distribution of different types of regions and the position of different types of regions in the image prior knowledge. Furthermore, the error between the number of identified different types of regions and the number of different types of regions in the image prior knowledge is determined based on the positions of misidentified regions and missed regions in the actual recognition results compared with the image prior knowledge. In the embodiment of the present application, the missed detection error is taken into account, and the missed detection error is determined based on the error between the number of identified different types of regions and the number of different types of regions in the image prior knowledge. After iterative training, the model continuously reduces the missed detection rate. Specifically, based on the error in the number of healthy tree regions, the model determines the missed detection rate based on the error in the number of healthy tree regions. , the error of the number of pine wilt disease areas , Background area recognition number error , and the errors of different types of regional locations Determine the image recognition accuracy. In the embodiment of the present application, the goal of each iterative update process is to synchronize , , , It can be understood that each iterative update process has a constraint condition, that is, the sum of the number of healthy tree areas, the number of pine wilt disease areas, and the number of background areas identified is consistent with the image prior knowledge. Based on this added constraint condition combined with , , , Compared with only setting the regional position error , which accelerates the convergence of the model training process while reducing the missed detection rate of each area. () indicates about and The function of represents the actual number of identified healthy tree areas, Indicates the number of target identifications in the healthy tree area, () indicates about and The function of represents the actual number of identified pine wilt disease areas, Indicates the number of target identifications in the pine wood nematode disease area, Indicates about and The function of Indicates the actual number of recognized background areas, Indicates the number of target recognitions in the background area.

[0073] In an optional embodiment, after the above step 424, after obtaining the network parameters of the trained second region detection module, the network parameters of the first feature extraction module and the second region detection module in the second target recognition model can be fine-tuned based on the training samples.

[0074] In one implementation, the step 422, based on the multi-scale feature map input into the second region detection module, performs image region detection on each scale feature map respectively, including:

[0075] Step 4221, for each scale feature map, respectively use the healthy tree area scanning feature map, the pine wood nematode area scanning feature map, and the background area scanning feature map to perform scanning operations with the scale feature map to obtain the healthy tree area distribution map, the pine wood nematode area distribution map, and the background area distribution map corresponding to the scale feature map;

[0076] Step 4222, the healthy tree area distribution map, pine wilt disease area distribution map, and background area distribution map identified by multiple scale feature maps are merged to obtain the distribution information of the healthy tree area, pine wilt disease area, and background area corresponding to the collected image.

[0077] In the embodiment of the present application, since the regional sizes of the healthy tree area, the pine wilt disease area, and the background area are uncertain, different scale feature maps are used to identify the target area respectively, which are adapted to the healthy tree areas, the pine wilt disease areas, and the background areas of different sizes, thereby reducing the missed detection rate. In addition, when the healthy tree area distribution map, the pine wilt disease area distribution map, and the background area distribution map identified by multiple scale feature maps are fused, the location information of different areas in the healthy tree area distribution map, the pine wilt disease area distribution map, and the background area distribution map identified by all scale feature maps can be directly retained, and after regional duplicate analysis and deduplication, the distribution information of the healthy tree area, the pine wilt disease area, and the background area corresponding to the final collected image can be obtained.

[0078] In one embodiment, the number of channels of the healthy tree area scanning feature map, the pine wilt disease area scanning feature map, and the background area scanning feature map in the above step 4221 is the same as the number of channels of the scale feature map, and the parameters of each pixel of the healthy tree area scanning feature map, the pine wilt disease area scanning feature map, and the background area scanning feature map are obtained through training, and the pine wilt disease area scanning feature map represents the complete pixel feature distribution information of the pine wilt disease area, the healthy tree area scanning feature map represents the complete pixel feature distribution information of the healthy tree area, and the background area scanning feature map represents the complete pixel feature distribution information of the healthy tree area;

[0079] The parameters of each pixel of the trained healthy tree area scanning feature map, pine wood nematode disease area scanning feature map, and background area scanning feature map make the pixel value belonging to the healthy tree area on the feature map after the scanning operation of the healthy tree area scanning feature map and the scale feature map is within a first preset range, the pixel value belonging to the pine wood nematode disease area on the feature map after the scanning operation of the pine wood nematode disease area scanning feature map and the scale feature map is within a second preset range, and the pixel value belonging to the background area on the feature map after the scanning operation of the background area scanning feature map and the scale feature map is within a third preset range, and the first preset range, the second preset range, and the third preset range are the same or different.

[0080] In an embodiment of the present application, the second area detection module in the second target recognition uses the healthy tree area scanning feature map, the pine wood nematode disease area scanning feature map, and the background area scanning feature map to scan the feature map output by the feature extraction module to obtain recognition results of different areas.

[0081] In the target detection and recognition part of a conventional target recognition model, such as the head part of the YOLOV5 model, the feature extraction module outputs feature maps of three different scales. In the target recognition part, three prior boxes are generated for each grid on the feature map of each scale. For each grid, the probability of belonging to different categories, the coordinate position data of the prior box, and the confidence rate need to be predicted. The target recognition part requires m*m*n*3*3*C*5 convolution kernels. During the training process, the network parameters that need to be trained in the target recognition part are (m*m*n*3*3*C*5), where m*m is the number of grids in the feature map, n is the number of channels of the feature map of each scale, C is the number of categories to be detected, and "5" contains 4 data of the coordinate position of the prior box and 1 confidence rate data.

[0082] In the embodiment of the present application, the a priori frame generation algorithm of the conventional target recognition model is not used. Instead, the feature map of the healthy tree area scan, the feature map of the pine wood nematode area scan, and the feature map of the background area scan are used to scan the feature map output by the feature extraction module to determine whether there is a target in each window area on the feature map. If so, the window area on the feature map is determined to be the target area. In the present application, there is no need to predict each grid of each feature map separately, which improves the detection efficiency. Only the parameters of the scanned feature map need to be trained during the training process.

[0083] In one implementation, the scanning operation process in step 4221 includes:

[0084] Step 42211, sliding the pine wood nematode disease area scanning feature map on the scale feature map to obtain a plurality of sliding window maps of the same size as the pine wood nematode disease area scanning feature map;

[0085] Step 42212, analyzing the similarity of image feature distribution with respect to each sliding window image and the pine wilt disease area scanning feature image, and determining the position of the sliding window image with a similarity greater than a preset threshold as the pine wilt disease area on the scale feature image.

[0086] To avoid missed identification, the sliding step size can be set to 1. In addition, the current sliding step size can be determined based on the recognition result of the last sliding window image. For example, if the last sliding window was determined to be a non-pine wilt disease area, the current sliding step size is set to 1. If the last sliding window was determined to be a pine wilt disease area, the current sliding step size is set to a value greater than 1 and less than the width of the scanning feature map of the pine wilt disease area.

[0087] It should be noted that, in the embodiment of the present application, the similarity of the image feature distribution between the sliding window image and the pine wilt disease area scanning feature map is analyzed, rather than directly calculating the similarity of the pixels at the same position of the two images. The pine wilt disease area scanning feature map contains the pixel feature distribution information in the complete pine wilt disease area. Considering that the pine wilt disease area contained in the sliding window image may be inconsistent in size with the pine wilt disease area scanning feature map, it may be smaller than the pine wilt disease area scanning feature map, and may be larger than the pine wilt disease area scanning feature map, considering that the pine wilt disease area contained in the sliding window image may be only a local pine wilt disease area, so in the embodiment of the present application, consider analyzing the similarity of the image feature distribution between the sliding window image and the pine wilt disease area scanning feature map, if the image feature distribution in the sliding window image characterizes all or part of the pine wilt disease area, it should have similarity with the image feature distribution of all or part of the pine wilt disease area scanning feature map.

[0088] Among them, the image pixel parameters of the scanning feature map of the pine wood nematode disease area, the scanning feature map of the healthy tree area, and the scanning feature map of the background area are obtained based on training. The training process of the scanning feature map can be carried out synchronously with the training process of the network in the second target recognition model, or it can be carried out asynchronously.

[0089] In one embodiment, the training process of the scanning feature map is carried out synchronously with the training process of each network in the second target recognition model, and the training process includes: using the network parameters of the feature extraction module of the first target recognition model to be fixed, using the training samples, training the second region detection module, including the pixel parameters of each scanning feature map, until the image recognition accuracy reaches the preset conditions, and then using the training samples to synchronously fine-tune the feature extraction module and the second region detection module of the second target recognition model until the training is completed. In this embodiment, based on the training of a large number of samples, a large number of samples contain a large number of pine wood nematode disease area images. After training, the pine wood nematode disease area scanning feature map contains the complete pixel distribution feature information of the pine wood nematode disease area in all samples, the healthy tree area scanning feature map contains the complete pixel distribution feature information of the healthy tree area in all samples, and the background area scanning feature map contains the complete pixel distribution feature information of the background area in all samples.

[0090] In another embodiment, the training process of scanning feature maps includes:

[0091] Step a, based on the pine wilt disease area in the training sample image as the core area sample, forming a pine wilt disease area image;

[0092] Step b, taking the pine wilt disease area image as the center and filling the border with a fixed pixel value to form a pine wilt disease area sample image with a shape and size that meets the preset standard;

[0093] Step c, obtaining a healthy tree area sample image and a background area sample image based on the above steps a and b;

[0094] Step d, using the first feature extraction module of the second target recognition model as a basic network, training the basic network based on sample images of pine wood nematode disease areas, so that the trained basic network has the greatest difference in encoding results for sample images of different types of areas and the greatest similarity in encoding results for sample images of the same type of areas;

[0095] Step e: obtaining the scanning feature map of the pine wood nematode disease area, the scanning feature map of the healthy tree area, and the scanning feature map of the background area based on the trained basic network.

[0096] Furthermore, in this embodiment, the parameter fine-tuning of the first feature extraction module in the second target recognition model and the basic network can be trained separately or alternately. In the case of alternating training, the network parameters of the first feature extraction module in the second target recognition model can be fine-tuned and optimized in combination with the existing first feature extraction module parameters in the second target recognition model and the network parameters of the most recently updated basic network, and the network parameters of the basic network can be optimized in combination with the network parameters of the previously existing basic network and the first feature extraction module parameters of the second target recognition model most recently updated.

[0097] In one embodiment, the above step 5, segmenting the boundary of the pine wilt disease area based on the image area of ​​the preliminary position of the pine wilt disease area and the neighboring area image, and obtaining the boundary of the pine wilt disease area, includes:

[0098] Step 51, using the image area based on the preliminary position of the pine wood nematode disease area and the neighboring area image as the target local image to be analyzed;

[0099] Step 52, performing edge pixel detection based on the target local image to obtain edge pixels of the pine wood nematode disease area, wherein the process of performing edge pixel detection can be implemented based on a variety of edge detection algorithms;

[0100] Step 53: segment the boundary of the pine wood nematode diseased area based on the edge pixels in the target local image.

[0101] See also Figure 3 Based on the above method embodiment, the present application embodiment provides a device for segmenting target object image regions based on image recognition, including:

[0102] An image acquisition unit, used for acquiring the collected aerial images of the pine forest;

[0103] An image processing unit, used for obtaining processed images of at least two areas including a mixed healthy tree area, a pine wood nematode disease area, and a background area through cutting based on the collected image;

[0104] A first model training unit is used to train a first target recognition model using a processed image as a training sample, and is used to identify the type and position of a target to be identified in each processed image, wherein the target to be identified is one of a healthy tree area, a pine wilt disease area, and a background area. The first target recognition model includes a first feature extraction module and a first area recognition module, so that the first target recognition model has the ability to perform different encodings on different areas based on image features of healthy tree areas, image features of pine wilt disease areas, and image features of background areas, and the encoding has the ability to distinguish and identify healthy tree areas, pine wilt disease areas, and background areas. The size of a feature map for identifying different areas obtained by the first target recognition model after performing image feature extraction on the processed image is determined based on the minimum size of each area in the training sample, and the reduction multiple of the original size of the processed image and the size of the feature map is less than the minimum size of each area in the training sample;

[0105] A second model building unit is used to build an initial second target recognition model based on the trained first feature extraction module and the untrained second region detection module, wherein the second region detection module and the first region recognition module have different processing processes for the input image;

[0106] A second model training unit is used to input an initial second target recognition model using the collected image as a training sample, and to perform network parameter training on a second region detection module in the initial second target recognition model to obtain a trained second target recognition model; the second target recognition model is used to identify the preliminary positions of healthy tree areas, pine wood nematode disease areas, and background areas in the collected image;

[0107] The region segmentation unit is used to segment the boundary of the pine wilt disease area based on the regional image of the preliminary position of the pine wilt disease area and the neighboring regional image to obtain the boundary of the pine wilt disease area.

[0108] For the specific definition of the target object image region segmentation device based on image recognition, please refer to the definition of the target object image region segmentation method based on image recognition above, which will not be repeated here. Each unit in the above-mentioned target object image region segmentation device based on image recognition can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned units can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned units.

[0109] An embodiment of the present application provides an electronic device, the electronic device comprising:

[0110] processor;

[0111] a memory for storing processor-executable instructions;

[0112] The processor implements the target object image region segmentation method described in the above method embodiment by running the executable instructions.

[0113] In some embodiments, the electronic device may also optionally include: an input interface and an output interface. The processor, the memory and the input interface and the output interface may be connected via a bus or a signal line. Each peripheral device may be connected to the input interface and the output interface via a bus, a signal line or a circuit board. The input interface and the output interface may be used to connect at least one peripheral device related to input / output to the processor and the memory.

[0114] The embodiment of the present application provides a computer-readable storage medium on which computer instructions are stored, and when the instructions are executed by a processor, the steps of the target object image region segmentation method described in the above method embodiment are implemented. The computer-readable storage medium includes: an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, and any suitable combination thereof.

[0115] The present invention is not limited to the above-mentioned specific implementation modes. Various changes made by ordinary technicians in this field based on the above-mentioned concepts without creative work are all within the protection scope of the present invention.

Claims

1. A method for segmenting target object image regions based on image recognition, characterized in that: The steps include: Access to collected aerial images of pine forests; Based on the collected image, a processed image of at least two areas, namely, a mixed healthy tree area, a pine wood nematode disease area, and a background area, is obtained by cutting; Using processed images as training samples, a first target recognition model is trained to identify the type and position of a target to be recognized in each processed image, wherein the target to be recognized is one of a healthy tree area, a pine wilt disease area, and a background area. The first target recognition model includes a first feature extraction module and a first area recognition module. In the first target recognition model, certain prior knowledge of the healthy tree area, the pine wilt disease area, and the background area is obtained. When further tasks are performed later, a part of the first target recognition model is improved as needed, and the improved part is trained using appropriate training samples. Building an initial second target recognition model based on the trained first feature extraction module and the untrained second region detection module, wherein the second region detection module and the first region recognition module process the input image differently; The collected image is used as a training sample to input an initial second target recognition model, and the network parameters of the second region detection module in the initial second target recognition model are trained to obtain a trained second target recognition model; the second target recognition model is used to identify the initial positions of healthy tree areas, pine wood nematode disease areas, and background areas in the collected image. The method for the second target recognition model to identify the initial positions of healthy tree areas, pine wood nematode disease areas, and background areas in the collected image includes: inputting the collected image into a first feature extraction module to extract features and obtain a multi-scale feature map; Based on the multi-scale feature map input, the second region detection module is input, and image region detection is performed on each scale feature map respectively, to determine the number and position distribution data of healthy tree regions, pine wilt disease regions, and background regions in the collected image; the image recognition accuracy is determined based on the error between the number of identified different types of regions and the number of different types of regions in the image prior knowledge, and the error between the position distribution of different types of regions and the position of different types of regions in the image prior knowledge; the parameters of the second region detection module are iteratively trained based on the image recognition accuracy until the image recognition accuracy reaches a preset threshold; the trained second target recognition model is input for the collected image to be recognized, and the healthy tree region, pine wilt disease region, background region and the corresponding position in the collected image are obtained; Based on the regional image of the preliminary position of the pine wilt disease area and the neighboring regional image, the boundary of the pine wilt disease area is segmented to obtain the boundary of the pine wilt disease area.

2. The target object image region segmentation method based on image recognition according to claim 1, characterized in that: The method further comprises: The area of ​​the pine wilt disease area is determined based on the identified boundaries of the pine wilt disease area, and the degree of damage caused by the pine wilt disease is determined based on the ratio of the area of ​​the pine wilt disease area to the area of ​​healthy trees.

3. The target object image region segmentation method based on image recognition according to claim 1, characterized in that: In the target recognition results, the pine wood nematode disease area includes three categories: early-stage diseased pine tree images, mid-stage diseased pine tree images, and late-stage diseased pine tree images; among them, the early-stage diseased pine tree images are the most similar to the normal healthy pine tree images.

4. The target object image region segmentation method based on image recognition according to claim 1, characterized in that: The initial second target recognition model is constructed based on the trained first feature extraction module and the untrained second region detection module, including: The second area detection module includes preset healthy tree area scanning feature maps, pine wood nematode disease area scanning feature maps, and background area scanning feature maps, which are used to perform scanning operations with the feature maps output by the first feature extraction module, and output scanning results, which include the distribution positions of healthy tree areas, pine wood nematode disease areas, and background areas.

5. The target object image region segmentation method based on image recognition according to claim 4, characterized in that: The inputting of the second region detection module based on the multi-scale feature map to perform image region detection on each scale feature map respectively includes: For each scale feature map, a healthy tree area scanning feature map, a pine wood nematode area scanning feature map, and a background area scanning feature map are respectively used to perform scanning operations with the scale feature map to obtain a healthy tree area distribution map, a pine wood nematode area distribution map, and a background area distribution map corresponding to the scale feature map; The distribution maps of healthy tree areas, pine wilt disease areas, and background areas identified by multiple scale feature maps are fused to obtain the distribution information of healthy tree areas, pine wilt disease areas, and background areas corresponding to the collected images.

6. The target object image region segmentation method based on image recognition according to claim 5, characterized in that: The number of channels of the healthy tree area scanning feature map, the pine wilt disease area scanning feature map, and the background area scanning feature map is the same as the number of channels of the scale feature map, and the parameters of each pixel of the healthy tree area scanning feature map, the pine wilt disease area scanning feature map, and the background area scanning feature map are obtained through training, the pine wilt disease area scanning feature map represents the complete pixel feature distribution information of the pine wilt disease area, the healthy tree area scanning feature map represents the complete pixel feature distribution information of the healthy tree area, and the background area scanning feature map represents the complete pixel feature distribution information of the healthy tree area; The parameters of each pixel of the trained healthy tree area scanning feature map, pine wilt disease area scanning feature map, and background area scanning feature map make the pixel values ​​belonging to the healthy tree area on the feature map after the scanning operation of the healthy tree area scanning feature map and the scale feature map is within a first preset range, the pixel values ​​belonging to the pine wilt disease area on the feature map after the scanning operation of the pine wilt disease area scanning feature map and the scale feature map is within a second preset range, and the pixel values ​​belonging to the background area on the feature map after the scanning operation of the background area scanning feature map and the scale feature map is within a third preset range.

7. A target object image region segmentation device based on image recognition, characterized in that: include: An image acquisition unit, used for acquiring the collected aerial images of the pine forest; An image processing unit, used for obtaining processed images of at least two areas including a mixed healthy tree area, a pine wood nematode disease area, and a background area through cutting based on the collected image; A first model training unit is used to train a first target recognition model using a processed image as a training sample, and is used to recognize the type and position of a target to be recognized in each processed image, wherein the target to be recognized is one of a healthy tree area, a pine wilt disease area, and a background area. The first target recognition model includes a first feature extraction module and a first area recognition module. In the first target recognition model, certain prior knowledge is obtained about the healthy tree area, the pine wilt disease area, and the background area. When further tasks are performed subsequently, a part of the first target recognition model is improved as needed, and the improved part is trained using a suitable training sample; A second model building unit is used to build an initial second target recognition model based on the trained first feature extraction module and the untrained second region detection module, wherein the second region detection module and the first region recognition module have different processing processes for the input image; The second model training unit is used to input an initial second target recognition model using the collected image as a training sample, and perform network parameter training on a second region detection module in the initial second target recognition model to obtain a trained second target recognition model; the second target recognition model is used to identify the initial positions of healthy tree regions, pine wood nematode disease regions, and background regions in the collected image. The method for the second target recognition model to identify the initial positions of healthy tree regions, pine wood nematode disease regions, and background regions in the collected image includes: inputting the collected image into a first feature extraction module for feature extraction to obtain a multi-scale feature map; inputting the multi-scale feature map into a second region detection module, respectively Perform image region detection on each scale feature map to determine the number and position distribution data of healthy tree areas, pine wilt disease areas, and background areas in the collected image; determine the image recognition accuracy based on the error between the number of identified different types of areas and the number of different types of areas in the image prior knowledge, and the error between the position distribution of different types of areas and the position of different types of areas in the image prior knowledge; iteratively train the parameters of the second area detection module based on the image recognition accuracy until the image recognition accuracy reaches a preset threshold; input the trained second target recognition model into the collected image to be recognized, and obtain the healthy tree area, pine wilt disease area, background area and the corresponding position in the collected image; The region segmentation unit is used to segment the boundary of the pine wilt disease area based on the regional image of the preliminary position of the pine wilt disease area and the neighboring regional image to obtain the boundary of the pine wilt disease area.

8. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing processor-executable instructions; The processor implements the target object image region segmentation method according to any one of claims 1 to 6 by running the executable instructions.

9. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instructions are executed by the processor, the steps of the target object image region segmentation method as described in any one of claims 1 to 6 are implemented.

Images