Region boundary recognition method and device based on model optimization iteration

Abstract

The embodiment of the present invention discloses a method and device for identifying region boundaries based on model optimization iteration, the method comprising: inputting a first image into a first training model, outputting a first scene feature vector corresponding to the first image through the first training model; determining whether there is a corresponding recognition scene for the first image according to the first scene feature vector and scene clustering information; if there is no corresponding recognition scene, adding the first image to a first sample training set to obtain a second sample training set, inputting the second sample training set into an original neural network model, and training the original neural network model for identifying region boundaries in an image. This solution realizes efficient and accurate recognition of image region boundaries.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing, and in particular to a method and device for region boundary recognition based on model optimization iteration. Background Art

[0002] As one of the important technologies in the field of image processing, the recognition of regional boundaries contained in images is widely used in various scenarios. For example, in the field of agricultural applications, the regional boundaries of captured images are recognized to determine the farmland areas and non-farmland areas contained in the images, which can be used for subsequent high-altitude operation processing of unmanned equipment.

[0003] In the prior art, most of the image semantic segmentation methods based on deep learning are used for model training. The model obtained through training can segment the input image into different regions, and then use the boundary search algorithm for each region to obtain the region boundary. However, the defect of the above method is that in complex and diverse scenes, the model generalization ability is insufficient, resulting in low accuracy and poor effect of region boundary recognition results, and the overall boundary recognition efficiency is not high. Summary of the invention

[0004] The embodiments of the present invention provide a method and device for region boundary recognition based on model optimization iteration, which solves the problems of low precision and poor effect of region boundary recognition results and low overall boundary recognition efficiency in the prior art, and realizes efficient and accurate recognition of image region boundaries.

[0005] In a first aspect, an embodiment of the present invention provides a method for identifying region boundaries based on model optimization iteration, the method comprising:

[0006] Inputting a first image into a first training model, and outputting a first scene feature vector corresponding to the first image through the first training model;

[0007] Determining whether the first image has a corresponding recognition scene according to the first scene feature vector and scene clustering information, wherein the scene clustering information includes one or more cluster centers, wherein each cluster center corresponds to a recognition scene;

[0008] If the corresponding recognition scene does not exist, the first image is added to the first sample training set to obtain a second sample training set, the second sample training set is input into the original neural network model, and the original neural network model is trained to identify the region boundaries in the image, wherein the first training model is a model obtained by modifying and training the original neural network model.

[0009] In a second aspect, an embodiment of the present invention further provides a region boundary identification device based on model optimization iteration, the device comprising:

[0010] An image input module, used for inputting a first image into a first training model;

[0011] A feature vector output module, configured to output a first scene feature vector corresponding to the first image through the first training model;

[0012] a scene recognition module, configured to determine whether the first image has a corresponding recognition scene according to the first scene feature vector and scene clustering information, wherein the scene clustering information includes one or more cluster centers, wherein each cluster center corresponds to a recognition scene;

[0013] A sample adding module, configured to add the first image to the first sample training set to obtain a second sample training set if the corresponding recognition scene does not exist;

[0014] A network model training module is used to input the second sample training set into the original neural network model, and train the original neural network model to identify region boundaries in the image, wherein the first training model is a model obtained by modifying and training the original neural network model.

[0015] In a third aspect, an embodiment of the present invention further provides a device for identifying region boundaries based on model optimization iteration, the device comprising:

[0016] Two or more processors; a storage device for storing two or more programs, when the two or more programs are executed by the two or more processors, the two or more processors implement the region boundary recognition method based on model optimization iteration described in an embodiment of the present invention.

[0017] In a fourth aspect, an embodiment of the present invention further provides a storage medium storing computer executable instructions, which, when executed by a computer processor, are used to execute the region boundary identification method based on model optimization iteration described in an embodiment of the present invention.

[0018] In an embodiment of the present invention, a first image is input into a first training model to output a first scene feature vector corresponding to the first image, and then it is determined whether the first image has a corresponding recognition scene based on the first scene feature vector and scene clustering information. If not, the first image is added to the first sample training set to obtain a second sample training set, and the second sample training set is input into the original neural network model for training, and the trained neural network model is used to recognize region boundaries in the image. The region boundary recognition method based on model optimization iteration provided by this solution can quickly identify training scenes that do not exist in the original training set, and optimize and iterate the neural network model, solving the problems of weak scene generalization ability of the existing neural network model used for region boundary recognition, poor recognition effect and low recognition accuracy for newly added scene region boundaries, improving the accuracy of boundary recognition of regions contained in the image, and significantly improving the recognition effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A flowchart of a method for identifying region boundaries based on model optimization iteration provided by an embodiment of the present invention;

[0020] Figure 2 A flowchart of another method for identifying region boundaries based on model optimization iteration provided by an embodiment of the present invention;

[0021] Figure 3 A flowchart of another method for identifying region boundaries based on model optimization iteration provided by an embodiment of the present invention;

[0022] Figure 4 A flowchart of another method for identifying region boundaries based on model optimization iteration provided by an embodiment of the present invention;

[0023] Figure 5 A flowchart of another method for identifying region boundaries based on model optimization iteration provided by an embodiment of the present invention;

[0024] Figure 6 A flow chart for determining a scene feature vector provided by an embodiment of the present invention;

[0025] Figure 7 A flowchart of training an original neural network model provided by an embodiment of the present invention;

[0026] Figure 8 A flowchart of another method for identifying region boundaries based on model optimization iteration provided by an embodiment of the present invention;

[0027] Fig. 9 A structural block diagram of a region boundary identification device based on model optimization iteration provided by an embodiment of the present invention;

[0028] Fig.10 A schematic structural diagram of a region boundary recognition device based on model optimization iteration provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0030] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0031] The information adjustment method provided in the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0032] Figure 1 A flowchart of a method for identifying region boundaries based on model optimization iteration provided in an embodiment of the present invention. This embodiment can be applied to optimize a neural network model for region boundary identification to identify the boundaries of each region contained in an image. The method can be executed by computing devices such as unmanned devices, handheld devices, smart mobile devices, smart cars, smart helmets, smart glasses, tablet computers, desktop computers, notebooks, and other devices with data processing functions. Specifically, the following steps are included:

[0033] Step S101: input a first image into a first training model, and output a first scene feature vector corresponding to the first image through the first training model.

[0034] Among them, the first training model is a model obtained by modifying and training the original neural network model. Exemplarily, the original neural network model can be a neural network segmentation model based on deep learning, through which the region boundary of the region contained in the input image can be obtained. Specifically, the input image can be divided into different regions by the neural network segmentation model, and then the region boundary is obtained by using a boundary search algorithm for each region. The original neural network model can be a model used in the prior art for image region boundary recognition, which has poor scene generalization ability and cannot adapt to image region boundary recognition in complex scenes.

[0035] The first training model in this embodiment is a model that is modified and trained based on an existing neural network model for image region recognition. The specific modification process and training process can be found in the subsequent explanations. This embodiment is mainly used to explain the process of iteratively optimizing the original neural network model to identify image region boundaries.

[0036] In one embodiment, the first image is a new image that is not used for the original neural network model training. The first training model includes a scene feature layer, and the scene feature layer is used to output a scene feature vector corresponding to the input image, and in this step, the first scene feature vector corresponding to the first image is output.

[0037] Among them, the first scene feature vector is used to characterize the specific scene of the first image. In one embodiment, taking the farmland area as an example, the first image is a high-precision image obtained by photographing the farmland area. The farmland area itself has a variety of different scenes, and the farmland area morphology in different scenes is quite different. Exemplarily, the farmland area scene can be a field in the green seedling period, a paddy field, a field in the harvest period, a loess field, a black soil field, a fruit forest, a tea garden, etc. The first scene feature vector represents the scene corresponding to the first image, and the corresponding scene can be a scene that has been identified during the training of the first training model and a new scene that has not been identified.

[0038] Step S102: determining whether the first image has a corresponding recognition scene according to the first scene feature vector and scene clustering information.

[0039] The scene clustering information includes one or more cluster centers, each of which corresponds to an identification scene. Taking the aforementioned farmland area as an example, it is assumed that the scene category information includes seven cluster centers, each of which corresponds to an identification scene, exemplarily corresponding to green seedling fields, paddy fields, harvest fields, loess fields, black soil fields, fruit forests, and tea gardens. The identification scene in the scene clustering information can be a manually defined scene category, or it can be a scene category obtained by clustering the scene feature vectors output during the training of the modified original neural network model.

[0040] In one embodiment, each cluster center included in the scene clustering information corresponds to a central scene feature vector, and the first scene feature vector is compared with each central scene feature vector to determine whether there is a corresponding recognition scene. Specifically, the Euler distance or cosine distance between the first scene feature vector and each central scene feature vector may be calculated. If the distance between the central scene feature vector corresponding to a certain cluster center is less than a set distance threshold, it is determined that the first image has a corresponding recognition scene. Conversely, if the distance between the first scene feature vector and each central scene feature vector is not less than the set distance threshold, it is determined that the first image does not have a corresponding recognition scene.

[0041] Step S103: if the corresponding recognition scene does not exist, then add the first image to the first sample training set to obtain a second sample training set, input the second sample training set to the original neural network model, and train the original neural network model.

[0042] In one embodiment, the first training sample set is the training sample set corresponding to the first training model obtained by training. That is, the first training model is obtained by training the modified original neural network model with the first training sample set. When it is determined in step S102 that the first image does not have a corresponding recognition scene, the first image is added to the first sample training set to obtain a second sample training set, and the original neural network model is trained with the second sample training set to use the trained neural network model for image region boundary recognition. The specific training method of training the original neural network model with the second sample training set in this step is not limited, and the conventional training method in the prior art can be used.

[0043] It can be seen from the above scheme that the existing original neural network model has insufficient scene generalization ability, and adding all the current image corresponding scenes to the training requires a lot of screening and labeling work. Through the optimization and iteration method of the original neural network model given in this scheme, the regional boundary recognition accuracy of the neural network model used for regional boundary recognition is improved. In this scheme, after the first image is input into the first training model to output the first scene feature vector, it is determined whether there is a recognition scene corresponding to the first image according to the first scene feature vector. If not, the first image is added to the sample training set to retrain the original neural network model. After the training is completed, the image can be input again, such as inputting the second image to determine whether there is a corresponding recognition scene for the second image. If not, the second image is added to the sample training set to which the first image has been added before to train the original neural network model that has been trained again, and so on, so that the neural network model finally trained can recognize the regional boundaries contained in images under various different scenes without a lot of manual screening and labeling work, and at the same time, it realizes efficient and accurate recognition of image regional boundaries.

[0044] Figure 2 The flowchart of another method for identifying region boundaries based on model optimization iteration provided by the embodiment of the present invention provides a specific method for modifying the original neural network model and training to obtain a first training model, and at the same time obtaining clustering information of clustering scenes. Figure 2 As shown, the technical solution is as follows:

[0045] Step S201, adding a scene feature layer to the original neural network model to obtain a first neural network model.

[0046] The original neural network model is an existing neural network segmentation model based on deep learning, which includes the structure of a conventional neural network model such as an encoder and a decoder. In one embodiment, after the scene feature layer is added to the downsampled encoder in the original neural network model, the scene feature layer is a fully connected layer, and the feature vector output by the scene feature layer is defined as a scene feature vector for characterizing the scene corresponding to the image.

[0047] Step S202: input the first sample training set into the first neural network model, train the first neural network model to obtain a first training model, and the scene feature layer is used to output a scene feature vector corresponding to each sample image in the first sample training set.

[0048] In one embodiment, a first sample training set is input into the first neural network model, and the method of training the first neural network model includes inputting sample image pairs into the first neural network model for training, so that when the trained first training model outputs a scene feature vector for a subsequently input image, if the scene corresponding to the first image already exists in the sample training set, the output scene feature vector is similar to a certain recognized scene that has been obtained; conversely, if the scene corresponding to the first image does not exist in the sample training set, the output scene feature vector has a large distance from the central feature vector corresponding to each recognized scene.

[0049] Step S203: performing a clustering operation on the scene feature vector output by the first neural network model to obtain scene clustering information.

[0050] In one embodiment, the first neural network model may perform a clustering operation on the scene feature vector output by the first neural network model during the process of training the first training model to obtain the scene clustering information. Alternatively, after the first neural network model is trained to obtain the first training model, the sample training set is input into the scene feature vector output by the first training model to perform a clustering operation. The output scene feature vector is clustered by clustering operation to obtain one or more cluster centers, and each cluster center corresponds to a recognition scene. The specific clustering algorithm is not limited here, and a conventional clustering algorithm in the prior art may be used.

[0051] It can be seen from the above scheme that by modifying the original neural network model and adding a scene feature layer, a modified first neural network model is obtained, and in the process of training the modified first neural network model, a clustering operation is performed on the output scene feature vector to obtain clustering information. Subsequently, the first training model obtained by the training and the clustering information are used to discriminate the newly input first image to determine whether it belongs to a known recognition scene. If not, it is determined that a new scene is recognized, and the original neural network model is retrained. Through multiple iterations, the scene generalization ability of the trained neural network model is significantly enhanced, and the regional boundaries of images in any scene can be accurately and efficiently recognized.

[0052] Figure 3 A flowchart of another method for identifying region boundaries based on model optimization iteration provided by an embodiment of the present invention, after modifying the original neural network model, adding a scene feature layer to obtain a first neural network model, before inputting a first sample training set into the first neural network model and training the first neural network model to obtain a first training model, also includes a process of pre-training the first neural network model. Figure 3 As shown, the technical solution is as follows:

[0053] Step S301, adding a scene feature layer to the original neural network model to obtain a first neural network model.

[0054] Step S302: Add a Softmax layer after the scene feature layer in the first neural network model to obtain a second neural network model, and perform back propagation training on the second neural network model to obtain a pre-trained network model.

[0055] In one embodiment, after the first neural network model is obtained, it is pre-trained. Specifically, a Softmax layer is added after the scene feature layer in the first neural network model to obtain a second neural network model. Among them, the Softmax layer can be understood as a normalization layer. For example, if the recognition scene of the current corresponding sample training set is 7, then the output of the Softmax layer is a seven-dimensional vector. The first value in the vector is the probability that the current sample image in the sample training set belongs to the first type of recognition scene, and the second value in the vector is the probability that the current sample image in the sample training set belongs to the second type of recognition scene, and so on. After adding the Softmax layer, the second neural network model is back-propagated to obtain a pre-trained network model, and then the pre-trained network model is trained using the images in the first sample training set to obtain the first training model.

[0056] Step S303: input the first sample training set into the pre-trained network model, and train the pre-trained network model to obtain a first training model.

[0057] Step S304: performing a clustering operation on the scene feature vector output by the pre-trained network model to obtain scene clustering information.

[0058] It can be seen from the above scheme that before training the first neural network model to obtain the first training model, a Softmax layer is added to the first neural network model to pre-train it to obtain a pre-trained network model, and then the first sample training set is input into the pre-trained network model to train the first training model, so that the training process of the first training model is more efficient and the training results are more reliable.

[0059] Figure 4 The flowchart of another method for identifying region boundaries based on model optimization iteration provided by the embodiment of the present invention provides a specific process of performing back propagation training on the second neural network model to obtain a pre-trained network model. Figure 4 As shown, the technical solution is as follows:

[0060] Step S401: Add a scene feature layer to the original neural network model to obtain a first neural network model.

[0061] Step S402: Add a Softmax layer after the scene feature layer in the first neural network model to obtain a second neural network model.

[0062] Step S403, freeze the backbone network parameters of the second neural network model, train the network parameters of the scene feature layer and the Softmax layer in the second neural network model, and after a preset number of iterative training, remove the Softmax layer in the second neural network model to obtain a pre-trained network model.

[0063] In one embodiment, the process of training the second neural network model to obtain the pre-trained network model is to freeze the backbone network parameters of the second neural network model and only train the network parameters of the scene feature layer and the Softmax layer. Specifically, the following code can be added to the second neural network model:

[0064] forp in self.parameters():

[0065] p.requires_grad=False

[0066] After the scene feature layer and the Softmax layer are loaded successfully, the original backbone network parameters are set not to be updated, and the following code is added to the optimizer:

[0067] filter(lambdap:p.requires_grad,model.parameters())

[0068] The specific ID of the backbone network parameter that needs to be frozen can be obtained by querying the corresponding data dictionary.

[0069] In one embodiment, after the second neural network model is iteratively trained for a preset number of times (such as Epoch=10), the Softmax layer is removed to obtain a final pre-trained network model, which is used for subsequent training to obtain a first training model.

[0070] Step S404: input the first sample training set into the pre-trained network model, and train the pre-trained network model to obtain a first training model.

[0071] Step S405: performing a clustering operation on the scene feature vector output by the pre-trained network model to obtain scene clustering information.

[0072] It can be seen from the above scheme that by freezing the backbone network parameters of the second neural network model and training the network parameters of the scene feature layer and the Softmax layer in the second neural network model, the model training can be completed quickly and efficiently. After a preset number of iterative training times, the Softmax layer in the second neural network model is removed to obtain a pre-trained network model, so that the recognition accuracy of the first training model obtained by subsequent training for image scenes is significantly improved.

[0073] Figure 5 The flowchart of another method for identifying region boundaries based on model optimization iteration provided by the embodiment of the present invention provides a specific process of inputting a first sample training set into a first neural network model and training the first neural network model. Figure 5 As shown, the technical solution is as follows:

[0074] Step S501, adding a scene feature layer to the original neural network model to obtain a first neural network model, obtaining each sample image pair in a first sample training set, and sequentially training the first neural network model with each sample image pair to obtain a first training model.

[0075] In one embodiment, when the first neural network model is trained, it is trained by inputting a sample image pair. Specifically, by calculating the similarity of two input sample images, the similarity of more similar sample images should be trained to have a closer calculated value, and the similarity of less similar sample images should be trained to have a more distant calculated value. Therefore, the loss function is calculated by the scene feature vector corresponding to the sample image in each sample image pair and the weight of the sample image pair.

[0076] In one embodiment, the loss function may be the KL-divergence loss between the scene feature vectors corresponding to the two sample images multiplied by a weight, or the cosine distance between the scene feature vectors corresponding to the two sample images multiplied by a weight. Optionally, taking the sample image pair as a map tile slice pair as an example, each map tile slice in the map tile slice pair includes corresponding geographic location information (expressed as a Mercator coordinate slice row number and a column number), and the weight of the sample image pair is the inverse of the distance information calculated from the geographic location information corresponding to the two map tile slices, wherein the distance information is defined as the bi-norm of the Mercator coordinate slice row number and the column number, and the weight value thus set makes it possible for the first neural network to output similar scene feature vectors for sample images with similar scenes when training it, and to output significantly different scene feature vectors for sample images with different scenes.

[0077] Step S502: performing a clustering operation on the scene feature vector output by the first neural network model to obtain scene clustering information.

[0078] Step S503: input the first image into a first training model, and output a first scene feature vector corresponding to the first image through the first training model.

[0079] The first image includes a first high-precision map image. Figure 6 A flowchart for determining a scene feature vector provided by an embodiment of the present invention defines inputting a first image into a first training model and outputting a first scene feature vector corresponding to the first image through the first training model. Figure 6 As shown, specifically including:

[0080] Step S5031: divide the first high-precision map image into multiple map tile slices, and input each map tile slice into the first training model to obtain multiple scene feature vectors.

[0081] Exemplarily, the size of the first high-precision map image is 1024*1024, which can be divided into multiple map tile slices of 256*256 image size, and each map tile slice is input into the first training model to obtain multiple scene feature vectors.

[0082] Step S5032: Determine the calculated mean of the multiple scene feature vectors as the first scene feature vector.

[0083] Step S504: determine whether the first image has a corresponding recognition scene according to the first scene feature vector and the scene clustering information.

[0084] Step S505: If the corresponding recognition scene does not exist, add the first image to the first sample training set to obtain a second sample training set, input the second sample training set to the original neural network model, and train the original neural network model.

[0085] In one embodiment, a specific process of processing the first image and then inputting it into the original neural network for training is provided. Figure 7 A flowchart of training an original neural network model is provided in an embodiment of the present invention. Figure 7 As shown, specifically including:

[0086] Step S5051: annotate a first image in a second sample training set that meets a preset condition to obtain a first annotated image.

[0087] In one embodiment, when the original neural network model is retrained using the second sample training set, taking the second sample training set containing multiple added first images as an example, the first images with similar scene feature vectors are used as sample images corresponding to the scene to be identified, and they can be manually annotated to obtain first annotated images. The manual annotations here are annotations for the original neural network model.

[0088] Step S5052: input the second sample training set containing the first labeled image into the original neural network model to train the original neural network model.

[0089] By retraining the second sample training set containing the first labeled image, the trained neural network model has better image boundary recognition ability.

[0090] It can be seen from the above scheme that the above-mentioned region boundary recognition method based on model optimization iteration can automatically identify scenes that are not in the sample training set, screen them and then mark them, reducing the workload of manual screening and labeling, and significantly improving the scene generalization ability of the neural network model obtained by iterative training, and can realize the boundary recognition of regions in images under complex scenes in different regions.

[0091] Figure 8 A flowchart of another method for identifying region boundaries based on model optimization iteration provided by an embodiment of the present invention provides a process including offline training, online training, and evaluation and retraining. Figure 8 As shown, the technical solution is as follows:

[0092] Step S601, adding a scene feature layer to the original neural network model to obtain a first neural network model;

[0093] Step S602: Add a Softmax layer after the scene feature layer in the first neural network model to obtain a second neural network model, and perform back propagation training on the second neural network model to obtain a pre-trained network model.

[0094] Step S603: input the first sample training set into the pre-trained network model, and train the pre-trained network model to obtain a first training model.

[0095] Step S604: performing a clustering operation on the scene feature vector output by the pre-trained network model to obtain scene clustering information.

[0096] Step S605: input the first image into a first training model, and output a first scene feature vector corresponding to the first image through the first training model.

[0097] Step S606: Determine whether the first image has a corresponding recognition scene according to the first scene feature vector and the scene clustering information; if so, execute step S607; otherwise, execute step S608.

[0098] Step S607: updating the cluster center of the recognized scene corresponding to the first image according to the first scene feature vector.

[0099] In one embodiment, if there is a recognized scene corresponding to the first image, optionally, the cumulative identification of the recognized scene is increased by 1, and the cluster center corresponding to the recognized scene is updated. Specifically, the updating method can be based on the center updating method of spectral clustering, and this solution does not limit the specific cluster center updating method.

[0100] Step S608: adding the first image to the first sample training set to obtain a second sample training set, inputting the second sample training set to the original neural network model, and training the original neural network model.

[0101] It can be seen from the above scheme that through the above-mentioned region boundary recognition method based on model optimization iteration, firstly, unsupervised classification is performed on the scene corresponding to the image offline training, and then it is determined whether a new scene that is not in the initial sample training set is recognized by online prediction, that is, inputting the first image. If there is no new scene, the clustering information during offline training is updated to ensure the accuracy of scene recognition. If a new scene is recognized, the original neural network model is re-optimized and trained so that the final neural network model scene generalization ability is enhanced, and the image region boundaries corresponding to various scenes can be efficiently and accurately recognized.

[0102] Fig. 9 This is a structural block diagram of a region boundary identification device based on model optimization iteration provided by an embodiment of the present invention. The device is used to execute the region boundary identification method based on model optimization iteration provided by the above embodiment, and has the corresponding functional modules and beneficial effects of the execution method. Fig. 9 As shown, the device specifically includes: an image input module 101, a feature vector output module 102, a scene recognition module 103, a sample adding module 104 and a network model training module 105, wherein:

[0103] An image input module 101, used to input a first image into a first training model;

[0104] A feature vector output module 102, configured to output a first scene feature vector corresponding to the first image through the first training model;

[0105] A scene recognition module 103, configured to determine whether the first image has a corresponding recognition scene according to the first scene feature vector and scene clustering information, wherein the scene clustering information includes one or more cluster centers, wherein each cluster center corresponds to a recognition scene;

[0106] A sample adding module 104, configured to add the first image to the first sample training set to obtain a second sample training set if the corresponding recognition scene does not exist;

[0107] The network model training module 105 is used to input the second sample training set into the original neural network model, and train the original neural network model to identify the region boundary in the image, wherein the first training model is a model obtained by modifying and training the original neural network model.

[0108] It can be seen from the above scheme that by inputting the first image into the first training model to output the first scene feature vector corresponding to the first image, and then determining whether the first image has a corresponding recognition scene based on the first scene feature vector and the scene clustering information, if not, then adding the first image to the first sample training set to obtain the second sample training set, inputting the second sample training set into the original neural network model for training, and using the trained neural network model to recognize the region boundaries in the image. The region boundary recognition method based on model optimization iteration provided by this scheme can quickly identify training scenes that do not exist in the original training set, and optimize and iterate the neural network model, solving the problems of weak scene generalization ability of the existing neural network model used for region boundary recognition, poor recognition effect and low recognition accuracy for the region boundary of the newly added scene, improving the accuracy of boundary recognition of the region contained in the image, and significantly improving the recognition effect.

[0109] In a possible embodiment, the network model training module 105 is further used to:

[0110] Before inputting the first image into the first training model, a scene feature layer is added to the original neural network model to obtain a first neural network model; inputting the first sample training set into the first neural network model, training the first neural network model to obtain the first training model, wherein the scene feature layer is used to output a scene feature vector corresponding to each sample image in the first sample training set;

[0111] The scene recognition module 103 is specifically used to perform a clustering operation on the scene feature vector output by the first neural network model to obtain scene clustering information.

[0112] In a possible embodiment, the network model training module 105 is further used to:

[0113] After adding a scene feature layer to the original neural network model to obtain a first neural network model, adding a Softmax layer after the scene feature layer in the first neural network model to obtain a second neural network model, and performing back propagation training on the second neural network model to obtain a pre-trained network model;

[0114] The first sample training set is input into the pre-trained network model, and the pre-trained network model is trained to obtain a first training model.

[0115] In a possible embodiment, the original neural network model includes a neural network segmentation model based on deep learning, and the network model training module 105 is specifically used for:

[0116] Freeze the backbone network parameters of the second neural network model, and train the network parameters of the scene feature layer and the Softmax layer in the second neural network model;

[0117] After a preset number of iterative training times, the Softmax layer in the second neural network model is removed to obtain a pre-trained network model.

[0118] In a possible embodiment, the network model training module 105 is specifically used for:

[0119] Obtain each sample image pair in the first sample training set, and train the first neural network model in sequence through each sample image pair, wherein the loss function when training the first neural network model is calculated by the scene feature vector corresponding to the sample image in each sample image pair and the weight of the sample image pair.

[0120] In one possible embodiment, the sample image pair includes a map tile slice pair, wherein each map tile slice in the map tile slice pair includes corresponding geographic location information, and the weight of the sample image pair includes the inverse of distance information calculated from the corresponding geographic location information of two map tile slices.

[0121] In a possible embodiment, the first image includes a first high-precision map image, and the image input module 101 is used to:

[0122] Slicing the first high-precision map image into a plurality of map tile slices, and inputting each map tile slice into a first training model respectively;

[0123] The feature vector output module 102 is specifically used to: after obtaining multiple scene feature vectors output by the first training model, determine the calculated mean of the multiple scene feature vectors as the first scene feature vector.

[0124] In a possible embodiment, the scene recognition module 103 is further configured to:

[0125] If the corresponding recognition scene exists, the cluster center of the recognition scene corresponding to the first image is updated according to the first scene feature vector.

[0126] In a possible embodiment, the network model training module 105 is further used to:

[0127] Before inputting the second sample training set into the original neural network model, annotating the first image in the second sample training set that meets the preset conditions to obtain a first annotated image;

[0128] A second sample training set including the first labeled image is input into the original neural network model.

[0129] Fig.10 A schematic diagram of a regional boundary recognition device based on model optimization iteration provided by an embodiment of the present invention is shown in FIG. Fig.10 As shown, the device includes a processor 201, a memory 202, an input device 203 and an output device 204; the number of processors 201 in the device can be one or more. Fig.10 A processor 201 is taken as an example; the processor 201, memory 202, input device 203 and output device 204 in the device can be connected by a bus or other means. Fig.10 The example of connecting through a bus is taken. The memory 202, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the message data processing method in the embodiment of the present invention. The processor 201 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 202, that is, implements the above-mentioned message data processing method. The input device 203 can be used to receive input digital or character information, and generate key signal input related to user settings and function control of the device. The output device 204 may include a display device such as a display screen.

[0130] An embodiment of the present invention further provides a storage medium containing computer executable instructions, wherein the computer executable instructions are used to execute a method for identifying region boundaries based on model optimization iteration when executed by a computer processor, the method comprising:

[0131] Inputting a first image into a first training model, and outputting a first scene feature vector corresponding to the first image through the first training model;

[0132] Determining whether the first image has a corresponding recognition scene according to the first scene feature vector and scene clustering information, wherein the scene clustering information includes one or more cluster centers, wherein each cluster center corresponds to a recognition scene;

[0133] If the corresponding recognition scene does not exist, the first image is added to the first sample training set to obtain a second sample training set, the second sample training set is input into the original neural network model, and the original neural network model is trained to identify the region boundaries in the image, wherein the first training model is a model obtained by modifying and training the original neural network model.

[0134] It is worth noting that in the above-mentioned embodiment of area boundary identification, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present invention.

[0135] The above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the embodiments of the present invention are not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the protection scope of the embodiments of the present invention. Therefore, although the embodiments of the present invention are described in more detail through the above embodiments, the embodiments of the present invention are not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the embodiments of the present invention, and the scope of the embodiments of the present invention is determined by the scope of the appended claims.

Claims

1. A region boundary identification method based on model optimization iteration, characterized in that: include: Inputting a first image into a first training model, and outputting a first scene feature vector corresponding to the first image through the first training model; Determining whether the first image has a corresponding recognition scene according to the first scene feature vector and scene clustering information, wherein the scene clustering information includes one or more cluster centers, wherein each cluster center corresponds to a recognition scene; If the corresponding recognition scene does not exist, the first image is added to the first sample training set to obtain a second sample training set, and the second sample training set is input into the original neural network model, and the original neural network model is trained to identify the region boundaries in the image, wherein the first training model is a model obtained by modifying and training the original neural network model, and the first sample training set is a sample training set corresponding to the first training model when it is trained.

2. The method for identifying region boundaries based on model optimization iteration according to claim 1, characterized in that: Before inputting the first image into the first training model, the method further includes: Adding a scene feature layer to the original neural network model obtains a first neural network model; Inputting the first sample training set into the first neural network model, training the first neural network model to obtain a first training model, wherein the scene feature layer is used to output a scene feature vector corresponding to each sample image in the first sample training set; A clustering operation is performed on the scene feature vector output by the first neural network model to obtain scene clustering information.

3. The method for identifying region boundaries based on model optimization iteration according to claim 2, characterized in that: After adding the scene feature layer to the original neural network model to obtain the first neural network model, it also includes: A Softmax layer is added after the scene feature layer in the first neural network model to obtain a second neural network model, and back-propagation training is performed on the second neural network model to obtain a pre-trained network model; The step of inputting the first sample training set into the first neural network model and training the first neural network model to obtain a first training model includes: The first sample training set is input into the pre-trained network model, and the pre-trained network model is trained to obtain a first training model.

4. The method for identifying region boundaries based on model optimization iteration according to claim 3, characterized in that: The original neural network model includes a neural network segmentation model based on deep learning, and the back-propagation training of the second neural network model to obtain a pre-trained network model includes: Freeze the backbone network parameters of the second neural network model, and train the network parameters of the scene feature layer and the Softmax layer in the second neural network model; After a preset number of iterative training times, the Softmax layer in the second neural network model is removed to obtain a pre-trained network model.

5. The method for identifying region boundaries based on model optimization iteration according to claim 2, characterized in that: The step of inputting the first sample training set into the first neural network model to train the first neural network model includes: Obtain each sample image pair in the first sample training set, and train the first neural network model in sequence through each sample image pair, wherein the loss function when training the first neural network model is calculated by the scene feature vector corresponding to the sample image in each sample image pair and the weight of the sample image pair.

6. The method for identifying region boundaries based on model optimization iteration according to claim 5, characterized in that: The sample image pair includes a map tile pair, wherein each map tile in the map tile pair includes corresponding geographic location information, and the weight of the sample image pair includes the inverse of distance information calculated from the corresponding geographic location information of two map tile slices.

7. The method for identifying region boundaries based on model optimization iteration according to claim 1, characterized in that: The first image includes a first high-precision map image, and the step of inputting the first image into a first training model and outputting a first scene feature vector corresponding to the first image through the first training model includes: Slicing the first high-precision map image into a plurality of map tile slices, and inputting each map tile slice into a first training model to obtain a plurality of scene feature vectors; The calculated mean of the multiple scene feature vectors is determined as the first scene feature vector.

8. The method for identifying region boundaries based on model optimization iteration according to any one of claims 1 to 7, characterized in that: If the corresponding recognition scene exists, the cluster center of the recognition scene corresponding to the first image is updated according to the first scene feature vector.

9. The method for identifying region boundaries based on model optimization iteration according to any one of claims 1 to 7, characterized in that: Before inputting the second sample training set into the original neural network model, the method further includes: annotating a first image in the second sample training set that meets a preset condition to obtain a first annotated image; The step of inputting the second sample training set into the original neural network model comprises: A second sample training set including the first labeled image is input into the original neural network model.

10. A region boundary identification device based on model optimization iteration, characterized in that: include: An image input module, used for inputting a first image into a first training model; A feature vector output module, configured to output a first scene feature vector corresponding to the first image through the first training model; a scene recognition module, configured to determine whether the first image has a corresponding recognition scene according to the first scene feature vector and scene clustering information, wherein the scene clustering information includes one or more cluster centers, wherein each cluster center corresponds to a recognition scene; A sample adding module, configured to add the first image to the first sample training set to obtain a second sample training set if the corresponding recognition scene does not exist; A network model training module is used to input the second sample training set into the original neural network model, and train the original neural network model to identify region boundaries in an image, wherein the first training model is a model obtained by modifying and training the original neural network model, and the first sample training set is a sample training set corresponding to the first training model when it is trained.

11. A device for identifying region boundaries based on model optimization iteration, the device comprising: two or more processors; A storage device for storing two or more programs. When the two or more programs are executed by the two or more processors, the two or more processors implement the region boundary recognition method based on model optimization iteration as described in any one of claims 1 to 9.

12. A storage medium storing computer executable instructions, wherein the computer executable instructions are used to execute the region boundary identification method based on model optimization iteration according to any one of claims 1 to 9 when executed by a computer processor.

Images