An image recognition method for adaptive scenarios

Abstract

The present invention discloses an image recognition method for an adaptive scenario, which relates to the technical field of image recognition. It solves the technical problem that in the prior art, during the image recognition process, the influence of the scene environment on image recognition is considered, but the extraction differences between various image features are not considered, resulting in inefficient and inaccurate image recognition. According to the image features and the correlation relationships between various image features, the present invention establishes a feature knowledge graph, combines the correlation relationships between various image features to establish a number of feature recognition models, and uses the corresponding feature recognition models to analyze and recognize the image features in the sub-images to obtain the corresponding feature information, reducing the training amount of the feature recognition models and being able to improve the accuracy of the feature information. The present invention introduces the knowledge graph technology to establish a feature knowledge graph, can quickly call the feature knowledge graph corresponding to the target feature, and can accurately recognize rare target features in combination with the unsupervised image recognition algorithm, expanding the application scenario.

Description

Technical Field

[0001] The present invention belongs to the field of image recognition, relates to image recognition technology for adaptive scenarios, and specifically is an image recognition method for adaptive scenarios. Background Art

[0002] Image recognition mainly includes two stages: feature extraction and feature comparison. Generally, a single model is used to extract different image features in different scenarios. However, due to the complex image data scenarios and rich texture information, it is easy to cause abnormal image recognition results.

[0003] The prior art (a patent for invention with application number 2019102611854) discloses an environment-aware adaptive image recognition method, which adaptively calls a scene model to extract features from image data, avoiding the influence of the scene environment on the extraction of image features. In the process of image recognition in the prior art, the influence of the scene environment on image recognition is considered, but the extraction differences between various image features are not considered, resulting in inefficient and inaccurate image recognition. Therefore, there is an urgent need for an image recognition method for adaptive scenarios. Summary of the Invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art. For this purpose, the present invention proposes an image recognition method for adaptive scenarios, which is used to solve the technical problem that in the process of image recognition in the prior art, the influence of the scene environment on image recognition is considered, but the extraction differences between various image features are not considered, resulting in inefficient and inaccurate image recognition.

[0005] The present invention establishes a feature knowledge graph according to the image features and the correlation relationships between the image features, builds a number of feature recognition models based on a neural network model, and uses the corresponding feature recognition models to analyze and recognize sub-images to obtain corresponding feature information, which can improve the accuracy of the feature information.

[0006] To achieve the above object, the first aspect of the present invention provides an image recognition method for adaptive scenarios, including:

[0007] Obtain feature correlation data, and establish a feature knowledge graph according to the feature correlation data; wherein, the feature correlation data includes image features and the correlation relationships between the image features;

[0008] Establish a number of feature recognition models based on the correlation relationships between the image features; wherein, the number of feature recognition models is obtained by training based on a neural network model;

[0009] Obtain a target image, combine image segmentation technology to segment the pre-processed target image into a number of sub-images, and generate an image sequence according to the number of sub-images;

[0010] Combine the target features in the sub-images with the feature knowledge graph, call the feature recognition model to identify and analyze the sub-images, and obtain the motion trajectory of the target features in combination with the image sequence.

[0011] Preferably, the feature knowledge graph is established through the feature association data, including:

[0012] Obtain the feature association data, and extract the image features in the feature association data as graph entities;

[0013] Combine a number of the graph entities with the corresponding association relationships to generate a number of triples; wherein, the triple includes two graph entities and the association relationship between the two graph entities;

[0014] Construct and generate the feature knowledge graph based on a number of the triples.

[0015] Preferably, classify the image features based on the association relationship, and establish the feature recognition model according to the classification result, including:

[0016] Obtain the classification rules; wherein, the classification rules include classification by family, classification by species, or classification by item category;

[0017] Divide the image features into several feature subclasses according to the classification rules, and train a neural network model with the training images corresponding to the feature subclasses, thereby obtaining a number of the feature recognition models.

[0018] Preferably, divide the target image into several of the sub-images according to the image recognition technology, including:

[0019] Obtain the target image; wherein, the target image is extracted from a video;

[0020] Divide the target image after image preprocessing into several of the sub-images through image segmentation technology; wherein, the image segmentation technology includes semantic segmentation and texture segmentation, and the image preprocessing includes grayscale transformation and image denoising.

[0021] Preferably, extract several of the target images from the video, including:

[0022] Obtain the video;

[0023] Extract several target images from the video at a set frame interval; wherein, the set frame interval is the number of frames between adjacent two target images.

[0024] Preferably, when dividing the target image by the image segmentation technology, record the position data of the several obtained sub-images in the target image;

[0025] Generate the image sequence based on the plurality of sub-images and the corresponding position data.

[0026] Preferably, classify the plurality of sub-images according to the feature knowledge graph and the target feature, obtain a plurality of target categories, and call the feature recognition model corresponding to the target category;

[0027] Identify the sub-images based on the feature recognition model to obtain the feature information corresponding to the target feature; wherein, the feature information includes type and size.

[0028] Preferably, after obtaining the feature information of the target feature, obtain the motion trajectory corresponding to the target feature through the plurality of image sequences, including:

[0029] Obtain the plurality of image sequences; wherein, the plurality of image sequences are sorted according to the acquisition time;

[0030] Obtain the plurality of sub-images corresponding to the target feature from the plurality of image sequences, and simulate and obtain the motion trajectory corresponding to the target feature according to the corresponding plurality of sub-images.

[0031] Compared with the prior art, the beneficial effects of the present invention are:

[0032] 1. The present invention establishes a feature knowledge graph according to the image features and the correlation relationships between the image features, establishes a plurality of feature recognition models in combination with the correlation relationships between the image features, analyzes and identifies the image features in the sub-images by using the corresponding feature recognition models, and obtains the corresponding feature information, reducing the training amount of the feature recognition models and being able to improve the accuracy of the feature information.

[0033] 2. The present invention introduces the knowledge graph technology to establish a feature knowledge graph, can quickly call the feature knowledge graph corresponding to the target feature, and can also achieve accurate recognition of rare target features in combination with the unsupervised image recognition algorithm, expanding the application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0035] Figure 1 It is a schematic diagram of the working steps of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0036] The technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0037] In the prior art, during the image recognition process, the influence of the scene environment on image recognition is considered, that is, the existing scenes are divided into different types, and an image recognition model is trained for each type of scene. Then, the image to be recognized is recognized and analyzed by each image recognition model; the extraction differences between image features are not considered. Using the same image recognition model for different features will cause recognition errors, resulting in inefficient and inaccurate image recognition.

[0038] The present invention establishes a feature knowledge graph based on image features and the association relationships between image features, establishes a number of feature recognition models in combination with the association relationships between image features, and uses the corresponding feature recognition models to analyze and recognize the image features in the sub-images to obtain the corresponding feature information, which can improve the accuracy of the feature information.

[0039] Please refer to Figure 1 , an image recognition method for an adaptive scene provided by the first aspect embodiment of the present invention includes:

[0040] Obtain feature association data, and establish a feature knowledge graph according to the feature association data; establish a number of feature recognition models based on the association relationships between image features;

[0041] Obtain a target image, combine the image segmentation technology to segment the pre-processed target image into a number of sub-images, and generate an image sequence according to the number of sub-images;

[0042] Combine the target features in the sub-images with the feature knowledge graph, call the feature recognition model to recognize and analyze the sub-images, and obtain the motion trajectory of the target features in combination with the image sequence.

[0043] The present invention mainly considers that different target features have different external manifestations. Therefore, if the feature recognition models used for different target features are the same, it will cause inaccurate extraction of some target features and increase the training difficulty of the feature recognition models. Build feature recognition models according to the association relationships between target features, and use appropriate feature recognition models to recognize and analyze sub-images to ensure that target features can be accurately recognized and extracted.

[0044] It should be noted that when the feature recognition model directly corresponding to the target feature cannot be searched, the appropriate feature recognition model can be obtained through the association relationship between each image feature in the feature knowledge graph, which can effectively reduce the training volume of the feature recognition model and then improve the target feature recognition efficiency in various scenarios.

[0045] The feature recognition model in this application is trained based on a neural network model, that is, several training images in the application scenario are obtained, and the neural network model is continuously trained through these training images to obtain the corresponding feature recognition model. For example, if the feature recognition model is applied to an autonomous driving vehicle, several training images are collected by the driving vehicle, and the labeled training images are input into the convolutional neural network model for training. The training and use of the feature recognition model can refer to the paper "Research on Automatic Recognition of Agricultural Machinery Images Based on Convolutional Neural Network" (Volume 43, Issue 5) published by Lei Xuemei, Zhang Guangqiang, etc. in "Journal of Chinese Agricultural Mechanization" and the paper "Research on Image Recognition Based on Deep Fully Convolutional Neural Network" (Volume 38, Issue 2) published by Ji Zhuangwei in "Journal of Shanxi Datong University".

[0046] In this invention application, a feature knowledge graph is established through feature association data, including:

[0047] Obtain feature association data, extract the image features in the feature association data as graph entities; combine several graph entities with the corresponding association relationships to generate several triples; construct and generate a feature knowledge graph based on several triples.

[0048] The feature association data includes image features and the association relationships between each image feature. The image features have the same attributes as the target features, that is, the image features are the targets in the feature association data, and the target features are the targets in the target images. The association relationship between image features indicates that there are certain similar features between the two. For example, cats and dogs both belong to pets, and poplar trees and roses both belong to plants, etc. The association relationship is also an essential content for constructing the feature knowledge graph. Extract the image features and association relationships from the feature association data, obtain several triples, and establish a feature knowledge graph based on the construction rules of the knowledge graph. The construction of the feature knowledge graph can refer to the paper "Review of Knowledge Graphs - Representation, Construction, Reasoning and Knowledge Hypergraph Theory" (Volume 41, Issue 8) published by Tian Ling, Zhang Jinchuan, etc. in "Computer Applications" and other existing technologies.

[0049] In this invention application, the image features are classified based on the association relationship, and a feature recognition model is established according to the classification result, including:

[0050] Obtain classification rules; divide the image features into several feature subclasses according to the classification rules, and train the neural network model through the training images corresponding to the feature subclasses, and then obtain several feature recognition models.

[0051] The feature correlation data contains numerous image features. Therefore, the image features are divided into several feature subclasses, and the neural network model is trained using the training images corresponding to the feature subclasses, and then the corresponding feature recognition model is obtained.

[0052] It should be noted that in order to adapt to various application scenarios, it is necessary to establish feature recognition models corresponding to various image features. However, no matter how much training and construction are carried out, all target features cannot be included. Therefore, classification principles are set to divide the image features to ensure that at least one feature recognition model corresponds to a certain type of image features. Of course, if resources and capabilities permit, it is also possible to ensure that each image feature corresponds to a feature recognition model.

[0053] In the process of ensuring that each type of image feature corresponds to a feature recognition model, it is necessary to ensure that the training images during the training of the feature recognition model should cover all image features. However, the data volume of some common image features is larger, and the training image data corresponding to uncommon image features can be less, and then the association relationship between the image features of this category and the feature recognition model is established.

[0054] It can be understood that the classification rules include classification according to family and genus, classification according to species, or classification according to item categories; classification according to family and genus (derived from ICBN, that is, the International Code of Botanical Nomenclature) is mainly used for plant classification. For example, Polygonatum sibiricum, Polygonatum odoratum, Rohdea japonica, Colchicum autumnale, etc. belong to the Liliaceae family, and plants of the same family and genus have similar characteristics and habits; species classification is used for animal classification. For example, cats and tigers belong to the Felidae family, and animals of the same species have similar external characteristics; item categories are used to classify daily-use items. For example, buildings, villas, etc. belong to architecture.

[0055] In the present invention application, the target image is divided into several sub-images according to the image recognition technology, including:

[0056] Obtain the target image; divide the target image after image preprocessing into several sub-images through image segmentation technology.

[0057] The target image may include multiple target features. For example, an image includes a person, a cat, a bicycle, a store, etc. Therefore, these target features are separated through image recognition technology to ensure that each sub-image includes at least one target feature. The image segmentation technology includes semantic segmentation and texture segmentation, and the image preprocessing includes grayscale transformation and image denoising.

[0058] It can be understood that there may be a phenomenon where two target features overlap in the target image. In this case, it can be solved by referring to existing image processing technologies. For example, the blog article "How to Achieve Image Recognition and Object Detection? You'll Understand After Reading" gives examples of image recognition and object detection.

[0059] This invention application extracts several target images from a video, including:

[0060] Obtain a video; extract several target images from the video at a set frame interval; where the set frame interval is the number of frames between adjacent two target images.

[0061] Target images are generally extracted from videos, such as surveillance videos, in-vehicle videos, etc. Of course, in some cases, target images can also be directly obtained by shooting. In many scenarios, identifying target features or even obtaining the motion trajectory of target features does not require continuous video frames. Therefore, a set frame interval is introduced, and the set frame interval is an integer greater than or equal to 0. When the set frame interval is 0, it means that the target images are continuous in the video.

[0062] In this invention application, when segmenting the target image through image segmentation technology, record the position data of several obtained sub-images in the target image; generate an image sequence based on the several sub-images and the corresponding position data.

[0063] When dividing the target image into several sub-images, the original position information of the several sub-images should be recorded to ensure that the sub-images can be restored to the target image. The several sub-images can be re-stitched into the target image, and the image sequence is a digital representation of the sub-images and the corresponding position data. For example, there are squares of the same size numbered 1, 2, 3, 4, and the corresponding position data are upper left, upper right, lower left, and lower right, then the image sequence is [(1, upper left), (2, upper right), (3, lower left), (4, lower right)]. It can be understood that the form of the position data can be various, as long as the specific position of the corresponding sub-image in the target image can be obtained through the position data.

[0064] In this invention application, classify several sub-images according to the feature knowledge graph and target features to obtain several target categories, and call the feature recognition model corresponding to the target category; identify the sub-images based on the feature recognition model to obtain the feature information corresponding to the target features.

[0065] Determine the feature recognition model corresponding to the target category, and then input the sub-images into the feature recognition model to obtain the corresponding feature information. It can be understood that image classification is a rough division of sub-images, which is not convenient for detailed recognition of the target features therein. The feature recognition model can perform detailed recognition on the target features, and then obtain information such as the size, color, and position of the target features.

[0066] It should be noted that if the target category corresponding to a certain target feature is determined according to the feature knowledge graph, and the training images corresponding to the target feature were not used when training the feature recognition model corresponding to the target type at that time, and the recognition accuracy of the feature recognition model cannot be guaranteed, then the neural network model can be replaced with other unsupervised image recognition models, such as fuzzy clustering algorithms, etc. For reference, please refer to the paper "Application of Fuzzy Clustering Unsupervised Algorithm in Image Recognition" (Volume 39, Issue 1) published by Mo Li, Li Longlong, etc. in "Techniques of Automation and Applications".

[0067] After obtaining the feature information of the target feature in the present invention application, the motion trajectory corresponding to the target feature is obtained through a plurality of image sequences, including:

[0068] Obtain a plurality of image sequences; obtain a plurality of sub-images corresponding to the target feature from the plurality of image sequences, and simulate and obtain the motion trajectory corresponding to the target feature according to the corresponding plurality of sub-images.

[0069] It should be noted that the plurality of image sequences are sorted according to the acquisition time, that is, one target object corresponds to one image sequence, so the number of image sequences is the same as that of the target images. According to the feature information, the position information of the target feature in the sub-image can be determined, and then combined with the position information of the sub-image in the target image and the time relationship of each target image, the motion trajectory of the target feature can be obtained.

[0070] When the present invention application is applied to different scenarios, it can reduce the training amount of the feature recognition model, and combined with the feature knowledge graph, it can also quickly locate the feature recognition model to ensure the image recognition efficiency.

[0071] The working principle of the present invention:

[0072] Obtain feature correlation data, and establish a feature knowledge graph according to the feature correlation data; establish a plurality of feature recognition models based on the correlation relationship between image features.

[0073] Obtain a target image, combine the image segmentation technology to segment the pre-processed target image into a plurality of sub-images, and generate an image sequence according to the plurality of sub-images.

[0074] Combine the target feature in the sub-image with the feature knowledge graph, call the feature recognition model to identify and analyze the sub-image, and obtain the motion trajectory of the target feature in combination with the image sequence.

[0075] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical method of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical method of the present invention.

Claims

1. An image recognition method for adaptive scenarios, characterized in that, it includes: Obtain feature correlation data, and establish a feature knowledge graph according to the feature correlation data; wherein, the feature correlation data includes image features and the correlation relationships between various image features; Establish a number of feature recognition models based on the correlation relationships between the image features; wherein, the number of feature recognition models is obtained by training based on a neural network model; Obtain a target image, combine image segmentation technology to segment the pre-processed target image into a number of sub-images, and generate an image sequence according to the number of sub-images; Combine the target features in the sub-images with the feature knowledge graph, call the feature recognition models to identify and analyze the sub-images, and obtain the motion trajectory of the target features in combination with the image sequence; Establishing the feature knowledge graph through the feature correlation data includes: Obtain the feature correlation data, and extract the image features in the feature correlation data as graph entities; Combine a number of the graph entities with the corresponding correlation relationships to generate a number of triples; wherein, each triple includes two graph entities and the correlation relationship between the two graph entities; Construct and generate the feature knowledge graph based on a number of the triples; Classify the image features based on the correlation relationships, and establish the feature recognition models according to the classification results, including: Obtain classification rules; wherein, the classification rules include classification by family, classification by species, or classification by item category; Divide the image features into a number of feature subclasses according to the classification rules, and train a neural network model with the training images corresponding to the feature subclasses, and then obtain a number of the feature recognition models; Dividing the target image into a number of the sub-images according to the image recognition technology includes: Obtain the target image; wherein, the target image is extracted from a video; Divide the pre-processed target image into a number of the sub-images through image segmentation technology; wherein, the image segmentation technology includes semantic segmentation and texture segmentation, and the image pre-processing includes grayscale transformation and image denoising; When segmenting the target image through the image segmentation technology, record the position data of the number of sub-images obtained in the target image; Generate the image sequence based on the number of sub-images and the corresponding position data; Classify a number of the sub-images according to the feature knowledge graph and the target features, obtain a number of target categories, and call the feature recognition models corresponding to the target categories; Identify the sub-images based on the feature recognition models, and obtain the feature information corresponding to the target features; wherein, the feature information includes type and size; After obtaining the feature information of the target features, obtain the motion trajectory corresponding to the target features through a number of the image sequences, including: Obtain a number of the image sequences; wherein, the number of image sequences is sorted according to the acquisition time; Obtain a number of the sub-images corresponding to the target features from a number of the image sequences, and simulate and obtain the motion trajectory corresponding to the target features according to the corresponding number of sub-images.

2. An image recognition method for an adaptive scenario according to claim 1, wherein, extracting a plurality of the target images from a video, including: obtaining a video; extracting a plurality of target images from the video at a set frame interval; wherein, the set frame interval is the number of frames between two adjacent target images.

Images