Animal identification method and device, electronic equipment and storage medium

By acquiring animal images and description text, performing feature extraction and model training, and converting image features into text features, solving the problem of low animal recognition accuracy in traditional technology and achieving higher recognition accuracy.

CN119027982APending Publication Date: 2024-11-26PING AN TECH (SHENZHEN) CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202411252055.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-06
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

The existing animal recognition technology relies on traditional image processing methods and cannot effectively handle complex recognition tasks, resulting in low recognition accuracy.

Method used

By obtaining sample animal images and sample image description text, feature extraction and feature conversion model training are performed, image features are converted into text features, and animal recognition is performed based on text features.

Benefits of technology

It improves the recognition accuracy of animal images, realizes complex recognition tasks through text dimensions, and improves recognition accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027982A_ABST
    Figure CN119027982A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides an animal identification method and device, electronic equipment and a storage medium, and belongs to the technical field of image processing. The method comprises the following steps: acquiring a sample animal image with a sample animal and a sample image description text of the sample animal image; wherein the sample image description text is used for describing animal features of a sample animal in the sample animal image; performing feature extraction on the sample animal image to obtain sample animal feature information; training a preset feature conversion model through the sample image description text and the sample animal feature information to obtain a target feature conversion model; obtaining target animal feature information of the target animal image, and performing feature conversion on the target animal feature information through a target feature conversion model to obtain a target image description text; and performing animal identification on the target animal image according to the target image description text to obtain target image animal information. According to the embodiment of the invention, the recognition accuracy of the animal image can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an animal identification method and device, electronic equipment and storage medium. Background Art

[0002] In the process of claims settlement in the breeding industry, it is necessary to identify the insured animals and determine the categories of the insured animals. However, most existing animal recognition technologies rely on traditional image processing methods, which are often unable to effectively handle complex recognition tasks, resulting in low recognition accuracy under complex recognition tasks. Therefore, how to improve the recognition accuracy of animal images has become an urgent problem to be solved. Summary of the invention

[0003] The main purpose of the embodiments of the present application is to propose an animal identification method and device, an electronic device and a storage medium, aiming to improve the recognition accuracy of animal images.

[0004] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application provides an animal identification method, the method comprising: obtaining a sample animal image having a sample animal and a sample image description text of the sample animal image; wherein the sample image description text is used to describe the animal features of the sample animal in the sample animal image;

[0005] Extracting features from the sample animal image to obtain sample animal feature information;

[0006] The preset feature conversion model is trained by using the sample image description text and the sample animal feature information to obtain a target feature conversion model;

[0007] Acquire target animal feature information of the target animal image, and perform feature conversion on the target animal feature information through the target feature conversion model to obtain a target image description text;

[0008] The target animal image is subjected to animal recognition according to the target image description text to obtain the target image animal information.

[0009] In some embodiments, extracting features from the sample animal image to obtain sample animal feature information includes:

[0010] Performing image segmentation on the sample animal image to obtain an animal segmentation image;

[0011] Performing target positioning according to the animal segmentation image to obtain animal position features;

[0012] Performing color extraction based on the animal segmentation image to obtain animal color features;

[0013] Perform classification prediction based on the animal segmentation image to obtain classification prediction information;

[0014] Feature aggregation is performed based on the animal position features, animal color features and the classification prediction information to obtain the sample animal feature information.

[0015] In some embodiments, performing image segmentation on the sample animal image to obtain an animal segmentation image includes:

[0016] Performing image convolution on the sample animal image to obtain an animal convolution feature map;

[0017] Perform sliding window selection on the animal convolution feature map through a preset sliding window to obtain multiple candidate prediction boxes;

[0018] Perform image prediction according to the plurality of candidate prediction frames to obtain image prediction data; wherein the image prediction data is used to indicate whether the image in the candidate prediction frame is a background image or an animal image;

[0019] Screening multiple candidate prediction frames according to the image prediction data to obtain an animal prediction frame;

[0020] The coordinates of the animal prediction frame are corrected to obtain the animal segmentation image.

[0021] In some embodiments, the step of performing color extraction based on the animal segmentation image to obtain animal color features includes:

[0022] Performing color space conversion on the animal segmentation image to obtain color space data;

[0023] Performing two-dimensional discrete cosine calculation on the color space data to obtain two-dimensional discrete cosine data;

[0024] Frequency domain feature extraction is performed based on the two-dimensional discrete cosine data to obtain the animal color feature.

[0025] In some embodiments, performing classification prediction based on the animal segmentation image to obtain classification prediction information includes:

[0026] Performing morphological extraction on the animal segmented image by using a preset morphological extraction model to obtain animal morphological features;

[0027] Performing texture extraction on the animal segmentation image by using a preset texture extraction model to obtain animal texture features;

[0028] Classification prediction is performed according to the animal morphological features and the animal texture features to obtain the classification prediction information.

[0029] In some embodiments, the target feature conversion model includes a text input layer, an image input layer and a feature alignment layer, and the target feature conversion model is used to convert the target animal feature information to obtain the target image description text, including:

[0030] The preset conversion instruction text is embedded through the text input layer to obtain text embedding feature information; wherein the preset conversion instruction text is used to instruct the target animal feature information to perform feature conversion;

[0031] Performing feature encoding on the target animal feature information through the image input layer to obtain image encoding feature information;

[0032] The image encoding feature information and the text embedding feature information are feature aligned through the feature alignment layer to obtain the target image description text.

[0033] In some embodiments, performing animal recognition on the target animal image according to the target image description text to obtain the target image animal information includes:

[0034] Performing semantic understanding on the target image description text through a preset semantic understanding model to obtain semantic information of the target text;

[0035] The target animal image is subjected to animal recognition based on the target text semantic information to obtain the target image animal information.

[0036] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides an animal identification device, the device comprising:

[0037] A data acquisition module, used to acquire a sample animal image having a sample animal and a sample image description text of the sample animal image; wherein the sample image description text is used to describe the animal features of the sample animal in the sample animal image;

[0038] A feature extraction module, used to extract features from the sample animal image to obtain feature information of the sample animal;

[0039] A model training module, used to train a preset feature conversion model through the sample image description text and the sample animal feature information to obtain a target feature conversion model;

[0040] A feature conversion module, used for acquiring target animal feature information of a target animal image, and performing feature conversion on the target animal feature information through the target feature conversion model to obtain a target image description text;

[0041] The animal recognition module is used to perform animal recognition on the target animal image according to the target image description text to obtain the target image animal information.

[0042] To achieve the above objectives, a third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0043] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0044] The present application proposes an animal identification method and device, electronic device and storage medium, which obtain sample animal images and sample image description text, then extract features from the animal images to obtain sample animal feature information, and then train a preset feature conversion model through the sample image description text and the sample animal feature information, so that the preset feature conversion model has the ability to convert image features into text features, and obtain a target conversion feature model; further, obtain target animal feature information of the target animal image, and then convert the target animal feature information into a target image description text through the target feature conversion model, and perform animal identification on the target animal image based on the target image description text to obtain target animal image information, identify animal images from the dimension of text, and implement complex identification tasks through the dimension of text, thereby improving the recognition accuracy of animal images. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is a flow chart of an animal identification method provided in an embodiment of the present application;

[0046] Figure 2 yes Figure 1 Flow chart of step S102 in FIG.

[0047] Figure 3 yes Figure 2 Flow chart of step S201 in FIG.

[0048] Figure 4 yes Figure 2 Flow chart of step S203 in FIG.

[0049] Figure 5 yes Figure 2 Flow chart of step S204 in FIG.

[0050] Figure 6 yes Figure 1 Flow chart of step S104 in FIG.

[0051] Figure 7 yes Figure 1 Flow chart of step S105 in FIG.

[0052] Figure 8 is a schematic diagram of the structure of an animal identification device provided in an embodiment of the present application;

[0053] Fig. 9 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0055] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0057] First, some nouns involved in this application are analyzed:

[0058] Artificial intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a branch of computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robots, language recognition, image recognition, natural language processing and expert systems. AI can simulate the information process of human consciousness and thinking. AI is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0059] Natural language processing (NLP): NLP uses computers to process, understand and apply human languages ​​(such as Chinese and English). NLP is a branch of artificial intelligence and an interdisciplinary subject between computer science and linguistics. It is often referred to as computational linguistics. Natural language processing includes grammatical analysis, semantic analysis, and text understanding. Natural language processing is often used in technical fields such as machine translation, handwritten and printed character recognition, speech recognition and text-to-speech conversion, information intent recognition, information extraction and filtering, text classification and clustering, public opinion analysis and opinion mining. It involves data mining related to language processing, machine learning, knowledge acquisition, knowledge engineering, artificial intelligence research, and linguistic research related to language computing.

[0060] Image description generates a natural language description for an image and uses the generated description to help applications understand the semantics expressed in the image visual scene. For example, image description can convert image retrieval into text retrieval, which is used to classify images and improve image retrieval results. People usually only need to take a quick glance to describe the details of the image visual scene, while automatically adding descriptions to images is a comprehensive and arduous computer vision task that requires converting the complex information contained in the image into a natural language description. Compared with ordinary computer vision tasks, image captioning requires not only identifying objects from images, but also associating the identified objects with natural semantics and describing them in natural language. Therefore, image description requires people to extract deep features of images, associate them with semantic features, and convert them for generating descriptions.

[0061] In the process of claims settlement in the breeding industry, it is necessary to identify the insured animals and determine the categories of the insured animals. However, most existing animal recognition technologies rely on traditional image processing methods, which are often unable to effectively handle complex recognition tasks, resulting in low recognition accuracy under complex recognition tasks. Therefore, how to improve the recognition accuracy of animal images has become an urgent problem to be solved.

[0062] Based on this, the embodiments of the present application provide an animal identification method and device, an electronic device and a storage medium, aiming to improve the recognition accuracy of animal images.

[0063] An animal identification method and device, electronic device and storage medium provided in the embodiments of the present application are specifically described through the following embodiments. First, the animal identification method in the embodiments of the present application is described.

[0064] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0065] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0066] The animal identification method provided in the embodiment of the present application relates to the field of image processing technology. The animal identification method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or can be configured as a server cluster or a distributed system composed of multiple physical servers, or can be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the animal identification method, etc., but is not limited to the above forms.

[0067] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0068] Figure 1 is an optional flow chart of the animal identification method provided in the embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S105.

[0069] Step S101, obtaining a sample animal image having a sample animal and a sample image description text of the sample animal image; wherein the sample image description text is used to describe the animal features of the sample animal in the sample animal image;

[0070] Step S102, extracting features from the sample animal image to obtain feature information of the sample animal;

[0071] Step S103, training a preset feature conversion model using sample image description text and sample animal feature information to obtain a target feature conversion model;

[0072] Step S104, acquiring target animal feature information of the target animal image, and performing feature conversion on the target animal feature information through a target feature conversion model to obtain a target image description text;

[0073] Step S105 , performing animal recognition on the target animal image according to the target image description text to obtain the target image animal information.

[0074] In steps S101 to S105 shown in the embodiment of the present application, sample animal images and sample image description texts are obtained, and then feature extraction is performed on the animal images to obtain sample animal feature information. Then, a preset feature conversion model is trained through the sample image description text and the sample animal feature information, so that the preset feature conversion model has the ability to convert image features into text features, thereby obtaining a target conversion feature model. Furthermore, target animal feature information of the target animal image is obtained, and then the target animal feature information is converted into a target image description text through the target feature conversion model, and the target animal image is identified based on the target image description text to obtain target animal image information. Animal images are identified from the dimension of text, and complex recognition tasks are implemented through the dimension of text, thereby improving the recognition accuracy of animal images.

[0075] In step S101 of some embodiments, the sample animal image refers to a digital image containing one or more specific animals, and the source of the image includes but is not limited to a wild environment, a zoo, a farm, or a laboratory. The sample image description text is a detailed description of the animal contained in the sample animal image, such as the animal's species, characteristics, and behavior, for example: "The picture shows an adult male lion lying on the grassland. The lion's hair is dark gold, and its eyes are vigilantly observing the surroundings. The background is sparse grass and a few distant trees.".

[0076] See also Figure 2In some embodiments, step S102 may include but is not limited to steps S201 to S205:

[0077] Step S201, performing image segmentation on the sample animal image to obtain an animal segmentation image;

[0078] Step S202, performing target positioning according to the animal segmentation image to obtain animal position features;

[0079] Step S203, performing color extraction based on the animal segmentation image to obtain animal color features;

[0080] Step S204, performing classification prediction based on the animal segmentation image to obtain classification prediction information;

[0081] Step S205, performing feature aggregation based on animal position features, animal color features and classification prediction information to obtain sample animal feature information.

[0082] In steps S201 to S205 shown in the embodiment of the present application, the sample animal image is segmented to obtain an animal segmentation image, and then the target is located, color is extracted, and classification prediction is performed based on the animal segmentation image to obtain animal position features, animal color features, and classification prediction information. Finally, the animal position features, animal color features, and classification prediction information are feature aggregated to obtain sample animal feature information, thereby extracting features from the image in a comprehensive manner in multiple dimensions, thereby providing a data basis for subsequent conversion into text features based on image features, thereby improving the accuracy of subsequent recognition of the target image.

[0083] See also Figure 3 In some embodiments, step S201 may include but is not limited to steps S301 to S305:

[0084] Step S301, performing image convolution on the sample animal image to obtain an animal convolution feature map;

[0085] Step S302, performing sliding window selection on the animal convolution feature map through a preset sliding window to obtain multiple candidate prediction boxes;

[0086] Step S303, performing image prediction according to the plurality of candidate prediction frames to obtain image prediction data; wherein the image prediction data is used to indicate whether the image in the candidate prediction frame is a background image or an animal image;

[0087] Step S304, screening multiple candidate prediction frames according to the image prediction data to obtain an animal prediction frame;

[0088] Step S305, coordinate correction is performed on the animal prediction frame to obtain an animal segmentation image.

[0089] In steps S301 to S305 shown in the embodiment of the present application, image convolution is performed on the sample animal image to obtain an animal convolution feature map, and then the animal convolution feature map is selected through a preset sliding window to obtain multiple candidate prediction frames, and then image prediction is performed based on the multiple candidate prediction frames to obtain image prediction data, and the multiple candidate prediction frames are screened based on the image prediction data to obtain an animal prediction frame, and finally the coordinates of the animal prediction frame are corrected to obtain an animal segmentation image, thereby accurately extracting the animal segmentation image from the sample animal image.

[0090] In step S301 of some embodiments, image convolution is to convolve the sample animal image through a preset convolutional neural network, and convolution processing is performed on the sample animal image through the convolutional neural network to obtain an animal convolution feature map.

[0091] In step S302 of some embodiments, the animal feature map is slid through a pre-set sliding window to obtain multiple candidate prediction boxes. It should be noted that there are multiple preset sliding windows, and the window size of each sliding window is different, for example, a 3×3 preset sliding window, a 1×3 sliding window or a 2×4 sliding window.

[0092] In step S303 of some embodiments, multiple candidate prediction boxes obtained by sliding window box selection are input into a preset classification model to perform classification prediction on the candidate prediction boxes, and the probability data of each candidate prediction box belonging to an animal image, i.e., image prediction data, is output. It should be noted that there are only two possible branches for the candidate prediction box, background image or animal image. Among them, the background image is an image that does not contain any animal image at all.

[0093] In step S304 of some embodiments, the image prediction data is compared with a preset image prediction threshold. When the image prediction data is greater than the image prediction threshold, the candidate prediction box is determined to be an animal prediction box. When the image prediction data is less than the image prediction threshold, the animal prediction box is determined to be a background prediction box.

[0094] In step S305 of some embodiments, the coordinates are corrected to merge and remove duplicates of multiple animal prediction frames. For example, for an animal in a sample animal image, there will be multiple animal prediction frames framed by it. Based on non-maximum suppression, it is determined to remove duplicates and merge the multiple animal prediction frames, so as to screen out a suitable animal prediction frame that is complete and has a size equal to that of the animal in the image, and the coordinates of the screened animal prediction frame are used as the coordinates of the animal segmentation image, and the animal segmentation image is segmented from the sample animal image according to the coordinates.

[0095] In step S202 of some embodiments, the target positioning is to obtain the coordinate points of the animal segmentation image in the sample animal image. The animal segmentation image is a square shape and is part of the sample animal image. The coordinate points of the animal segmentation image in the sample animal image are directly obtained to obtain the animal position features.

[0096] See also Figure 4 In some embodiments, step S203 may include but is not limited to steps S401 to S403:

[0097] Step S401, performing color space conversion on the animal segmentation image to obtain color space data;

[0098] Step S402, performing two-dimensional discrete cosine calculation on the color space data to obtain two-dimensional discrete cosine data;

[0099] Step S403, performing frequency domain feature extraction based on the two-dimensional discrete cosine data to obtain animal color features.

[0100] In steps S401 to S403 shown in the embodiment of the present application, color space data is obtained by performing color space conversion on the animal segmentation image, and then two-dimensional discrete cosine calculation is performed on the color space data to obtain two-dimensional discrete cosine data. Finally, frequency domain feature extraction is performed based on the two-dimensional discrete cosine data to extract the frequency domain features of the animal segmentation image from the frequency domain dimension, and obtain animal color features that have good expression capabilities for the appearance of the animal in the animal segmentation image.

[0101] In step S401 of some embodiments, the color space is converted to obtain pixel points on the RGB channel of the animal segmentation image, and then converted into a brightness-blue chromaticity-red chromaticity color space based on the pixel points on the RGB channel, that is, color space data.

[0102] In step S402 of some embodiments, two-dimensional discrete cosine calculation is performed on the color space data to convert the color space into a frequency domain space, thereby obtaining two-dimensional discrete cosine data.

[0103] In step S403 of some embodiments, frequency domain feature extraction is to perform spectrum analysis on two-dimensional discrete cosine data to obtain the intensity of different frequency components in the two-dimensional discrete cosine data and obtain animal color features. It should be noted that for animal images, the feathers, fur and skin of animals have unique textures, which are expressed as specific distributed frequencies in the frequency domain. By performing frequency domain feature extraction on two-dimensional discrete cosine data, animal color features have better expression capabilities for animal segmentation images.

[0104] See also Figure 5In some embodiments, step S204 includes but is not limited to steps S501 to S503:

[0105] Step S501, performing morphological extraction on the animal segmentation image by using a preset morphological extraction model to obtain animal morphological features;

[0106] Step S502, extracting texture from the animal segmentation image using a preset texture extraction model to obtain animal texture features;

[0107] Step S503, performing classification prediction based on the animal morphological features and the animal texture features to obtain classification prediction information.

[0108] In steps S501 to S503 shown in the embodiment of the present application, morphology extraction is performed on the animal segmentation image through a preset morphology extraction model to obtain animal morphological features, and then texture extraction is performed on the animal segmentation image through a preset texture extraction model to obtain animal texture features. Finally, classification prediction is performed based on the animal morphological features and animal texture features to obtain classification prediction information, and the animal category in the animal segmentation image is predicted from two dimensions to accurately predict the animal category in the animal segmentation image.

[0109] In step S501 of some embodiments, an opening operation is performed on the animal segmentation image to ignore the local features of the animal, such as small hairs and burrs, and enhance the features of the overall contour of the animal. Then, edge detection is performed on the image after the opening operation to obtain the morphological features of the animal in the animal segmentation image.

[0110] In step S502 of some embodiments, a closing operation is performed on the animal segmented image to enhance local details such as animal hair and burrs, and then a gray level co-occurrence matrix (GLCM) analysis is performed on the image after the closing operation to obtain animal texture features.

[0111] In step S503 of some embodiments, the classification prediction is to first splice the animal morphological features and the animal texture features together, then perform KNN classification with the pre-classified clustering areas, determine the distance between the animal segmentation image and the center of each clustering area, and finally select the classification corresponding to the smallest distance as the category of the sample animal, thereby obtaining classification prediction information.

[0112] In step S205 of some embodiments, feature aggregation is to perform feature aggregation on animal position features, animal color features, and classification features through a preset fully connected layer and an activation function. First, the animal position features, animal color features, and classification prediction information are spliced ​​together and input into a preset fully connected layer, and then the output of the fully connected layer is input into the activation function to obtain sample animal information. It should be noted that this application does not limit the number of fully connected layers.

[0113] In step S103 of some embodiments, the sample animal feature information and the preset sample conversion instruction text are input into the preset feature conversion model for feature conversion to obtain the sample prediction description text. The sample conversion instruction text is used to indicate the conversion of the sample animal feature information into the sample image description text. Then, the mean square error loss value is calculated based on the sample image description text and the sample description prediction text, and the parameters of the preset feature conversion model are optimized based on the loss value to obtain the target feature conversion model.

[0114] See also Figure 6 In some embodiments, the target feature conversion model includes a text input layer, an image input layer and a feature alignment layer, and step S104 includes but is not limited to steps S601 to S603:

[0115] Step S601, embedding a preset conversion instruction text through a text input layer to obtain text embedding feature information; wherein the preset conversion instruction text is used to indicate feature conversion of target animal feature information;

[0116] Step S602, feature encoding the target animal feature information through the image input layer to obtain image encoding feature information;

[0117] Step S603, aligning the image coding feature information and the text embedding feature information through a feature alignment layer to obtain a target image description text.

[0118] In steps S601 to S603 shown in the embodiment of the present application, text embedding is performed on the preset conversion instruction text through the text input layer to obtain text embedding feature information, and then feature encoding is performed on the target animal feature information through the image input layer to obtain image encoding feature information. Finally, feature alignment is performed on the image encoding feature information and the text embedding feature information through the feature alignment layer to realize the conversion from image features to text features, thereby providing a data basis for subsequent animal recognition based on the text dimension.

[0119] In step S601 of some embodiments, the target feature conversion model is a Q-Former model, which has the ability to convert image features into text features. The Q-Former includes a text input layer, an image input layer, and a feature alignment layer. The text input layer is used to perform text embedding on the preset conversion instruction text, that is, to convert the text vector into a high-dimensional space vector to obtain text embedding feature information;

[0120] In step S602 of some embodiments, feature encoding is to perform feature encoding on the target animal feature information through an image input layer, converting the dimension of the target animal feature information into the same dimension as the text embedded feature information to obtain image encoding feature information.

[0121] In step S603 of some embodiments, feature alignment is to align the image coding feature information and the text embedding feature information, and output the target image description text based on the aligned information. The image coding feature information and the text embedding feature information are input to the feature alignment layer, and the feature alignment layer converts the image coding feature information into the target image description text based on the text embedding feature information.

[0122] For example, the preset conversion instruction text is: identify the category, action and color of the animal in the picture. Then the preset conversion instruction text is embedded to obtain text embedding feature information, and then the image encoding feature information is aligned based on the text embedding feature information, and the target image description text is output.

[0123] See also Figure 7 In some embodiments, step S105 may include but is not limited to steps S701 to S702:

[0124] Step S701, performing semantic understanding on the target image description text through a preset semantic understanding model to obtain semantic information of the target text;

[0125] Step S702: performing animal recognition on the target animal image based on the target text semantic information to obtain the target image animal information.

[0126] In steps S701 to S702 shown in the embodiment of the present application, semantic understanding is performed on the target image description text through a preset semantic understanding model to obtain the target text semantic information, and then the target image is identified as an animal based on the target text semantic information to obtain the target image animal information. The target animal image is identified in the form of text, which improves the recognition accuracy of animal images in complex recognition tasks from the perspective of natural language.

[0127] In step S701 of some embodiments, the preset semantic understanding model is a preset generative model with natural language semantic understanding capability, such as a conversational robot. The target image description text is input into the preset semantic understanding model, so that the preset semantic understanding model performs semantic understanding to obtain the target text semantic information.

[0128] In step S702 of some embodiments, animal recognition is performed on the target animal image based on the semantic information of the target text. For example, a complex recognition task is: "Help me find the monkey with scars on its face in the picture", and the preset semantic understanding model performs animal recognition based on the semantic information of the target text and the recognition task to find the target animal image information. For example, the preset semantic understanding model outputs: "The third monkey from left to right in the third picture is the target animal, the color is golden, and the action is jumping." It should be noted that animal recognition based on the semantic understanding model can be directly executed after the recognition task is changed, without the need for re-training. For example, the first recognition task is "Help me find the monkey with scars on its face in the picture", and the second recognition task is "Help me find the mother monkey with a baby monkey in the picture". The preset semantic understanding model can directly perform the recognition task again based on the semantic information of the target text without the need to repeatedly train the model.

[0129] See also Figure 8 The present application also provides an animal identification device that can implement the above-mentioned animal identification method. The device includes:

[0130] The data acquisition module 801 is used to acquire a sample animal image having a sample animal and a sample image description text of the sample animal image; wherein the sample image description text is used to describe the animal features of the sample animal in the sample animal image;

[0131] A feature extraction module 802 is used to extract features from the sample animal image to obtain feature information of the sample animal;

[0132] The model training module 803 is used to train the preset feature conversion model through the sample image description text and the sample animal feature information to obtain the target feature conversion model;

[0133] A feature conversion module 804 is used to obtain target animal feature information of the target animal image, and perform feature conversion on the target animal feature information through a target feature conversion model to obtain a target image description text;

[0134] The animal recognition module 805 is used to perform animal recognition on the target animal image according to the target image description text to obtain the target image animal information.

[0135] The specific implementation of the animal identification device is basically the same as the specific implementation of the above-mentioned animal identification method, and will not be repeated here.

[0136] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above-mentioned animal identification method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0137] See also Fig. 9 , Fig. 9 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0138] The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0139] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the animal identification method of the embodiment of this application;

[0140] Input / output interface 903, used to implement information input and output;

[0141] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WI FI, Bluetooth, etc.);

[0142] A bus 905 that transmits information between the various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);

[0143] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0144] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned animal identification method is implemented.

[0145] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0146] The animal identification method, animal identification device, electronic device and storage medium provided in the embodiments of the present application obtain sample animal images and sample image description text, then perform feature extraction on the animal images to obtain sample animal feature information, and then train a preset feature conversion model through the sample image description text and the sample animal feature information, so that the preset feature conversion model has the ability to convert image features into text features, thereby obtaining a target conversion feature model; further, target animal feature information of the target animal image is obtained, and then the target animal feature information is converted into a target image description text through the target feature conversion model, and animal identification is performed on the target animal image based on the target image description text to obtain target animal image information, animal images are identified from the dimension of text, complex identification tasks are implemented through the dimension of text, and the recognition accuracy of animal images is improved.

[0147] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0148] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0149] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0150] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0151] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0152] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0153] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0154] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0155] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0156] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0157] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A method for animal identification, characterized in that: The method comprises: Acquire a sample animal image having a sample animal and a sample image description text of the sample animal image; wherein the sample image description text is used to describe the animal features of the sample animal in the sample animal image; Extracting features from the sample animal image to obtain sample animal feature information; The preset feature conversion model is trained by using the sample image description text and the sample animal feature information to obtain a target feature conversion model; Acquire target animal feature information of the target animal image, and perform feature conversion on the target animal feature information through the target feature conversion model to obtain a target image description text; The target animal image is subjected to animal recognition according to the target image description text to obtain the target image animal information.

2. The method according to claim 1, characterized in that The step of extracting features from the sample animal image to obtain sample animal feature information includes: Performing image segmentation on the sample animal image to obtain an animal segmentation image; Performing target positioning according to the animal segmentation image to obtain animal position features; Performing color extraction based on the animal segmentation image to obtain animal color features; Perform classification prediction based on the animal segmentation image to obtain classification prediction information; Feature aggregation is performed based on the animal position features, animal color features and the classification prediction information to obtain the sample animal feature information.

3. The method according to claim 2, characterized in that The step of performing image segmentation on the sample animal image to obtain an animal segmentation image includes: Performing image convolution on the sample animal image to obtain an animal convolution feature map; Perform sliding window selection on the animal convolution feature map through a preset sliding window to obtain multiple candidate prediction boxes; Perform image prediction according to the plurality of candidate prediction frames to obtain image prediction data; wherein the image prediction data is used to indicate whether the image in the candidate prediction frame is a background image or an animal image; Screening multiple candidate prediction frames according to the image prediction data to obtain an animal prediction frame; The coordinates of the animal prediction frame are corrected to obtain the animal segmentation image.

4. The method according to claim 2, characterized in that: The step of performing color extraction based on the animal segmentation image to obtain animal color features includes: Performing color space conversion on the animal segmentation image to obtain color space data; Performing two-dimensional discrete cosine calculation on the color space data to obtain two-dimensional discrete cosine data; Frequency domain feature extraction is performed based on the two-dimensional discrete cosine data to obtain the animal color feature.

5. The method according to claim 2, characterized in that: The step of performing classification prediction based on the animal segmentation image to obtain classification prediction information includes: Performing morphological extraction on the animal segmented image by using a preset morphological extraction model to obtain animal morphological features; Performing texture extraction on the animal segmentation image by using a preset texture extraction model to obtain animal texture features; Classification prediction is performed according to the animal morphological features and the animal texture features to obtain the classification prediction information.

6. The method according to any one of claims 1 to 5, characterized in that: The target feature conversion model includes a text input layer, an image input layer and a feature alignment layer. The target feature conversion model is used to convert the feature information of the target animal to obtain a target image description text, including: The preset conversion instruction text is embedded through the text input layer to obtain text embedding feature information; wherein the preset conversion instruction text is used to instruct the target animal feature information to perform feature conversion; Performing feature encoding on the target animal feature information through the image input layer to obtain image encoding feature information; The image encoding feature information and the text embedding feature information are feature aligned through the feature alignment layer to obtain the target image description text.

7. The method according to any one of claims 1 to 5, characterized in that: The step of performing animal recognition on the target animal image according to the target image description text to obtain the target image animal information includes: Performing semantic understanding on the target image description text through a preset semantic understanding model to obtain semantic information of the target text; The target animal image is subjected to animal recognition based on the target text semantic information to obtain the target image animal information.

8. An animal identification device, characterized in that: The device comprises: A data acquisition module, used to acquire a sample animal image having a sample animal and a sample image description text of the sample animal image; wherein the sample image description text is used to describe the animal features of the sample animal in the sample animal image; A feature extraction module, used to extract features from the sample animal image to obtain feature information of the sample animal; A model training module, used to train a preset feature conversion model through the sample image description text and the sample animal feature information to obtain a target feature conversion model; A feature conversion module, used for acquiring target animal feature information of a target animal image, and performing feature conversion on the target animal feature information through the target feature conversion model to obtain a target image description text; The animal recognition module is used to perform animal recognition on the target animal image according to the target image description text to obtain the target image animal information.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the animal identification method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the animal identification method according to any one of claims 1 to 7 is implemented.

Citation Information

Patent Citations

  • Model training method, image description generation method and device, equipment and medium

    CN114723996A

  • Image description generation method and device, electronic equipment and storage medium

    CN114781393A

  • Animal husbandry image recognition method and device based on deep learning

    CN116452792A