Target identification method and device, electronic device, and storage medium

Abstract

The present disclosure relates to a target identification method and device, an electronic device and a storage medium. The method comprises: obtaining a first spectral image of a target region by a spectral camera; detecting the spectral image to obtain a first position of a target object in the spectral image; determining a target shooting angle of a color camera according to the first position; shooting according to the target shooting angle by the color camera to obtain a first color image of the target object; and performing identification processing on the target object in the first color image to obtain an identification result. The target identification method according to the embodiments of the present disclosure can simultaneously utilize the characteristics of a larger field of view of the spectral camera and the characteristics of a high resolution of the color camera, shoot a large range of spectral images by the spectral camera with a larger field of view to quickly determine the position of the target object, and shoot a color image of the position where the target object is located by the color camera with a high resolution, thereby improving the accuracy of identifying the target object.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and particularly relates to a target identification method and device, electronic equipment and storage medium. BACKGROUND

[0002] Target identification is to find a target object of interest based on sensor data. In the related art, a neural network can be used to identify the target object in an image. When the distance to the target is relatively far, such as in the scenarios of unmanned aerial vehicle aerial photography, satellite remote sensing, etc., the range of the target region can be very large, and a large field of view camera must be used to cover a larger range. However, in the large field of view camera, the corresponding pixels of the target object on the imaging surface are relatively few, and even become spots without geometric shapes. Too little information can be obtained from the image, which greatly reduces the accuracy of target identification. One solution is to lose part of the field of view and use a high-definition camera to take pictures and identify targets. However, in this solution, the same scene must be photographed and identified multiple times, which can improve the accuracy to a certain extent, but reduces the time efficiency of identification. SUMMARY

[0003] The present disclosure provides a target identification method and device, electronic equipment and storage medium.

[0004] According to an aspect of the present disclosure, a target identification method is provided, including: obtaining a first spectral image of a target region by a spectral camera; performing detection processing on the spectral image to obtain a first position of a target object in the spectral image; determining a target shooting angle of a color camera according to the first position, wherein the shooting angle is a shooting angle of the target object in a field of view of the color camera, and a field of view of the spectral camera is greater than that of the color camera; performing shooting on the target object according to the target shooting angle by the color camera to obtain a first color image of the target object; and performing identification processing on the target object in the first color image to obtain an identification result.

[0005] In a possible implementation, the method further includes: calibrating the spectral camera and the color camera to obtain a first intrinsic parameter matrix of the spectral camera and a second intrinsic parameter matrix of the color camera; and determining a transformation matrix between the spectral camera and the color camera according to the first intrinsic parameter matrix and the second intrinsic parameter matrix.

[0006] In a possible implementation, the transformation matrix between the spectral camera and the color camera includes a position transformation matrix between second position information of a target position in second color images captured by the color camera at multiple shooting angles and corresponding third position information in a second spectral image captured by the spectral camera, and the transformation matrix between the spectral camera and the color camera is determined according to the first intrinsic parameter matrix and the second intrinsic parameter matrix, including: determining a translation vector and a rotation matrix according to the second position information in the second color image captured by the color camera at a first shooting angle, the corresponding third position information in the second spectral image, and the first intrinsic parameter matrix, the first shooting angle being an arbitrary shooting angle of the color camera; and determining a position transformation matrix corresponding to the first shooting angle according to the second position information, the third position information, the translation vector, and the rotation matrix.

[0007] In a possible implementation, the target shooting angle of the color camera is determined according to the first position, including: determining a corresponding position of a target position in a color image captured by the color camera at multiple shooting angles in the first spectral image according to the transformation matrix; determining a second position with a minimum distance from the first position in the corresponding position; and determining a shooting angle corresponding to the second position as the target shooting angle.

[0008] In a possible implementation, the target position is a center position of the color image captured by the color camera.

[0009] In a possible implementation, the target object in the first color image is identified to obtain an identification result, including: determining a target region of the target object in the first color image; and identifying the target object in the target region to obtain an identification result.

[0010] In a possible implementation, the target region of the target object in the first color image is determined, including: determining a target region of the target object in the first color image according to the transformation matrix and position information of the first position.

[0011] According to an aspect of the present disclosure, a target recognition device is provided, comprising: a spectral image module configured to obtain a first spectral image of a target region by a spectral camera; a detection module configured to perform detection processing on the spectral image to obtain a first position of a target object in the spectral image; an angle module configured to determine a target shooting angle of the color camera according to the first position, wherein the shooting angle is a shooting angle of the target object in a field of view of the color camera, and a field of view of the spectral camera is larger than that of the color camera; a color image module configured to obtain a first color image of the target object by shooting the target object according to the target shooting angle by the color camera; and a recognition module configured to perform recognition processing on the target object in the first color image to obtain a recognition result.

[0012] In a possible implementation, the device further comprises: a calibration module configured to calibrate the spectral camera and the color camera to obtain a first intrinsic parameter matrix of the spectral camera and a second intrinsic parameter matrix of the color camera; and a transformation matrix module configured to determine a transformation matrix between the spectral camera and the color camera according to the first intrinsic parameter matrix and the second intrinsic parameter matrix.

[0013] In a possible implementation, the transformation matrix between the spectral camera and the color camera comprises a position transformation matrix between second position information of a target position in second color images shot by the color camera at multiple shooting angles and corresponding third position information in a second spectral image shot by the spectral camera, and the transformation matrix module is further configured to: determine a translation vector and a rotation matrix according to the second position information in the second color image shot by the color camera at a first shooting angle, the corresponding third position information in the second spectral image, and the first intrinsic parameter matrix, wherein the first shooting angle is an arbitrary shooting angle of the color camera; and determine the position transformation matrix corresponding to the first shooting angle according to the second position information, the third position information, the translation vector, and the rotation matrix.

[0014] In a possible implementation, the angle module is further configured to: determine corresponding positions of the target position in the color images shot by the color camera at multiple shooting angles in the first spectral image according to the transformation matrix; determine a second position with a minimum distance from the first position in the corresponding positions; and determine the shooting angle corresponding to the second position as the target shooting angle.

[0015] In a possible implementation, the target position is a center position of the color image shot by the color camera.

[0016] In a possible implementation, the identification module is further configured to: determine a target region of the target object in the first color image; and perform identification processing on the target object in the target region to obtain an identification result.

[0017] In a possible implementation, the identification module is further configured to: determine a target region of the target object in the first color image according to the transformation matrix and position information of the first position.

[0018] According to an aspect of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the target identification method.

[0019] According to an aspect of the present disclosure, a computer-readable storage medium is provided, which stores computer program instructions, and the computer program instructions are executed by a processor to implement the target identification method.

[0020] The target identification method according to the embodiments of the present disclosure can simultaneously utilize the characteristics of a larger field of view of a spectral camera and the characteristics of a high resolution of a color camera, capture a large range of spectral images by the spectral camera with a larger field of view to quickly determine the position of a target object, determine the shooting angle of the color camera by a transformation matrix, and then capture a color image including the target object by the color camera at the shooting angle. The color image has a higher resolution and the target object is clear, and the target object can be further identified according to the color image to improve the identification accuracy. In summary, the larger field of view of the spectral camera can be utilized to improve the efficiency of determining the position of the target object, and the high resolution of the color camera can be utilized to improve the accuracy of identifying the target object.

[0021] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, rather than limiting the present disclosure.

[0022] Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the drawings. BRIEF DESCRIPTION OF DRAWINGS

[0023] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present disclosure and serve to explain the technical solutions of the present disclosure together with the specification.

[0024] Figure 1 A flow chart of a target identification method according to an embodiment of the present disclosure is shown;

[0025] Figure 2 An application schematic diagram of a target identification method according to an embodiment of the present disclosure is shown;

[0026] Figure 3 a block diagram of a target recognition device according to an embodiment of the present disclosure is shown;

[0027] Figure 4 a block diagram of an electronic device according to an embodiment of the present disclosure is shown;

[0028] Figure 5 a block diagram of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0029] Various exemplary embodiments, features and aspects of the present disclosure will be explained in detail below with reference to the accompanying drawings. The same reference numerals in the drawings denote the same or similar elements. Although various aspects of the embodiments are illustrated in the drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

[0030] The term "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any implementation described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other implementations.

[0031] The term "and / or", merely describes association relationship of associated objects, and means that three relationships can exist, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. In addition, the term "at least one" herein means any one of multiple or any combination of at least two of multiple, for example, including at least one of A, B and C can mean including any one or more elements selected from the set consisting of A, B and C.

[0032] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the specific embodiments below. Those skilled in the art should understand that the present disclosure can also be implemented without some specific details. In some examples, methods, means, elements and circuits well known to those skilled in the art are not described in detail, in order to highlight the main idea of the present disclosure.

[0033] Figure 1 A flowchart of a target recognition method according to an embodiment of the present disclosure is shown, as shown in Figure 1 The method comprises:

[0034] In step S11, a first spectral image of a target region is obtained by a spectral camera;

[0035] In step S12, the spectral image is detected to obtain a first position of a target object in the spectral image;

[0036] In step S13, a target shooting angle of the color camera is determined according to the first position, wherein the shooting angle is a shooting angle of the target object in a field of view of the color camera, and a field of view of the spectral camera is larger than that of the color camera.

[0037] In step S14, a first color image of the target object is obtained by shooting according to the target shooting angle by the color camera.

[0038] In step S15, a recognition result is obtained by performing a recognition process on the target object in the first color image.

[0039] According to the target recognition method of the embodiments of the present disclosure, the characteristics of the large field of view of the spectral camera and the high resolution of the color camera can be utilized simultaneously, a large range of spectral images is shot by the spectral camera with a large field of view to quickly determine the position of the target object, and a color image of the position of the target object is shot by the color camera with high resolution to further recognize the target object. The large field of view of the spectral camera can be utilized to improve the efficiency of determining the position of the target object, and the high resolution of the color camera can be utilized to improve the accuracy of recognizing the target object.

[0040] In a possible implementation, the target recognition method can be executed by an electronic device such as a terminal device or a server. The terminal device can be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The method can be implemented by a processor invoking computer-readable instructions stored in a memory. Alternatively, the target recognition method can be executed by a server.

[0041] In a possible implementation, in the case of unmanned aerial vehicle aerial photography, satellite remote sensing, etc., a large range of area images can be obtained by a camera with a large field of view (for example, a spectral camera), and a target object in the area images can be searched. However, although the large range of area images has a large field of view, the area where the target object is located in the area images can be small, especially in the case of a small target object, the area occupied by the target object in the area images with a large field of view is very small, and even only one or several pixels, so it is difficult to determine the geometric shape of the target object, and it is difficult to recognize the target object. If a high-resolution camera is used, the field of view will be reduced, and the search efficiency will be low when searching for the position of the target object. Therefore, the spectral camera with a large field of view and the color camera with high definition can be combined to efficiently and accurately recognize the target object.

[0042] In a possible implementation, the spectral camera can obtain the spectral information of each position in the target region, that is, each pixel point in the first spectral image captured by the spectral camera can have spectral information, and the spectral information is the information of the multiple lights reflected by the pixel point, which can include rich optical information and can be used to determine the position of the target object. That is, the position of the target object can be determined in the spectral image with a large field of view, without the need to capture multiple images by a high-resolution small field of view camera and find the position of the target object in each image. However, although the spectral image can provide the spectral information of each pixel point, the spectra of different targets can be similar or the same, and therefore, the detection of the target object by using the spectrum is a large-range and coarse-grained detection method, and the detected target object can have errors. Therefore, after the position of the target object is detected, a high-definition color camera can be used to capture a high-definition color image of the position, and the high-definition color image can be used for identification, so that an accurate identification result can be obtained.

[0043] In summary, by using the method of combining the spectral camera with a large field of view and the color camera with high resolution, the position of the target object can be quickly found by using the spectral image with a large field of view, and the image of the position can be captured by using the color camera, without the need to capture multiple images by using the color camera with high resolution to find the position of the target object in each image. In addition, the color camera with high resolution can be used to obtain an accurate identification result.

[0044] In a possible implementation, in step S11, the first spectral image can be a spectral image obtained by a spectral camera with a large field of view arranged on a flight vehicle such as a drone or a satellite, and each pixel point of the first spectral image can represent the spectral information of a position in a large range corresponding to the pixel point, that is, the composition information of the light reflected by the position, that is, which bands of light are included in the light reflected by the position.

[0045] In a possible implementation, in step S12, the first spectral image can be detected and processed by using the spectral information to determine the first position of the target object in the spectral image. For example, because the reflected light of different objects is different, different objects have different shapes, colors, and the like, and the spectral information of the reflected light of different objects is also different. The position of the pixel point with the target spectral information can be determined by using the spectral information of each pixel point. In an example, the spectral information of each pixel point can be compared with the target spectral information to determine the pixel point with the target spectral information. In another example, the target object is an object with multiple reflected lights, for example, the target object has multiple different colors, or the reflected light of the target object at multiple shooting angles is different, and the like. In this case, the first position of the target object in the first spectral image can be determined by using a neural network or the like. The method for detecting the first position is not limited in the present disclosure.

[0046] In a possible implementation, after the first position is determined, the shooting angle of the high-definition color camera can be adjusted so that the target object can appear in the field of view of the color camera with a smaller field of view, that is, so that the color camera can shoot the position where the target object is located after the shooting angle is determined.

[0047] In a possible implementation, the relationship between the spectral camera and the color camera can be determined, that is, the shooting angle of the color camera can be determined according to the relationship and the first position, that is, the target shooting angle. The relationship can be a transformation matrix, that is, a relationship matrix between a certain position in an image shot by the color camera and a corresponding position in an image shot by the spectral camera. The method further includes calibrating the spectral camera and the color camera, obtaining a first intrinsic parameter matrix of the spectral camera, and a second intrinsic parameter matrix of the color camera, and determining the transformation matrix between the spectral camera and the color camera according to the first intrinsic parameter matrix and the second intrinsic parameter matrix.

[0048] In a possible implementation, the spectral camera and the color camera can be calibrated respectively first, and a first intrinsic parameter matrix of the spectral camera and a second intrinsic parameter matrix of the color camera can be obtained. The distortion parameters and other camera parameters of the spectral camera and the color camera can also be determined. The elements in the first intrinsic parameter matrix and the second intrinsic parameter matrix are parameters for describing focal length and optical center position, which can be used as reference parameters of the cameras and for position transformation between the two cameras. In an example, the first intrinsic parameter matrix and the second intrinsic parameter matrix can be obtained by a Zhang Zhengyou plane calibration method or other calibration methods, and the disclosure does not limit the calibration method.

[0049] In a possible implementation, the transformation matrix between the spectral camera and the color camera can be determined by using the first intrinsic parameter matrix and the second intrinsic parameter matrix. In an example, the transformation matrix is usually fixed, that is, the relative position between the two cameras is fixed, so that the relative position relationship of the same position in two images shot by the two cameras can be determined by the transformation matrix. However, the fields of view of the spectral camera and the color camera are different, and the position shot by the spectral camera can need to be shot by the color camera after adjusting the shooting angle, so that the position transformation matrix of the color camera with the spectral camera at multiple shooting angles (that is, the relative position relationship of the same target in two images shot at multiple shooting angles) can be determined.

[0050] In a possible implementation, the transformation matrix between the spectral camera and the color camera includes a position transformation matrix between second position information of a target position in a second color image captured by the color camera at a plurality of shooting angles and corresponding third position information in a second spectral image captured by the spectral camera, and the transformation matrix between the spectral camera and the color camera is determined according to the first intrinsic parameter matrix and the second intrinsic parameter matrix, including: determining a translation vector and a rotation matrix according to the second position information in the second color image captured by the color camera at a first shooting angle, the corresponding third position information in the second spectral image, and the first intrinsic parameter matrix, the first shooting angle being an arbitrary shooting angle of the color camera; and determining a position transformation matrix corresponding to the first shooting angle according to the second position information, the third position information, the translation vector, and the rotation matrix.

[0051] In a possible implementation, the field of view of the spectral camera is greater than that of the color camera, and the range of the area that can be captured is relatively large. The field of view of the color camera is relatively small, and the range of the area that can be captured is relatively small, but the color camera can capture a plurality of areas by adjusting the shooting angle, and the range of the plurality of areas can cover the range of the area captured by the spectral camera. In an example, the long side of the area captured by the color camera is not less than the short side of the area captured by the spectral camera, so that the color camera only needs to adjust the angle of one direction, that is, the pitch angle of the pitch direction, to make the range of the plurality of areas captured cover the range of the area captured by the spectral camera. Otherwise, the color camera needs to change the angles of two directions, for example, the pitch angle and the azimuth angle, to make the range of the plurality of areas captured cover the range of the area captured by the spectral camera.

[0052] In a possible implementation, the shooting angle of the color camera can be set as a first shooting angle θ0 (an arbitrary shooting angle), and a position where a same target in the images captured by the two cameras is selected, for example, a center position of a second color image captured by the color camera and a corresponding position of the center position in a second spectral image captured by the spectral camera can be selected. For example, the target position (for example, the center position) of the second color image captured by the color camera is a position where a target A is located, and the corresponding position is a position where the target A is located in the second spectral image. The position information of the position in the second spectral image is third position information p1(u1, v1), and the position information of the position in the second color image is second position information p2(u2, v2).

[0053] In one possible implementation, the translation vector and rotation matrix can be determined based on the second position information, the third position information, the first intrinsic parameter matrix, and the second intrinsic parameter matrix. In the example, the rotation matrix and translation vector can be determined by the following formula (1):

[0054] s(u1,v1,1)=K2(R(x,y,z)+t) (1)

[0055] Where s is the matrix coefficient, R(x,y,z) is the rotation matrix, t is the translation vector, and K2 is the second intrinsic parameter matrix.

[0056] In the example, the matrix coefficients can be determined according to the following formula (2):

[0057] s(u1,v1,1)=K1(x,y,z) (2)

[0058] Where K1 is the first intrinsic parameter matrix.

[0059] In one possible implementation, the expressions for the translation vector and rotation matrix can be determined according to formulas (1) and (2). Furthermore, the position transformation matrix can be determined based on the second position information, the third position information, the translation vector, and the rotation matrix.

[0060] In the example, the position transformation matrix relating the pixel coordinates of the two cameras can be determined based on the second and third position information. For example, the fundamental matrix can be represented by the following formula (3):

[0061]

[0062] Where F is the position transformation matrix.

[0063] Furthermore, the relationship between the first intrinsic parameter matrix, the second intrinsic parameter matrix, the rotation matrix, the translation vector, and the fundamental matrix can be determined according to the following formula (4):

[0064]

[0065] Furthermore, by combining formulas (3) and (4), the position transformation matrix F can be solved, and F can be determined as the position transformation matrix corresponding to the first shooting angle θ0.

[0066] In one possible implementation, the above method can be used to obtain multiple shooting angles θ. i The corresponding position transformation matrix F i , where i is a positive integer.

[0067] In a possible implementation, in step S13, the first position is a position of the target object in the spectral image. The color camera can be caused to select a target shooting angle such that the target object appears in the field of view of the color camera.

[0068] In a possible implementation, the target shooting angle can be determined according to the first position and the transformation matrix. Step S13 can include: determining, according to the transformation matrix, a corresponding position of a target position in a color image shot by the color camera at a plurality of shooting angles in the first spectral image; determining, in the corresponding position, a second position with a minimum distance from the first position; and determining a shooting angle corresponding to the second position as the target shooting angle.

[0069] In a possible implementation, after the transformation matrix is determined, the corresponding position of the target position in the color image shot at each angle in the first spectral image can be determined based on the transformation matrix. For example, the corresponding position of the target position in the color image shot by the color camera at the shooting angle θ0 in the first spectral image can be determined according to the position transformation matrix F0 corresponding to the shooting angle θ0; the corresponding position of the target position in the color image shot by the color camera at the shooting angle θ1 in the first spectral image can be determined according to the position transformation matrix F1 corresponding to the shooting angle θ1; and so on. In this way, a plurality of corresponding positions corresponding to the respective shooting angles can be determined. In an example, the target position in the color image can be a center position of the color image shot by the color camera. The present disclosure does not limit the selection of the target position.

[0070] In a possible implementation, the corresponding position with a minimum distance from the first position, i.e., the second position, can be determined from the plurality of corresponding positions. For example, the Euclidean distance between the first position and each corresponding position can be determined, and the second position with the minimum Euclidean distance from the first position can be determined. In an example, the second position is one of the plurality of corresponding positions, for example, the corresponding position of the shooting angle θ k (k is a positive integer) corresponding position in response, the shooting angle θ k is the target shooting angle.

[0071] In a possible implementation, in step S14, after the target shooting angle is determined, the color camera can be caused to shoot at the target shooting angle θ k In an example, the second position is the corresponding position of the center point of the image shot by the color camera, and the distance between the second position and the first position is minimum, i.e., the distance between the center point of the image shot by the color camera and the first position is minimum. Therefore, when the color camera shoots at the target shooting angle, the target object at the first position can appear near the center position of the first color image.

[0072] In a possible implementation, in step S15, the target object is presented in the first color image, and the first color image can be subjected to recognition processing to obtain a recognition result. For example, the target object is a person, and the person can be subjected to recognition processing to obtain identity information of the target object, or the target object is an object, and the object can be subjected to recognition processing to determine classification information of the object. The present disclosure does not limit the category of the target object.

[0073] In a possible implementation, step S15 can include determining a target region of the target object in the first color image, and subjecting the target object in the target region to recognition processing to obtain a recognition result.

[0074] In a possible implementation, the target region of the target object in the first color image can be determined. In an example, the first color image can be subjected to detection processing to determine the target region where the target object is located. For example, the detection processing can be performed by using a neural network or the like to determine the target region where the target object is located.

[0075] In a possible implementation, determining the target region of the target object in the first color image includes determining the target region of the target object in the first color image according to the transformation matrix and position information of the first position.

[0076] In an example, the target region in the first color image can also be determined according to the first position and the transformation matrix. For example, the first color image is a color camera, and the target object is photographed at a target shooting angle θ k The color image subjected to photographing is a color image subjected to photographing at a target shooting angle θ k The corresponding position transformation matrix is F k The target region in the first color image can be determined by using F k The first position is subjected to transformation processing to obtain a position of the target region in the first color image.

[0077] In a possible implementation, the target object in the target region can be subjected to recognition processing. For example, the recognition processing can be performed by using a neural network or the like to obtain a recognition result.

[0078] The target identification method according to the embodiments of the present disclosure can simultaneously utilize the large field of view of the spectral camera and the high resolution of the color camera, capture a large range of spectral images by the spectral camera with a large field of view to quickly determine the position of the target object, determine the shooting angle of the color camera by the transformation matrix, and then capture a color image including the target object by the color camera at the shooting angle. The color image has a high resolution and the target object is clear, and the target object can be further identified according to the color image to improve the identification accuracy. In summary, the large field of view of the spectral camera can be utilized to improve the efficiency of determining the position of the target object, and the high resolution of the color camera can be utilized to improve the accuracy of identifying the target object.

[0079] Figure 2 An application diagram of the target identification method according to the embodiments of the present disclosure is shown. As shown in Figure 2 The spectral camera with a large field of view can be arranged on an aircraft such as a drone or a satellite, and a first spectral image of a large range of area can be obtained. The field of view of the color camera with a small field of view is smaller than that of the spectral camera, for example, the color camera only captures a partial area of the area that can be captured by the spectral camera, but the image captured by the color camera has a high resolution and a high definition.

[0080] In a possible implementation, the first position of the target object can be determined according to the spectral information of each pixel point in the spectral image. Then, the shooting angle of the color camera can be determined according to the first position and the transformation matrix between the spectral camera and the color camera, so that the target object can be presented in the field of view of the color camera. For example, the center positions of the color images captured by the color camera at multiple shooting angles in the spectral image can be determined according to the transformation matrix, and the second position closest to the first position in the corresponding positions is determined, and the shooting angle corresponding to the second position is the target shooting angle.

[0081] In a possible implementation, the color image can be captured by the color camera at the target shooting angle, and the target object can be presented in the color image. Further, the first position can be transformed based on the position transformation matrix corresponding to the target shooting angle to determine the target region of the target object in the color image, and the target region is identified to obtain an identification result, for example, the target object is a person, and the identity information of the person can be identified.

[0082] In a possible implementation, the target identification method can be applied to the field of target identification, for example, a narrow field of view push-broom imaging spectrometer is combined with a color camera to identify a target object. The carrier of the spectral camera and the color camera can also be a translation stage or a handheld device. The present disclosure does not limit the application field of the target identification method.

[0083] Figure 3A block diagram of a target recognition device according to an embodiment of the present disclosure is shown as Figure 3 As shown, the device comprises: a spectral image module 11 configured to obtain a first spectral image of a target region by a spectral camera; a detection module 12 configured to perform detection processing on the spectral image to obtain a first position of a target object in the spectral image; an angle module 13 configured to determine a target shooting angle of the color camera according to the first position, wherein the shooting angle is a shooting angle of the target object in a field of view of the color camera, and a field of view of the spectral camera is larger than that of the color camera; a color image module 14 configured to obtain a first color image of the target object by shooting according to the target shooting angle by the color camera; and a recognition module 15 configured to perform recognition processing on the target object in the first color image to obtain a recognition result.

[0084] In a possible implementation, the device further comprises: a calibration module configured to calibrate the spectral camera and the color camera to obtain a first intrinsic parameter matrix of the spectral camera and a second intrinsic parameter matrix of the color camera; and a transformation matrix module configured to determine a transformation matrix between the spectral camera and the color camera according to the first intrinsic parameter matrix and the second intrinsic parameter matrix.

[0085] In a possible implementation, the transformation matrix between the spectral camera and the color camera comprises a position transformation matrix between second position information of a target position in second color images shot by the color camera at multiple shooting angles and corresponding third position information in a second spectral image shot by the spectral camera, and the transformation matrix module is further configured to: determine a translation vector and a rotation matrix according to the second position information in the second color image shot by the color camera at a first shooting angle, the corresponding third position information in the second spectral image, and the first intrinsic parameter matrix, wherein the first shooting angle is an arbitrary shooting angle of the color camera; and determine the position transformation matrix corresponding to the first shooting angle according to the second position information, the third position information, the translation vector, and the rotation matrix.

[0086] In a possible implementation, the angle module is further configured to: determine corresponding positions of the target position in the color images shot by the color camera at multiple shooting angles in the first spectral image according to the transformation matrix; determine a second position with a minimum distance from the first position in the corresponding positions; and determine the shooting angle corresponding to the second position as the target shooting angle.

[0087] In a possible implementation, the target position is a center position of the color image shot by the color camera.

[0088] In a possible implementation, the identification module is further configured to: determine a target region of the target object in the first color image; and perform identification processing on the target object in the target region to obtain an identification result.

[0089] In a possible implementation, the identification module is further configured to: determine a target region of the target object in the first color image according to the transformation matrix and position information of the first position.

[0090] It can be understood that the above-mentioned various method embodiments mentioned in the disclosure can be combined with each other to form combined embodiments without violating the principle logic. Limited to the length of the disclosure, the disclosure will not be described again.

[0091] In addition, the disclosure also provides a target identification apparatus, an electronic device, a computer readable storage medium, and a program, which can be used to implement any one of the target identification methods provided by the disclosure. The corresponding technical solutions and descriptions are described in the method part and are not described again.

[0092] Those skilled in the art can understand that the writing order of the steps in the above-mentioned method of the specific embodiment does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0093] In some embodiments, the apparatus provided by the embodiments of the disclosure has functions or contains modules that can be used to execute the methods described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be described here

[0094] The embodiments of the disclosure also propose a computer readable storage medium having computer program instructions stored thereon, the computer program instructions being executed by a processor to implement the above method. The computer readable storage medium can be a non-volatile computer readable storage medium.

[0095] The embodiments of the disclosure also propose an electronic device, comprising: a processor; a memory for storing processor executable instructions; wherein the processor is configured to perform the above method.

[0096] The electronic device can be provided as a terminal, a server or other forms of devices.

[0097] Figure 4 is a block diagram of an electronic device 800 according to an exemplary embodiment. For example, the electronic device 800 can be a terminal such as a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0098] Referring toFigure 4 The electronic device 800 can include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0099] The processing component 802 usually controls overall operations of the electronic device 800, such as operations associated with displaying, making phone calls, data communications, camera operations and recording operations. The processing component 802 can include one or more processors 820 to execute instructions to complete all or part of steps of the methods described above. In addition, the processing component 802 can include one or more modules to facilitate

[0100] The memory 804 is configured to store various types of data to support operations of the electronic device 800. Examples of these data include instructions for any application or method operating on the electronic device 800, contact data, phonebook data, messages, pictures, videos, and the like. The memory 804 can be implemented by any type of volatile or non-volatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic storage, flash memory, magnetic or optical disk.

[0101] The power supply component 806 provides power for the various components of the electronic device 800. The power supply component 806 can include a power supply management system, one or more power supplies, and other components associated with generating, managing and distributing power for the electronic device 800.

[0102] The multimedia component 808 includes a screen to provide an output interface between the electronic device 800 and a user. In some embodiments, the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense a touch, a slide, and a gesture on the touch panel. The touch sensor can not only sense a boundary of a touching or a sliding action, but also detect duration and intensity of the touching or sliding action. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have a focal length and optical zooming capability.

[0103] The audio component 810 is configured to output and / or input an audio signal. For example, the audio component 810 includes a microphone (MIC) to receive an external audio signal when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 further includes a speaker to output an audio signal.

[0104] The I / O interface 812 provides an interface for the processing component 802 and peripheral interface modules, which can be a keypad, a click wheel, buttons, and the like. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0105] The sensor component 814 includes one or more sensors to provide various state assessments for the electronic device 800. For example, the sensor component 814 can detect an open / closed state of the electronic device 800, relative positioning of components, such as a display and a keypad of the electronic device 800, a change in position of the electronic device 800 or a component of the electronic device 800, presence or absence of user contact with the electronic device 800, an orientation or acceleration / deceleration of the electronic device 800, and a temperature change of the electronic device 800. The sensor component 814 can include a proximity sensor configured to detect presence of a nearby object without any physical touch. The sensor component 814 can further include a light sensor, such as a CMOS or CCD image sensor, for use in an imaging application. In some embodiments, the sensor component 814 can further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0106] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) techniques, infrared data association (IrDA) techniques, ultra-wideband (UWB) techniques, Bluetooth (BT) techniques, and other techniques.

[0107] In an exemplary embodiment, the electronic device 800 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic elements, for performing the above-described methods.

[0108] In an exemplary embodiment, a non-transitory computer-readable storage medium, such as the memory 804 including computer program instructions, is also provided, which can be executed by the processor 820 of the electronic device 800 to complete the above-described methods.

[0109] Figure 5 is a block diagram of an electronic device 1900 according to an exemplary embodiment. For example, the electronic device 1900 can be provided as a server. Referring to Figure 5 , the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932, for storing instructions, such as application programs, executable by the processing component 1922. The application programs stored in the memory 1932 can include one or more than one module each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute the instructions to perform the above-described methods.

[0110] The electronic device 1900 can also include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 can operate based on an operating system stored in the memory 1932, such as a Microsoft server operating system (Windows Server TM ), a graphical user interface-based operating system (Mac OSX TM), multi-user multi-tasking computer operating system (Unix TM ), free and open-source Unix-like operating system (Linux TM ), open-source Unix-like operating system (FreeBSD TM ), or the like.

[0111] In an example embodiment, there is also provided a non-transitory computer- readable storage medium, such as memory 1932 including computer program instructions that are executable by processing component 1922 of electronic device 1900 to perform the above-described methods.

[0112] The present disclosure can be a system, a method, and / or a computer program product. The computer program product can include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present disclosure.

[0113] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or punched tape, a magnetically encoded device such as magnetic strip cards, an optically encoded device such as a compact disc (CD) or DVD, and / or any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0114] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0115] Computer readable program instructions for carrying out operations of the present disclosure can be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate array (FPGA), or programmable logic array (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.

[0116] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0117] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0118] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0119] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0120] Embodiments of the present disclosure have been described above, and the description is intended to be illustrative, and not restrictive, of the disclosed embodiments. Many modifications and variations of the described embodiments are possible, and all such modifications and variations are intended to be within the scope of the described embodiments. The description used herein is intended to be illustrative, and not restrictive, of the described embodiments. The scope of the described embodiments is not limited to the examples and / or embodiments described herein but only by the claims that follow and their equivalents.

Claims

1. A target recognition method characterized by, The method comprises: obtaining a first spectral image of a target region by a spectral camera; performing detection processing on the first spectral image to obtain a first position of a target object in the first spectral image, wherein the first position is determined by using spectral information of each pixel point in the first spectral image to determine the position of a pixel point with target spectral information as the first position in the first spectral image; determining a target shooting angle of a color camera according to the first position, wherein the target shooting angle is a shooting angle of the target object in a field of view of the color camera, and a field of view of the spectral camera is larger than a field of view of the color camera; obtaining a first color image of the target object by shooting according to the target shooting angle by the color camera; performing recognition processing on the target object in the first color image to obtain a recognition result; determining a target shooting angle of a color camera according to the first position comprises: determining a corresponding position of a target position in a color image shot by the color camera at a plurality of shooting angles in the first spectral image according to a transformation matrix, wherein the transformation matrix comprises a relationship matrix between the target position in the color image shot by the color camera and the corresponding position in the spectral image shot by the spectral camera; determining a second position with the smallest distance from the first position in the corresponding position; determining the shooting angle corresponding to the second position as the target shooting angle.

2. The method of claim 1, wherein, The method further comprises: calibrating the spectral camera and the color camera to obtain a first intrinsic parameter matrix of the spectral camera and a second intrinsic parameter matrix of the color camera; determining a transformation matrix between the spectral camera and the color camera according to the first intrinsic parameter matrix and the second intrinsic parameter matrix.

3. The method of claim 2, wherein, The transformation matrix between the spectral camera and the color camera comprises a position transformation matrix between second position information of the target position in a second color image shot by the color camera at a plurality of shooting angles and corresponding third position information in a second spectral image shot by the spectral camera, determining the transformation matrix between the spectral camera and the color camera according to the first intrinsic parameter matrix and the second intrinsic parameter matrix comprises: determining a translation vector and a rotation matrix according to the second position information in the second color image shot by the color camera at a first shooting angle, the corresponding third position information in the second spectral image, and the first intrinsic parameter matrix, wherein the first shooting angle is an arbitrary shooting angle of the color camera; determining a position transformation matrix corresponding to the first shooting angle according to the second position information, the third position information, the translation vector, and the rotation matrix.

4. The method according to claim 1 or 3, characterized in that, The target position is a center position of the color image shot by the color camera.

5. The method of claim 2, wherein, The recognition processing on the target object in the first color image to obtain a recognition result comprises: determining a target region of the target object in the first color image; performing recognition processing on the target object in the target region to obtain a recognition result.

6. The method of claim 5, wherein, Determining a target region of the target object in the first color image comprises: According to the transformation matrix and position information of the first position, a target region of the target object in the first color image is determined.

7. A target recognition device, characterized by The method comprises the following steps: An optical spectrum image module is configured to obtain a first optical spectrum image of a target region by an optical spectrum camera; A detection module is configured to perform detection processing on the first optical spectrum image to obtain a first position of a target object in the first optical spectrum image, wherein the first position is determined by using spectral information of each pixel point in the first optical spectrum image to determine a position of a pixel point with target spectral information as the first position in the first optical spectrum image; An angle module is configured to determine a target shooting angle of a color camera according to the first position, wherein the target shooting angle is a shooting angle of the target object in a field of view of the color camera, and a field of view of the optical spectrum camera is larger than a field of view of the color camera; A color image module is configured to obtain a first color image of the target object by shooting according to the target shooting angle by the color camera; An identification module is configured to perform identification processing on the target object in the first color image to obtain an identification result; The angle module is further configured to: According to a transformation matrix, determine a corresponding position of a target position in a color image shot by the color camera at a plurality of shooting angles in the first optical spectrum image, wherein the transformation matrix comprises a relationship matrix between a position in an image shot by the color camera and a corresponding position in an image shot by the optical spectrum camera; In the corresponding position, determine a second position with a minimum distance from the first position; Determine a shooting angle corresponding to the second position as the target shooting angle.

8. An electronic device, comprising: The method comprises the following steps: A processor; A memory for storing processor-executable instructions; The processor is configured to execute the method in any one of claims 1 to 6.

9. A computer-readable storage medium having stored thereon computer program instructions, wherein, The computer program instructions are executed by the processor to implement the method in any one of claims 1 to 6.

Images