OBJECT IDENTIFICATION METHOD, OBJECT IDENTIFICATION DEVICE, ELECTRONIC DEVICE AND COMPUTER PROGRAM

Abstract

The present application discloses an object identification method, apparatus, electronic device, computer-readable storage medium, and computer program product, which are applicable to various scenarios such as cloud technology, artificial intelligence, and intelligent transportation, and which determine a reference pixel point from each pixel point of an infrared light image, obtain depth information of the reference pixel point, obtain depth information of each pixel point in the infrared light image based on the position information of the reference pixel point and the depth information, perform a position mapping process for each pixel point in the infrared light image based on the depth information corresponding to each pixel point, obtain corresponding mapping point position information in the visible light image, perform an alignment process for the pixel points in the infrared light image and the visible light image based on the mapping point position information, and perform object identification for the object to be identified based on the infrared light image and the visible light image after the alignment process, so as to obtain an object identification result.

Description

[Technical field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application is filed based on a Chinese patent application having application number 202210295192.8 and filing date March 23, 2022, and claims priority to the Chinese patent application, the entire contents of which are incorporated herein by reference.

[0002] The present application relates to the field of computer technology, and in particular to an object identification method, apparatus, electronic device, computer-readable storage medium and computer program product. [Background technology]

[0003] With the development of computer technology, image processing technology is applied to more and more fields, for example, biometric feature identification technology is widely applied to many fields such as entrance and exit management, information security, electronic certificates, etc. Specifically, biometric feature identification technology is a technology that automatically extracts biometric features from an image to be identified, and then performs identity authentication based on these features. In the process of biometric feature identification, infrared light images and visible light images of the object to be identified are usually collected, where the infrared light images can be used to perform live body detection on the object to be identified, and before live body detection, pixel alignment processing needs to be further performed on the collected infrared light images and visible light images.

[0004] Related techniques mainly rely on depth maps to achieve pixel alignment processing for infrared and visible light images, but such methods do not apply to identification scenarios where depth maps are lacking. Summary of the Invention

[0005] Embodiments of the present application provide an object identification method, apparatus, electronic device, computer-readable storage medium, and computer program product that can achieve pixel alignment for infrared light images and visible light images in scenarios where a depth map is missing, thereby performing object identification.

[0006] An embodiment of the present application provides an object identification method executed by an electronic device, the object identification method including: collecting infrared and visible light images of an object to be identified; determining a reference pixel point from each pixel point of the infrared light image, and obtaining depth information of the reference pixel point with respect to the object to be identified; obtaining depth information of each pixel point in the infrared light image with respect to the object to be identified based on position information of the reference pixel point in the infrared light image and the depth information; performing a position mapping process for each pixel point in the infrared light image based on depth information corresponding to each pixel point in the infrared light image, and obtaining corresponding mapping point position information in the visible light image for each pixel point in the infrared light image; performing a registration process on pixel points in the infrared light image and the visible light image based on the mapping point position information; and performing object identification on the object to be identified based on the infrared light image and the visible light image after the alignment process, to obtain an object identification result for the object to be identified.

[0007] An embodiment of the present application provides an object identification device, the object identification device comprising: a collection unit configured to collect infrared and visible light images of the object to be identified; a determining unit configured to determine a reference pixel point from each pixel point of the infrared light image and obtain depth information of the reference pixel point relative to the object to be identified; an acquisition unit configured to acquire depth information of each pixel point in the infrared light image for the object to be identified based on position information of the reference pixel point in the infrared light image and the depth information; a mapping unit configured to perform a position mapping process on each pixel point in the infrared light image based on depth information corresponding to each pixel point in the infrared light image, and obtain corresponding mapping point position information in the visible light image for each pixel point in the infrared light image; a registration unit configured to perform a registration process on pixel points in the infrared light image and the visible light image based on the mapping point position information; and an identification unit configured to perform object identification on the object to be identified based on the infrared light image and the visible light image after the alignment process, and obtain an object identification result for the object to be identified.

[0008] An embodiment of the present application provides an electronic device, the electronic device including a processor and a memory, wherein a plurality of instructions are stored in the memory, and the processor loads the instructions to perform steps in an object identification method provided by an embodiment of the present application.

[0009] An embodiment of the present application further provides a computer-readable storage medium storing a computer program which, when executed by a processor, implements steps in the object identification method provided by the embodiment of the present application.

[0010] An embodiment of the present application further provides a computer program product comprising a computer program or instructions which, when executed by a processor, implements steps in the object identification method provided by an embodiment of the present application.

[0011] The embodiments of the present application obtain depth information of pixel points from an infrared light image, and assign the depth information to pixel points in a visible light image through mapping between the infrared light image and the visible light image, and can restore depth information from images collected by inexpensive sensors based on infrared light and visible light, and can support object identification without relying on expensive depth sensors, and have a wide range of application scenarios. [Brief description of the drawings]

[0012] [Figure 1a] FIG. 2 is a schematic scenario diagram of an object identification method according to an embodiment of the present application; [Figure 1b] 1 is a flowchart of an object identification method according to an embodiment of the present application. [Figure 1c] 1 is a structural schematic diagram of a speckle structured light imaging system according to an embodiment of the present application; [Figure 1d] FIG. 2 is a schematic diagram of a pinhole imaging model according to an embodiment of the present application. [Figure 1e] FIG. 2 is a schematic diagram of a transformation relationship between a camera coordinate system and a pixel coordinate system according to an embodiment of the present application; [Figure 1f] FIG. 2 is a schematic diagram of the relationship between camera coordinate systems according to an embodiment of the present application; [Figure 1g] FIG. 2 is a schematic diagram of the relationship between a camera coordinate system corresponding to an infrared view and a camera coordinate system corresponding to a color view according to an embodiment of the present application. [Figure 1h] 4 is another flowchart of an object identification method according to an embodiment of the present application. [Diagram 2] 4 is another flowchart of an object identification method according to an embodiment of the present application. [Diagram 3] 1 is a structural schematic diagram of an object identification device according to an embodiment of the present application; [Figure 4] FIG. 1 is a structural schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] In order to more clearly describe the technical solutions in the embodiments of the present application, the above briefly introduces the drawings required for the description of the embodiments. Obviously, the drawings described above are only some embodiments of the present application, and those skilled in the art can also obtain other drawings based on these drawings without any creative efforts.

[0014] The technical solutions in the embodiments of the present application are described below clearly and completely with reference to the drawings in the embodiments of the present application, and obviously, the described embodiments are only a part of the embodiments of the present application, and are not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without any creative efforts belong to the protection scope of the present application.

[0015] The embodiments of the present application provide an object identification method and related devices. The related devices can include an object identification device, an electronic device, a computer-readable storage medium and a computer program product. The object identification device can be specifically integrated into an electronic device, and the electronic device can be a device such as a terminal or a server.

[0016] It can be understood that the object identification method of the embodiment of the present application may be executed in a terminal, may be executed in a server, or may be executed jointly by a terminal and a server. The above examples should not be understood as limitations on the present application.

[0017] As shown in Fig. 1a, an object identification method is taken as an example in which a terminal and a server jointly execute the object identification method. The object identification system provided by the embodiment of the present application includes a terminal 10 and a server 11, etc. The terminal 10 and the server 11 are connected via a network, for example, via a wired or wireless network, where the object identification device may be integrated into the terminal.

[0018] Here, the terminal 10 may be configured to collect an infrared light image and a visible light image for an object to be identified, determine a reference pixel point from each pixel point of the infrared light image, obtain depth information of the reference pixel point for the object to be identified, obtain depth information of each pixel point in the infrared light image for the object to be identified based on the position information of the reference pixel point in the infrared light image and the depth information, perform position mapping processing for each pixel point in the infrared light image based on the depth information corresponding to each pixel point in the infrared light image, obtain corresponding mapping point position information in the visible light image for each pixel point in the infrared light image, perform alignment processing for pixel points in the infrared light image and the visible light image based on the mapping point position information, transmit the infrared light image and the visible light image after the alignment processing to a server 11, perform object identification for the object to be identified via the server 11, and obtain an object identification result for the object to be identified. Here, the terminal 10 may include a mobile phone, a smart TV, a tablet computer, a notebook computer, a personal computer (PC: Personal Computer), or the like. A client may be configured in the terminal 10, and the client may be an application program client or a browser client, etc.

[0019] Here, the server 11 may be configured to receive the infrared light image and the visible light image after the alignment process transmitted by the terminal 10, perform object identification on the object to be identified based on the infrared light image and the visible light image after the alignment process, obtain an object identification result for the object to be identified, and transmit the object identification result to the terminal 10. Here, the server 11 may be a single server, or may be a server cluster or a cloud server consisting of multiple servers. In the object identification method or device disclosed in the present application, a blockchain network can be formed by multiple servers, and the servers are nodes on the blockchain network.

[0020] The above steps of object identification by the server 11 may also be performed by the terminal 10.

[0021] The object identification method provided by the embodiment of the present application relates to computer vision technology in the field of artificial intelligence. Artificial intelligence (AI) is a theory, method, technology, and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend, and expand human intelligence, sense the environment, obtain knowledge, and use the knowledge to obtain optimal results. That is, artificial intelligence is a comprehensive technology of computer science that aims to understand the essence of intelligence and produce a new intelligent machine that responds in a manner similar to human intelligence. That is, artificial intelligence studies the design principles and realization methods of various intelligent machines, and gives the machine the functions of sensing, reasoning, and decision-making. Artificial intelligence technology is a comprehensive discipline with a wide range of fields, including hardware technology and software technology. Here, artificial intelligence software technology mainly includes computer vision technology, voice processing technology, natural language processing technology, and several themes such as machine learning / deep learning, autonomous driving, and intelligent transportation.

[0022] Each of the embodiments will be described in detail below. It should be noted that the order of description of the following embodiments is not intended to limit the preferred order of the embodiments.

[0023] Although the embodiments of the present application are described in terms of an object identification device, the object identification device may be specifically integrated into an electronic device, which may be a device such as a server or a terminal.

[0024] It can be understood that, in accordance with specific embodiments of the present application, relevant data such as user information needs to obtain user permission or consent when the above examples of the present application are applied to specific products or technologies, and the collection, use and processing of relevant data needs to comply with the relevant laws, regulations and standards of relevant countries and regions.

[0025] The object identification method of the embodiment of the present application can be applied to various scenarios that require object identification, such as palm payment, entrance and exit management, etc. The embodiment of the present application can be applied to various scenarios such as cloud technology, artificial intelligence, intelligent transportation, assisted driving, etc.

[0026] As shown in FIG. 1b, the specific process of the object identification method is as follows.

[0027] In step 101, infrared and visible light images are collected for the object to be identified.

[0028] Here, the object to be identified may be an object for which biometric identification is required, palm print information corresponding to a human body, a human face, etc., but is not limited to this in the embodiments of the present application.

[0029] Here, the infrared light image is specifically an infrared image imaged by pan-infrared light collected by an infrared sensor, and may be used to perform live body detection on the object to be identified. The visible light image is specifically a two-dimensional color image imaged by natural light collected by an image sensor, and may be used to perform identity identification on the object to be identified. It should be noted that in the embodiment of the present application, the infrared light image and the visible light image of the object to be identified may be collected synchronously, and thus the infrared light image and the visible light image can be associated with each other.

[0030] Specifically, in the process of performing biometric feature identification, it is generally necessary to further obtain a depth image for the object to be identified. The depth image may specifically be a depth map obtained by collecting speckle-structured infrared light by an infrared sensor and further analyzing the speckle by a depth unit. In three-dimensional (3D) computer graphics and computer vision, a depth map is an image or image channel that contains information about the distance from the surface of the collected scenario object to the viewpoint. Each pixel point of the depth map represents the vertical distance between the depth camera plane and the plane of the photographed object, and is usually represented by 16 bits and has a unit of millimeters. In object identification scenarios, the depth image may be generally used for live body detection and auxiliary identity identification. By utilizing the depth image to assist identity identification, the accuracy and robustness of identity identification can be greatly improved.

[0031] Here, biometric detection is a detection method for determining whether the object to be identified is a real person, a photograph, a head mold, etc. In general, it is possible to determine whether the object to be identified is a photograph based on a depth image, and to determine whether the object to be identified is a silicone head mold based on the brightness of an infrared light image.

[0032] Here, the speckle structured light is a lattice light arranged according to a certain structure rule projected by an infrared speckle projector. As shown in Fig. 1c, the speckle structured light imaging system is composed of an infrared laser projector 11 and an infrared sensor 12, and these lattice lights are projected onto the surface 13 of the object (i.e., the reference sample), and after imaging by the infrared sensor, the 3D coordinate information of the object surface can be restored based on the triangulation principle, thereby obtaining a depth image.

[0033] Compared with a depth image, a point cloud image records the 3D coordinate information of an object in the real world, and the 3D coordinate information is obtained by calculating the depth image and camera parameters, and the depth image also contains the depth information of the object, but the horizontal and vertical coordinates of a point in the depth image are pixel coordinates in the imaging plane (hereinafter also referred to as the pixel plane), while the point cloud image records the physical coordinates of the point in the real world. The point cloud image may be displayed in a 3D rendering engine, reflecting the three-dimensional positional relationship of each point.

[0034] In related art, Red Green Blue-Depth (RGB-D) data is usually used for identification. RGB is a color mode and an industry color standard. Various colors can be obtained by adjusting the three color channels of red, green, and blue and overlaying between them. RGB stands for the three channels of red, green, and blue. Here, D stands for Depth Map. In three-dimensional computer graphics, a Depth Map is an image or image channel that contains information about the distance of the surface of a scenario object from the viewpoint. A Depth Map is similar to a grayscale image, but the pixel value of each pixel point is the actual distance from the sensor to the object. Usually, an RGB image (i.e., a visible light image) and a Depth image are mapped, so that there is a one-to-one correspondence between the pixel points of the RGB image and the pixel points of the Depth image.

[0035] However, in some identity identification scenarios, only visible light images and infrared light images can be collected, and the depth image is lacking, so the pixel-level alignment of the three images cannot be realized by relying on the depth image, where the pixel point-level alignment of the three images refers to the one-to-one correspondence between the pixel points of the visible light image, the infrared light image, and the depth image. The present application can provide an object identification method, which can realize the pixel-level alignment of the visible light image and the infrared light image without relying on the depth image.

[0036] The object identification method provided in this application can be applied to the field of palmprint identification, and palmprint identification is a biometric feature identification technology. Palmprint identification can extract palmprint information of the object to be identified, and palmprint information specifically refers to palm image information from the fingertips to the wrist. Because the palmprint features of different objects are different, identity authentication can be performed based on the palmprint information.

[0037] Here, biometric identification technology specifically refers to a technology that closely combines high-tech means such as computers, optics, acoustics, biological sensors, and biostatistics principles to identify individuals by utilizing physiological characteristics unique to the human body (e.g., fingerprints, face, irises, etc.) and behavioral characteristics (e.g., handwriting, voice, walking style, etc.).

[0038] In step 102, a reference pixel point is determined from each pixel point of the infrared light image, and depth information of the reference pixel point relative to the object to be identified is obtained.

[0039] In some embodiments, three pixel points that are not collinear with a pixel point of the infrared light image can be arbitrarily selected as the reference pixel point.

[0040] For example, the reference pixel points include at least three pixel points that are not on the same line.

[0041] In a specific scenario, for the depth information of the reference pixel point for the object to be identified, a component proximity sensor (Psensor: Proximity Sensor) can be specifically used to obtain the depth information corresponding to each reference pixel point. Specifically, the number of reference pixel points is first determined, and the number of Psensors can be determined according to the number of reference pixel points, and each Psensor is used to obtain the depth information of one reference pixel point. The principle of measuring the depth information of the Psensor can refer to dToF, where the direct measurement of the time of flight (dToF: direct Time-of-Flight) is to directly measure the distance based on the time difference between the transmission and reception of the pulse, and the Psensor can calculate the distance between the imaging plane and the object surface of the object to be identified based on the round-trip time of the transmitted light beam between the imaging plane and the object surface and the propagation speed of light, and the distance is the depth information.

[0042] For example, in the absence of a depth camera, if the object to be identified is planar (e.g., a palm of a hand), the Psensor can be used as an alternative solution to derive a planar depth value in which the object to be identified is located, and three Psensors can be mounted at positions that are not on the same line in the pixel plane of the infrared light image, thereby utilizing the Psensors to obtain depth information corresponding to three or more reference pixel points.

[0043] In step 103, based on the position information of the reference pixel point in the infrared image and the depth information, depth information of each pixel point in the infrared image for the object to be identified is obtained.

[0044] Specifically, the embodiments of the present application can obtain the pixel plane pixel depth of the infrared light image, i.e., the depth information of each pixel point in the infrared light image for the object to be identified, based on the position information and depth information of the reference pixel point.

[0045] For example, in an embodiment of the present application, step 103 may include the steps of: determining corresponding mapping point spatial position information of the reference pixel point in the imaging space of the infrared light image based on the position information and the depth information of the reference pixel point in the infrared light image; constructing a plane orientation equation corresponding to the object to be identified based on the mapping point spatial position information; and obtaining depth information of each pixel point in the infrared light image relative to the object to be identified based on the plane orientation equation and the positional relationship between each pixel point in the infrared light image and the reference pixel point.

[0046] In some embodiments, the step of "determining corresponding mapping point spatial position information of the reference pixel point in the imaging space of the infrared light image based on the position information of the reference pixel point in the infrared light image and the depth information" may include a step of performing a position transformation process on the position information of the reference pixel point in the infrared light image based on a position transformation relationship between a pixel plane of the infrared light image and an imaging space, to obtain corresponding mapping point initial position information of the reference pixel point in the imaging space of the infrared light image, and a step of determining corresponding mapping point spatial position information of the reference pixel point in the imaging space of the infrared light image based on the depth information corresponding to the reference pixel point and the mapping point initial position information.

[0047] Here, the position transformation relationship between the pixel plane of the infrared light image and the imaging space may be specifically represented by the camera internal parameters of the camera sensor corresponding to the infrared light image. The embodiment of the present application can establish a pixel coordinate system corresponding to the infrared map in the pixel plane of the infrared light image, the pixel coordinate system may be a two-dimensional coordinate system, and establish a camera coordinate system corresponding to the infrared map in the imaging space of the infrared light image, the camera coordinate system may be a three-dimensional coordinate system.

[0048] For the simplified camera model shown in Fig. 1d, the camera model is specifically a pinhole imaging model, and the camera internal parameters are parameters for describing the conversion relationship between the three-dimensional space coordinates (i.e., coordinates on the pixel plane 14) when a real-world object is imaged on the camera sensor and the two-dimensional pixel coordinates on the physical imaging plane 13 after imaging. The depth image and the point cloud image in the above embodiment can also be converted to each other by the camera internal parameters.

[0049] Here, the camera coordinate system refers to a coordinate system in which the optical center O17 of the camera 16 is the coordinate origin, the optical axis is the z-axis, and the x-axis and y-axis are parallel to the x-axis and y-axis of the imaging plane. The 3D coordinates of the depth image and the point cloud image obtained by transformation using the camera internal parameters are coordinates in the camera coordinate system (Oxyz).

[0050] Specifically, based on the position transformation relationship between the pixel plane of the infrared light image and the imaging space, camera internal parameters of the camera sensor corresponding to the infrared light image can be obtained, and further based on the camera internal parameters, position transformation processing can be performed on the position information of the reference pixel point in the infrared light image, and the processing process may be as shown in equation (1).

[0051] TIFF2025509691000002.tif15170, where fx, fy, u0, v0 are camera internal parameters, specifically, fx represents the focal length 15 in the x-axis direction, fy represents the focal length in the y-axis direction, and u0, v0 are the position of the principal point. The position information of the reference pixel point in the infrared light image can be expressed as (u, v), and the depth information corresponding to the reference pixel point is zc. According to formula (1), the corresponding mapping point initial position information xc, yc in the imaging space of the infrared light image of the reference pixel point can be obtained, where xc, yc are the x-axis coordinate and y-axis coordinate included in the corresponding mapping point spatial position information in the imaging space of the infrared light image of the reference pixel point.

[0052] After obtaining the initial position information xc and yc of the mapping point, the corresponding mapping point spatial position information (xc, yc, zc) in the imaging space of the infrared light image of the reference pixel point can be constructed based on the depth information zc corresponding to the reference pixel point.

[0053] Here, the step of "constructing a plane orientation equation corresponding to the object to be identified based on the mapping point spatial position information" can include the steps of: setting an initial plane orientation equation corresponding to the object to be identified, the initial plane orientation equation including at least one orientation parameter; performing an analysis process on the at least one orientation parameter based on the mapping point spatial position information to obtain analyzed orientation parameters; and updating the initial plane orientation equation based on the analyzed orientation parameters to obtain a plane orientation equation corresponding to the object to be identified.

[0054] Here, the initial plane orientation equation may be set as AX+BY+CZ+D=0, where A, B, C and D are orientation parameters, and the mapping point spatial position information is substituted into the initial plane orientation equation, and the initial plane orientation equation is solved, that is, the orientation parameters are analyzed, and specific values ​​of A, B, C and D are obtained, that is, the analyzed orientation parameters, so that the plane orientation equation of the object to be identified is obtained.

[0055] In one specific embodiment, the position information of three reference pixel points m, n, and p in the infrared light image can be written as m=(x1, y1), n=(x2, y2), and p=(x3, y3). Based on the formula (1) in the above embodiment, the corresponding mapping point space position information of the three reference pixel points in the imaging space of the infrared light image can be calculated, which are respectively written as M=(X1, Y1, Z1), N=(X2, Y2, Z2), and P=(X3, Y3, Z3). Then, M=(X1, Y1, Z1), N=(X2, Y2, Z2), and P=(X3, Y3, Z3) are substituted into the following formula (2): AX+BY+CZ+D=0 (2) By solving the coefficients A, B, C and D in the above equation, the plane orientation equation of the object to be identified can be obtained.

[0056] In an embodiment of the present application, the step of "obtaining depth information of each pixel point in the infrared image for the object to be identified based on the plane orientation equation and the positional relationship between each pixel point in the infrared image and the reference pixel point" may include a step of performing an interpolation operation on positional information of each pixel point in the infrared image and positional information of the reference pixel point to obtain positional relationship parameters between each pixel point in the infrared image and the reference pixel point, and a step of determining depth information of each pixel point in the infrared image for the object to be identified based on the positional relationship parameters and the plane orientation equation.

[0057] In one specific embodiment, the position information of three reference pixel points m, n, and p in the infrared light image can be written as m=(x1, y1), n=(x2, y2), and p=(x3, y3). If it is necessary to obtain the depth information of another pixel point k on the infrared light image, the position relationship parameters between each pixel point in the infrared light image and the reference pixel point can be calculated by using the interpolation of three points m, n, and p, and the position information of the pixel point k in the infrared light image is (x4, y4), and the interpolation process can be as shown in Equation (3) and Equation (4).

[0058] x4=x1+α*(x2-x1)+β*(x3-x1) (3) y4=y1+α*(y2-y1)+β*(y3-y1) (4) By substituting the coordinates of the m, n, p, and k pixel points into equations (3) and (4), the positional relationship parameters α and β between each pixel point in the infrared image and the reference pixel point can be obtained.

[0059] In an embodiment of the present application, the step of "determining depth information of each pixel point in the infrared light image for the object to be identified based on the positional relationship parameters and the plane orientation equation" may include a step of performing an interpolation operation on corresponding mapping point spatial position information of the reference pixel point in the imaging space of the infrared light image based on the positional relationship parameters to obtain corresponding mapping point initial position information of each pixel point in the infrared light image in the imaging space of the infrared light image, and a step of determining depth information of each pixel point in the infrared light image for the object to be identified based on the plane orientation equation and the mapping point initial position information.

[0060] In one embodiment, the corresponding mapping points of three reference pixel points m, n, and p in the imaging space of the infrared light image are M, N, and P, and the spatial position information of the mapping points is called mapping point spatial position information, which is M=(X1, Y1, Z1), N=(X2, Y2, Z2), and P=(X3, Y3, Z3) respectively. The x-axis coordinates and y-axis coordinates of these three mapping point spatial position information are substituted into the above equations (3) and (4), thereby obtaining the corresponding mapping point initial position information X4 and Y4 of the pixel point k in the imaging space of the infrared light image, where X4 and Y4 are specifically the x-axis coordinates and y-axis coordinates of the corresponding mapping point spatial position information of the pixel point k in the imaging space of the infrared light image.

[0061] After obtaining the mapping point initial position information X4, Y4, they can be substituted into the formula (2) in the above embodiment to obtain the corresponding mapping point spatial position information (X4, Y4, Z4) in the imaging space of the infrared image of pixel point k, that is, the depth information Z4 of pixel point k relative to the object to be identified can be obtained. By sequential analogy, the depth information corresponding to each pixel point in the infrared image can be obtained.

[0062] In step 104, based on the depth information corresponding to each pixel point in the infrared image, a position mapping process is performed for each pixel point in the infrared image to obtain corresponding mapping point position information in the visible light image for each pixel point in the infrared image.

[0063] In an embodiment of the present application, the step of "performing a position mapping process for each pixel point in the infrared light image based on depth information corresponding to each pixel point in the infrared light image, and obtaining corresponding mapping point position information in the visible light image for each pixel point in the infrared light image" may include the steps of mapping each pixel point in the infrared light image into an imaging space of the visible light image based on depth information corresponding to each pixel point in the infrared light image, and obtaining corresponding target mapping point spatial position information in the imaging space for each pixel point in the infrared light image, and performing a position transformation process on the target mapping point spatial position information based on a position transformation relationship between the pixel plane and imaging space of the visible light image, and obtaining corresponding mapping point position information in the visible light image for each pixel point in the infrared light image.

[0064] Here, the position transformation relationship between the pixel plane of the visible light image and the imaging space may be specifically represented by camera internal parameters of the camera sensor corresponding to the visible light image, and the embodiments of the present application may establish a pixel coordinate system corresponding to the color figure in the pixel plane of the visible light image, which may be a two-dimensional coordinate system, and establish a camera coordinate system corresponding to the color figure in the imaging space of the visible light image, which may be a three-dimensional coordinate system.

[0065] Here, the camera internal parameters are parameters for describing the transformation relationship between the three-dimensional spatial coordinates when a real-world object is imaged on the camera sensor and the two-dimensional pixel coordinates in the physical imaging plane after imaging.

[0066] In one specific embodiment, the camera internal parameters can be used to convert the target mapping point in the camera coordinate system corresponding to the color map into the pixel coordinate system. The conversion between the camera coordinate system and the pixel coordinate system can be seen in Fig. 1e, where P(XC, YC, ZC) is a point in the camera coordinate system, p(x, y) is a point in the pixel coordinate system, and the coordinate conversion relationship between the two can be converted by the camera internal parameters.

[0067] Specifically, based on the position transformation relationship between the pixel plane of the visible light image and the imaging space, the camera internal parameters of the camera sensor corresponding to the visible light image can be obtained, and then based on the camera internal parameters, position transformation processing can be performed on the target mapping point space position information, and the processing process may be as shown in the following equation (5).

[0068] TIFF2025509691000003.tif15170, where fx, fy, u0, and v0 are camera internal parameters, specifically, fx represents the focal length in the x-axis direction, fy represents the focal length in the y-axis direction, and u0 and v0 are the position of the principal point. The target mapping point spatial position information can be expressed as (xc, yc, zc), and the corresponding mapping point position information (u, v) in the visible light image for each pixel point in the infrared light image can be obtained through formula (5), that is, the mapping point position information in the pixel coordinate system corresponding to the color map.

[0069] In an embodiment of the present application, the step of "mapping each pixel point in the infrared light image into the imaging space of the visible light image based on depth information corresponding to each pixel point in the infrared light image, and obtaining corresponding target mapping point spatial position information in the imaging space of each pixel point in the infrared light image" may include a step of obtaining corresponding mapping point spatial position information in the imaging space of the infrared light image of each pixel point in the infrared light image based on depth information corresponding to each pixel point in the infrared light image, and a step of performing a position transformation process on the mapping point spatial position information based on a position transformation relationship between the imaging space of the infrared light image and the imaging space of the visible light image, and obtaining corresponding target mapping point spatial position information in the imaging space of the visible light image of each pixel point in the infrared light image.

[0070] In some embodiments, the step of "obtaining corresponding mapping point spatial position information in the imaging space of the infrared light image for each pixel point in the infrared light image" may specifically construct corresponding mapping point spatial position information in the imaging space of the infrared light image for each pixel point in the infrared light image based on the depth information corresponding to each pixel point in the infrared light image obtained in step 103 and corresponding mapping point initial position information in the imaging space of the infrared light image for each pixel point in the infrared light image.

[0071] For example, for each pixel point, the depth information corresponding to the pixel point in the infrared light image and the corresponding mapping point initial position information in the imaging space of the infrared light image are each one-dimensional and combined as the corresponding mapping point spatial position information of the pixel point in the imaging space of the infrared light image.

[0072] Here, the position transformation relationship between the imaging space of the infrared light image and the imaging space of the visible light image may be specifically represented by the camera external parameters between the camera sensor corresponding to the infrared light image and the camera sensor corresponding to the visible light image. The embodiment of the present application can establish a camera coordinate system corresponding to an infrared figure in the imaging space of the infrared light image, and establish a camera coordinate system corresponding to a color figure in the imaging space of the visible light image, and the two camera coordinate systems are both three-dimensional coordinate systems.

[0073] As shown in Fig. 1f, the camera extrinsic parameters may be parameters for describing the transformation relationship between other 3D coordinate systems and the camera coordinate system. When there are multiple cameras, the coordinates of an object point in the camera coordinate system of one camera can be transformed into another camera coordinate system through a rotation matrix R and a translation matrix T, and the rotation matrix R and the translation matrix (also called offset vector) T are extrinsic parameters between these two cameras, and the camera extrinsic parameters describe the transformation relationship between the two camera coordinate systems. c0 and c1 are the cameras in the two camera coordinate systems respectively, and there are angle and distance offsets between the two cameras. Similar to the relationship between the world coordinate system and the camera coordinate system, the transformation from the world coordinate system to the camera coordinate system belongs to a rigid transformation, that is, the object does not deform, so only rotation and translation are required.

[0074] In one specific embodiment, the camera external parameters between the camera sensor corresponding to the infrared light image and the camera sensor corresponding to the visible light image can be used to convert the mapping point spatial position information in the camera coordinate system corresponding to the infrared image into the camera coordinate system corresponding to the color image. The conversion between the camera coordinate system corresponding to the infrared image and the camera coordinate system corresponding to the color image can be referred to in FIG. 1g, where P(Xw, Yw, Zw) is a point in the camera coordinate system corresponding to the infrared image, and P(Xw, Yw, Zw) can be converted into a point in the camera coordinate system corresponding to the color image by using the camera external parameters (i.e., the above-mentioned rotation matrix R and translation matrix T).

[0075] Specifically, based on the position transformation relationship between the imaging space of the infrared light image and the imaging space of the visible light image, camera external parameters between the camera sensor corresponding to the infrared light image and the camera sensor corresponding to the visible light image can be obtained, and then position transformation processing can be performed on the mapping point space position information based on the camera external parameters, and the processing process is as shown in Equation (6).

[0076] TIFF2025509691000004.tif15170Here, in formula (6), the matrix composed of r00 to r22 is the rotation matrix R, the matrix composed of tx to tz is the offset vector T, the rotation matrix R and the offset vector T are the camera external parameters, and the mapping point spatial position information can be expressed as (xw, yw, zw). According to formula (6), the corresponding target mapping point spatial position information (xc, yc, zc) in the imaging space of the visible light image for each pixel point in the infrared light image is That is, the spatial position information of the target mapping points in the camera coordinate system corresponding to the color map can be obtained.

[0077] In an embodiment of the present application, the step of "obtaining corresponding mapping point spatial position information of each pixel point in the infrared light image in the imaging space of the infrared light image based on depth information corresponding to each pixel point in the infrared light image" may include a step of performing a position transformation process on the position information of each pixel point in the infrared light image based on a position transformation relationship between a pixel plane of the infrared light image and an imaging space, to obtain corresponding mapping point initial position information of each pixel point in the infrared light image in the imaging space of the infrared light image, and a step of determining corresponding mapping point spatial position information of each pixel point in the infrared light image in the imaging space of the infrared light image based on the depth information corresponding to each pixel point in the infrared light image and the mapping point initial position information.

[0078] Here, the position transformation relationship between the pixel plane of the infrared light image and the imaging space may be specifically represented by the camera internal parameters of the camera sensor corresponding to the infrared light image. The embodiment of the present application can establish a pixel coordinate system corresponding to the infrared map in the pixel plane of the infrared light image, the pixel coordinate system may be a two-dimensional coordinate system, and establish a camera coordinate system corresponding to the infrared map in the imaging space of the infrared light image, the camera coordinate system may be a three-dimensional coordinate system.

[0079] In one specific embodiment, the position information of a pixel point in the pixel coordinate system corresponding to the infrared image can be converted into the camera coordinate system by using the camera internal parameters. For example, point P is a point in the camera coordinate system, and point p is a point in the pixel coordinate system, and the coordinate conversion relationship between the two can be converted by the camera internal parameters.

[0080] Specifically, based on the position transformation relationship between the pixel plane of the infrared light image and the imaging space, camera internal parameters of the camera sensor corresponding to the infrared light image are obtained, and further based on the camera internal parameters, a position transformation process is performed on the position information of each pixel point in the infrared light image, so as to obtain the initial position information of the corresponding mapping point in the imaging space of the infrared light image for each pixel point in the infrared light image, and the processing process may be as shown in the following equation (7).

[0081] TIFF2025509691000005.tif15170, where fx, fy, u0, v0 are camera internal parameters, specifically, fx represents the focal length in the x-axis direction, fy represents the focal length in the y-axis direction, and u0, v0 are the position of the principal point. The position information of each pixel point in the infrared light image can be expressed by (u, v), and the depth information corresponding to each pixel point in the infrared light image is zc. According to formula (1), the corresponding mapping point initial position information xc, yc in the imaging space of the infrared light image of each pixel point in the infrared light image can be obtained, where xc, yc are the x-axis coordinate and y-axis coordinate of the corresponding mapping point spatial position information in the imaging space of the infrared light image of each pixel point in the infrared light image.

[0082] After obtaining the mapping point initial position information xc and yc, based on the depth information zc corresponding to each pixel point in the infrared light image, corresponding mapping point spatial position information (xc, yc, zc) in the imaging space of the infrared light image of each pixel point in the infrared light image can be constructed.

[0083] In step 105, a registration process is performed on the pixel points in the infrared light image and the visible light image based on the mapping point position information.

[0084] Specifically, by using the visible light image as a reference, an alignment process can be performed on each pixel point in the infrared light image and the corresponding pixel point in the visible light image based on the mapping point position information, i.e., the pixel points in the infrared light image are aligned with the pixel points in the visible light image that represent the same target object, and the aligned infrared light image is made to correspond to the positions of the pixel points in the visible light image where the same target object is located.

[0085] For example, the position in the pixel coordinate system of the entire visible light image is kept unchanged, i.e., the coordinates in the coordinate system of each pixel in the visible light image are kept unchanged, and the position in the pixel coordinate system of the infrared light image is moved as a whole by at least one of the movement methods of rotation and translation, and the coordinates in the pixel coordinate system of each pixel point in the infrared light image are transformed and moved, so that the coordinates of pixel points representing the same target object in the infrared light image and the visible light image are made the same.

[0086] In a specific scenario of palm print identification, the palm (specifically, the object to be identified) can be regarded as a plane, so the embodiment of the present application can use Psensor to calculate the slope distance corresponding to the palm (specifically, it can be the depth information of each pixel point in the infrared light image for the object to be identified in the above embodiment), and then perform pixel registration for the infrared light image and the visible light image based on the slope distance, thereby realizing pixel-level registration of the two images at low cost.

[0087] It should be noted that the object identification method provided by the present application can also determine a reference pixel point from each pixel point of the visible light image, obtain depth information of the reference pixel point for the object to be identified, and then obtain depth information of each pixel point in the visible light image for the object to be identified based on the position information and depth information of the reference pixel point in the visible light image, and further perform position mapping processing for each pixel point in the visible light image based on the depth information corresponding to each pixel point in the visible light image, obtain corresponding mapping point position information in the infrared light image for each pixel point in the visible light image, and finally perform alignment processing for the pixel points in the infrared light image and the visible light image based on the mapping point position information, and further perform object identification. It can be understood that the process of obtaining the depth information corresponding to the pixel point and the position mapping processing can be correspondingly referred to the above embodiment, and will not be described in detail here.

[0088] In one specific embodiment, as shown in FIG. 1h, it is a flow chart of pixel registration for an infrared light image and a visible light image. Specifically, in step 201, depth information corresponding to each pixel point in the infrared light image is obtained based on depth information of a reference pixel point in the infrared light image. In step 202, position information of pixel points in the pixel coordinate system corresponding to the infrared light image is transformed into the camera coordinate system of the infrared light image by using the camera internal parameters of the camera sensor corresponding to the infrared light image. In step 203, mapping point spatial position information in the camera coordinate system corresponding to the infrared light image is transformed into the camera coordinate system corresponding to the color image by using the camera external parameters between the camera sensor corresponding to the infrared light image and the camera sensor corresponding to the visible light image. In step 204, the camera internal parameters of the camera sensor corresponding to the visible light image are transformed into the target mapping point in the camera coordinate system corresponding to the color image into the pixel coordinate system corresponding to the color image to obtain a mapping result, thereby performing pixel registration for the visible light image and the infrared light image based on the mapping result.

[0089] In step 106, object identification is performed on the object to be identified based on the infrared light image and the visible light image after the registration process, and an object identification result of the object to be identified is obtained.

[0090] Here, after the registration process, liveness detection can be performed on the infrared light image after the registration process, and after passing the liveness detection, object identification can be performed based on the visible light image after the registration process. The object identification can specifically be a technology of exchanging object identity information with biometric multimedia information. In some embodiments, a related payment operation can be further performed based on the object identification result.

[0091] In a specific scenario, for example, in a palmprint identification scenario, performing object identification on an object to be identified may specifically be extracting palmprint information of the object to be identified. After obtaining the palmprint information, the palmprint information can be matched with palmprint feature information stored in a preset palmprint information database, thereby determining the object identity of the object to be identified. The preset palmprint information database may store a target mapping relationship set, which includes a mapping relationship between the preset palmprint feature information and the preset object identity. For example, the target mapping relationship set may be a relationship table between the preset palmprint feature information and the preset object identity.

[0092] In some embodiments, a matching degree between the palm print information and each of the preset palm print feature information can be calculated, and target palm print feature information can be determined from the preset palm print feature information based on the matching degree. For example, the preset palm print feature information having the highest matching degree with the palm print information can be determined as the target palm print feature information, and an object identity corresponding to the target palm print feature information can be determined as the object identity of the object to be identified based on the target mapping relationship set.

[0093] For example, the Euclidean distance between the vectorized representation of the palm line features (e.g., length, number of lines) and texture features of different palm print information is calculated, and the inverse of the Euclidean distance is taken as the matching degree.

[0094] As can be seen from the above, the embodiment of the present application obtains the depth information of pixel points from an infrared light image, and assigns the depth information to pixel points in a visible light image through mapping between the infrared light image and the visible light image, so that the image collected by an inexpensive sensor based on infrared light and visible light can accurately restore 3D data including depth information, thereby enabling subsequent object identification. Compared with the related art solutions that rely on depth sensors to identify objects, the cost of object identification at the hardware implementation level is reduced, and therefore the present application can be widely used in various application scenarios that require large-scale object identification, such as various intelligent transportation.

[0095] Based on the method described in the previous embodiment, the following will be described in more detail by taking the object identification device specifically integrated into a terminal as an example.

[0096] An embodiment of the present application provides an object identification method, and as shown in FIG. 2, a specific process of the object identification method may be as follows:

[0097] In step 301, the terminal collects infrared and visible light images for the object to be identified.

[0098] Here, the object to be identified may be an object for which biometric identification is required, palm print information corresponding to a human body, a human face, etc., but is not limited to this in the embodiments of the present application.

[0099] Here, the infrared light image may be an infrared image obtained by collecting an infrared sensor and using a pan-infrared light, and may be used to perform live body detection on the object to be identified. The visible light image may be a color image obtained by collecting an image sensor and using a natural light, and may be used to perform identity identification on the object to be identified. It should be noted that in the embodiment of the present application, the infrared light image and the visible light image of the object to be identified may be collected at the same time, and thus the infrared light image and the visible light image may be associated with each other.

[0100] In step 302, the terminal determines a reference pixel point from each pixel point of the infrared light image, and obtains depth information of the reference pixel point relative to the object to be identified.

[0101] In some embodiments, three pixel points that are not collinear with a pixel point of the infrared light image can be arbitrarily selected as the reference pixel point.

[0102] In an embodiment of the present application, the reference pixel points include at least three pixel points that are not on the same line.

[0103] In a specific scenario, for the depth information of the reference pixel points of the object to be identified, specifically, the Psensor can be used to obtain the depth information corresponding to each reference pixel point. Specifically, the number of reference pixel points is first determined, and the number of Psensors can be determined according to the number of reference pixel points, and each Psensor can only obtain the depth information of one reference pixel point. The principle of measuring the depth information of the Psensor can refer to dToF, which directly measures the distance based on the time difference between the transmission and reception of the pulse, and the Psensor can calculate the distance between the imaging plane and the object surface of the object to be identified based on the round-trip time of the transmitted light between the imaging plane and the object surface and the propagation speed of light, and the distance is the depth information.

[0104] For example, in the absence of a depth camera, if the object to be identified is planar (e.g., the palm of a hand), an inexpensive Psensor can be used as an alternative solution to derive a planar depth value in which the object to be identified is located, and three Psensors can be mounted at positions that are not on the same line in the pixel plane of the infrared light image, thereby utilizing the Psensors to obtain depth information corresponding to three or more reference pixel points.

[0105] In step 303, the terminal obtains depth information of each pixel point in the infrared image for the object to be identified according to the position information of the reference pixel point in the infrared image and the depth information.

[0106] In an embodiment of the present application, the step of "obtaining depth information of each pixel point in the infrared light image for the object to be identified based on the position information and the depth information of the reference pixel point in the infrared light image" may include the steps of determining corresponding mapping point spatial position information of the reference pixel point in the imaging space of the infrared light image based on the position information and the depth information of the reference pixel point in the infrared light image, constructing a plane orientation equation corresponding to the object to be identified based on the mapping point spatial position information, and obtaining depth information of each pixel point in the infrared light image for the object to be identified based on the plane orientation equation and the positional relationship between each pixel point in the infrared light image and the reference pixel point.

[0107] For example, the step of "determining corresponding mapping point spatial position information of the reference pixel point in the imaging space of the infrared light image based on the position information and the depth information of the reference pixel point in the infrared light image" may include the steps of: performing a position conversion process on the position information of the reference pixel point in the infrared light image based on a position conversion relationship between a pixel plane of the infrared light image and an imaging space, to obtain corresponding mapping point initial position information of the reference pixel point in the imaging space of the infrared light image; The method may include a step of determining corresponding mapping point spatial position information of the reference pixel point in an imaging space of the infrared light image based on depth information corresponding to the reference pixel point and the mapping point initial position information.

[0108] Here, the position transformation relationship between the pixel plane of the infrared light image and the imaging space may be specifically represented by the camera internal parameters of the camera sensor corresponding to the infrared light image. The embodiment of the present application can establish a pixel coordinate system corresponding to the infrared diagram in the pixel plane (also called pixel plane) of the infrared light image, the pixel coordinate system may be a two-dimensional coordinate system, and establish a camera coordinate system corresponding to the infrared diagram in the imaging space of the infrared light image, the camera coordinate system may be a three-dimensional coordinate system.

[0109] Here, the step of "constructing a plane orientation equation corresponding to the object to be identified based on the mapping point spatial position information" can include the steps of: setting an initial plane orientation equation corresponding to the object to be identified, the initial plane orientation equation including at least one orientation parameter; performing an analysis process on the at least one orientation parameter based on the mapping point spatial position information to obtain analyzed orientation parameters; and updating the initial plane orientation equation based on the analyzed orientation parameters to obtain a plane orientation equation corresponding to the object to be identified.

[0110] Here, the initial plane orientation equation can be set as AX+BY+CZ+D=0, where A, B, C and D are orientation parameters. The embodiment of the present application substitutes mapping point spatial position information into the initial plane orientation equation, thereby analyzing the orientation parameters, and obtains the specific values ​​of A, B, C and D, that is, the analyzed orientation parameters, thereby obtaining the plane orientation equation of the object to be identified.

[0111] In an embodiment of the present application, the step of "obtaining depth information of each pixel point in the infrared image for the object to be identified based on the plane orientation equation and the positional relationship between each pixel point in the infrared image and the reference pixel point" may include a step of performing an interpolation operation on positional information of each pixel point in the infrared image and positional information of the reference pixel point to obtain positional relationship parameters between each pixel point in the infrared image and the reference pixel point, and a step of determining depth information of each pixel point in the infrared image for the object to be identified based on the positional relationship parameters and the plane orientation equation.

[0112] In an embodiment of the present application, the step of "determining depth information of each pixel point in the infrared light image for the object to be identified based on the positional relationship parameters and the plane orientation equation" may include a step of performing an interpolation operation on corresponding mapping point spatial position information of the reference pixel point in the imaging space of the infrared light image based on the positional relationship parameters to obtain corresponding mapping point initial position information of each pixel point in the infrared light image in the imaging space of the infrared light image, and a step of determining depth information of each pixel point in the infrared light image for the object to be identified based on the plane orientation equation and the mapping point initial position information.

[0113] In step 304, the terminal maps each pixel point in the infrared light image into an imaging space of the visible light image based on depth information corresponding to each pixel point in the infrared light image, and obtains corresponding target mapping point spatial position information in the imaging space of each pixel point in the infrared light image.

[0114] In an embodiment of the present application, the step of "mapping each pixel point in the infrared light image into the imaging space of the visible light image based on depth information corresponding to each pixel point in the infrared light image, and obtaining corresponding target mapping point spatial position information in the imaging space of each pixel point in the infrared light image" may include a step of obtaining corresponding mapping point spatial position information in the imaging space of the infrared light image of each pixel point in the infrared light image based on depth information corresponding to each pixel point in the infrared light image, and a step of performing a position transformation process on the mapping point spatial position information based on a position transformation relationship between the imaging space of the infrared light image and the imaging space of the visible light image, and obtaining corresponding target mapping point spatial position information in the imaging space of the visible light image of each pixel point in the infrared light image.

[0115] Here, the position transformation relationship between the imaging space of the infrared light image and the imaging space of the visible light image may be specifically represented by the camera external parameters between the camera sensor corresponding to the infrared light image and the camera sensor corresponding to the visible light image. The embodiment of the present application can establish a camera coordinate system corresponding to an infrared figure in the imaging space of the infrared light image, and establish a camera coordinate system corresponding to a color figure in the imaging space of the visible light image, and both of the two camera coordinate systems are three-dimensional coordinate systems.

[0116] In an embodiment of the present application, the step of "obtaining corresponding mapping point spatial position information of each pixel point in the infrared light image in the imaging space of the infrared light image based on depth information corresponding to each pixel point in the infrared light image" may include a step of performing a position transformation process on the position information of each pixel point in the infrared light image based on a position transformation relationship between a pixel plane of the infrared light image and an imaging space, to obtain corresponding mapping point initial position information of each pixel point in the infrared light image in the imaging space of the infrared light image, and a step of determining corresponding mapping point spatial position information of each pixel point in the infrared light image in the imaging space of the infrared light image based on the depth information corresponding to each pixel point in the infrared light image and the mapping point initial position information.

[0117] Here, the position transformation relationship between the pixel plane of the infrared light image and the imaging space may be specifically represented by the camera internal parameters of the camera sensor corresponding to the infrared light image. The embodiment of the present application can establish a pixel coordinate system corresponding to the infrared diagram in the pixel plane (also called pixel plane) of the infrared light image, the pixel coordinate system may be a two-dimensional coordinate system, and establish a camera coordinate system corresponding to the infrared diagram in the imaging space of the infrared light image, the camera coordinate system may be a three-dimensional coordinate system.

[0118] In step 305, the terminal performs a position transformation process on the target mapping point spatial position information based on the position transformation relationship between the pixel plane of the visible light image and the imaging space, to obtain corresponding mapping point position information in the visible light image for each pixel point in the infrared light image.

[0119] In step 306, the terminal performs a registration process on the pixel points in the infrared light image and the visible light image based on the mapping point position information.

[0120] Specifically, by using the visible light image as a reference, an alignment process can be performed on each pixel point in the infrared light image and the corresponding pixel point in the visible light image based on the mapping point position information, i.e., the pixel points in the infrared light image are aligned with the pixel points in the visible light image that represent the same target object, and the aligned infrared light image is made to correspond to the positions of the pixel points in the visible light image where the same target object is located.

[0121] In step 307, the terminal performs object identification on the object to be identified based on the infrared light image and the visible light image after the alignment process, and obtains an object identification result for the object to be identified.

[0122] Here, after the registration process, liveness detection can be performed on the infrared light image after the registration process, and after passing the liveness detection, object identification can be performed based on the visible light image after the registration process. The object identification can specifically be a technology of exchanging object identity information with biometric multimedia information. In some embodiments, a related payment operation can be further performed based on the object identification result.

[0123] As can be seen, the embodiments of the present application can achieve pixel alignment for infrared and visible light images in scenarios where depth maps are lacking, thereby enabling object identification.

[0124] An embodiment of the present application further provides an object identification apparatus, which may include a collection unit 301, a determination unit 302, an acquisition unit 303, a mapping unit 304, a registration unit 305 and an identification unit 306, as shown in FIG.

[0125] The collection unit 301 is configured to collect infrared and visible light images for an object to be identified.

[0126] The determining unit 302 is configured to determine a reference pixel point from each pixel point of the infrared light image, and obtain depth information of the reference pixel point relative to the object to be identified.

[0127] In some embodiments of the present application, the reference pixel points include at least three pixel points that are not collinear.

[0128] The obtaining unit 303 is configured to obtain depth information of each pixel point in the infrared light image relative to the object to be identified based on the position information of the reference pixel point in the infrared light image and the depth information.

[0129] In some embodiments of the present application, the acquisition unit may include a determination subunit, a construction subunit, and an acquisition subunit, wherein the determination subunit is configured to determine corresponding mapping point spatial position information of the reference pixel point in an imaging space of the infrared light image based on the position information of the reference pixel point in the infrared light image and the depth information, and the construction subunit is configured to construct a plane orientation equation corresponding to the object to be identified based on the mapping point spatial position information; The acquisition subunit is configured to acquire depth information of each pixel point in the infrared light image relative to the object to be identified based on the plane orientation equation and a positional relationship between each pixel point in the infrared light image and the reference pixel point.

[0130] In some embodiments of the present application, the acquisition subunit may be specifically configured to perform an interpolation process on the position information of each pixel point in the infrared light image and the position information of the reference pixel point to obtain position relationship parameters between each pixel point in the infrared light image and the reference pixel point, and determine depth information of each pixel point in the infrared light image relative to the object to be identified based on the position relationship parameters and the plane orientation equation.

[0131] The mapping unit 304 is configured to perform a position mapping process for each pixel point in the infrared light image based on depth information corresponding to each pixel point in the infrared light image, and obtain corresponding mapping point position information in the visible light image for each pixel point in the infrared light image.

[0132] In some embodiments of the present application, the mapping unit may include a mapping subunit and a position transformation subunit, wherein the mapping subunit is configured to map each pixel point in the infrared light image into an imaging space of the visible light image based on depth information corresponding to each pixel point in the infrared light image, and obtain corresponding target mapping point spatial position information in the imaging space of each pixel point in the infrared light image, and the position transformation subunit is configured to perform a position transformation process on the target mapping point spatial position information based on a position transformation relationship between a pixel plane of the visible light image and the imaging space, and obtain corresponding mapping point position information in the visible light image of each pixel point in the infrared light image.

[0133] In some embodiments of the present application, the mapping subunit may be specifically configured to obtain corresponding mapping point spatial position information in the imaging space of the infrared light image for each pixel point in the infrared light image based on depth information corresponding to each pixel point in the infrared light image, and perform a position transformation process on the mapping point spatial position information based on a position transformation relationship between the imaging space of the infrared light image and the imaging space of the visible light image, to obtain corresponding target mapping point spatial position information in the imaging space of the visible light image for each pixel point in the infrared light image.

[0134] The alignment unit 305 is configured to perform alignment processing on pixel points in the infrared light image and the visible light image based on the mapping point position information.

[0135] The identification unit 306 is configured to perform object identification on the object to be identified based on the infrared light image and the visible light image after the alignment process, and obtain an object identification result for the object to be identified.

[0136] An embodiment of the present application further provides an electronic device, and as shown in FIG. 4, a structural schematic diagram of an electronic device according to an embodiment of the present application is shown. The electronic device may be a terminal or a server, etc. The electronic device may include components such as a processor 401 of one or more processing cores, a memory 402 of one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art can understand that the structure of the electronic device shown in FIG. 4 does not constitute a limitation on the electronic device, and the electronic device may include more or less components than those shown, or may combine some components, or may arrange different components.

[0137] The processor 401 is a control center of the electronic device, and uses various interfaces and lines to connect each part of the entire electronic device, runs or executes software programs and / or modules stored in the memory 402, and accesses data stored in the memory 402 to perform various functions of the electronic device and process data. For example, the processor 401 may include one or more processing cores, and for example, the processor 401 may integrate an application processor and a modem processor, where the application processor mainly processes the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communication. It may be understood that the above modem processor does not have to be integrated into the processor 401.

[0138] The memory 402 may be used to store software programs and modules, and the processor 401 executes various functional applications and data processing by executing the software programs and modules stored in the memory 402. The memory 402 may mainly include a storage program area and a storage data area, where the storage program area may store an operating system, an application program required for at least one function (e.g., an audio playback function, an image playback function, etc.), and the storage data area may store data created based on the use of the electronic device, etc.

[0139] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail here. Specifically, in the embodiment of the present application, the processor 401 in the electronic device loads executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and executes the application programs stored in the memory 402 by the processor 401, thereby realizing the above object identification method of the embodiment of the present application. For specific implementation of each of the above operations, reference may be made to the previous embodiment, which will not be described in detail here.

[0140] As can be understood by those skilled in the art, all or part of the steps in the various methods of the above embodiments may be completed by instructions or by controlling associated hardware through instructions, which may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0141] The embodiment of the present application provides a computer-readable storage medium on which a number of instructions are stored, and the instructions can be loaded by a processor to execute any steps in the object identification method provided by the embodiment of the present application. The specific implementation of each of the above operations can be referred to the previous embodiment and will not be described in detail here. The computer-readable storage medium can include a read only memory (ROM), a random access memory (RAM), a magnetic disk, an optical disk, etc.

[0142] An embodiment of the present application further provides a computer program product or a computer program, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer readable storage medium, a processor of a computing device reading the computer instructions from the computer readable storage medium, and the processor executing the computer instructions to cause the computing device to perform the methods provided in the various selectable implementations of the object identification aspect described above.

[0143] Although the object identification method and related devices provided by the embodiments of the present application have been introduced in detail above, the present specification applies specific examples to describe the principles and embodiments of the present application, and the description of the above examples is only intended to assist in understanding the method and core idea of ​​the present application. At the same time, those skilled in the art will recognize that there are changes in the specific embodiments and scope of application based on the ideas of the present application, and as described above, the contents of the present specification should not be understood as limitations on the present application.

Claims

1. 1. A method for identifying an object, implemented by an electronic device, comprising: collecting infrared and visible light images of an object to be identified; determining a reference pixel point from each pixel point of the infrared light image, and obtaining depth information of the reference pixel point with respect to the object to be identified; obtaining depth information of each pixel point in the infrared light image with respect to the object to be identified based on position information of the reference pixel point in the infrared light image and the depth information; performing a position mapping process for each pixel point in the infrared light image based on depth information corresponding to each pixel point in the infrared light image, and obtaining corresponding mapping point position information in the visible light image for each pixel point in the infrared light image; performing a registration process on pixel points in the infrared light image and the visible light image based on the mapping point position information; performing object identification on the object to be identified based on the infrared light image and the visible light image after the alignment process, and obtaining an object identification result for the object to be identified.

2. The step of performing a position mapping process on each pixel point in the infrared light image based on depth information corresponding to each pixel point in the infrared light image, and obtaining corresponding mapping point position information in the visible light image for each pixel point in the infrared light image, includes: Mapping each pixel point in the infrared light image into an imaging space of the visible light image based on depth information corresponding to each pixel point in the infrared light image, and obtaining corresponding target mapping point spatial position information in the imaging space of each pixel point in the infrared light image; and performing a position transformation process on the target mapping point space position information based on a position transformation relationship between a pixel plane of the visible light image and an imaging space to obtain corresponding mapping point position information in the visible light image for each pixel point in the infrared light image. The method of claim 1 .

3. The step of mapping each pixel point in the infrared light image into an imaging space of the visible light image based on depth information corresponding to each pixel point in the infrared light image, and obtaining corresponding target mapping point spatial position information in the imaging space of each pixel point in the infrared light image, includes: obtaining corresponding mapping point spatial position information of each pixel point in the infrared light image in an imaging space of the infrared light image based on depth information corresponding to each pixel point in the infrared light image; and performing a position transformation process on the mapping point spatial position information based on a position transformation relationship between the imaging space of the infrared light image and the imaging space of the visible light image, to obtain corresponding target mapping point spatial position information in the imaging space of the visible light image for each pixel point in the infrared light image. The method of claim 2 .

4. The step of acquiring corresponding mapping point spatial position information of each pixel point in the infrared light image in an imaging space of the infrared light image based on depth information corresponding to each pixel point in the infrared light image includes: performing a position transformation process on position information of each pixel point in the infrared light image based on a position transformation relationship between a pixel plane of the infrared light image and an imaging space, to obtain initial position information of a corresponding mapping point in the imaging space of the infrared light image for each pixel point in the infrared light image; determining corresponding mapping point spatial position information of each pixel point in the infrared light image in an imaging space of the infrared light image based on depth information corresponding to each pixel point in the infrared light image and the mapping point initial position information; The method of claim 3 .

5. The step of acquiring depth information of each pixel point in the infrared light image with respect to the object to be identified based on the position information and the depth information of the reference pixel point in the infrared light image, determining corresponding mapping point spatial position information of the reference pixel point in an imaging space of the infrared light image based on the position information of the reference pixel point in the infrared light image and the depth information; constructing a plane orientation equation corresponding to the object to be identified based on the mapping point spatial position information; and acquiring depth information of each pixel point in the infrared light image relative to the object to be identified based on the plane orientation equation and a positional relationship between each pixel point in the infrared light image and the reference pixel point. The method of claim 1 .

6. The step of acquiring depth information of each pixel point in the infrared light image relative to the object to be identified based on the plane orientation equation and a positional relationship between each pixel point in the infrared light image and the reference pixel point includes: performing an interpolation process on position information of each pixel point in the infrared image and position information of the reference pixel point to obtain positional relationship parameters between each pixel point in the infrared image and the reference pixel point; determining depth information of each pixel point in the infrared image relative to the object to be identified based on the positional relationship parameters and the plane orientation equation; The method of claim 5 .

7. The step of determining depth information of each pixel point in the infrared light image relative to the object to be identified based on the positional relationship parameters and the plane orientation equation includes: performing an interpolation operation on corresponding mapping point spatial position information of the reference pixel point in the imaging space of the infrared light image based on the positional relationship parameters, to obtain corresponding mapping point initial position information of each pixel point in the infrared light image in the imaging space of the infrared light image; determining depth information of each pixel point in the infrared light image relative to the object to be identified based on the plane orientation equation and the mapping point initial position information; The method of claim 6 .

8. The reference pixel points include at least three pixel points that are not on the same straight line. The method of claim 1 .

9. The step of performing a registration process on pixel points in the infrared light image and the visible light image based on the mapping point position information includes: keeping unchanged the coordinates of each pixel point in the visible light image in a pixel point coordinate system, and generally moving the position of the infrared light image in the pixel coordinate system by at least one of a rotation and a translation, so that after the movement, the coordinates of pixel points representing the same target object in the infrared light image and the visible light image are the same. The method of claim 1 .

10. 1. An object identification device, comprising: a collection unit configured to collect infrared and visible light images of the object to be identified; a determining unit configured to determine a reference pixel point from each pixel point of the infrared light image and obtain depth information of the reference pixel point relative to the object to be identified; an acquisition unit configured to acquire depth information of each pixel point in the infrared light image for the object to be identified based on position information of the reference pixel point in the infrared light image and the depth information; a mapping unit configured to perform a position mapping process on each pixel point in the infrared light image based on depth information corresponding to each pixel point in the infrared light image, and obtain corresponding mapping point position information in the visible light image for each pixel point in the infrared light image; a registration unit configured to perform a registration process on pixel points in the infrared light image and the visible light image based on the mapping point position information; and an identification unit configured to perform object identification on the object to be identified based on the infrared light image and the visible light image after the alignment process, and obtain an object identification result for the object to be identified.

11. A memory for storing an application program; A processor that executes the application program in the memory to perform the operations in the object identification method according to any one of claims 1 to 8.

12. A computer-readable storage medium having stored thereon a plurality of instructions, comprising: A computer readable storage medium having the instructions loaded thereon by a processor for performing steps in the method for identifying objects according to any one of claims 1 to 8.

13. A computer program product comprising a computer program or instructions which, when executed by a processor, implements the steps of the method for object identification according to any one of claims 1 to 8.

Images