Space alignment method and device, electronic equipment, storage medium and program product

Abstract

The embodiment of the invention provides a space alignment method and device, electronic equipment, a storage medium and a program product, which are at least applied to the field of cloud technology or image recognition, and the method comprises the following steps: obtaining a first original image and a second original image; respectively performing effective image clipping on the first original image and the second original image by adopting a specific clipping ratio to correspondingly obtain a first clipping image and a second clipping image; performing distortion correction processing on the first cut image and the second cut image to correspondingly obtain a first distortion correction image and a second distortion correction image; distortion image cutting and spatial alignment are carried out on the first distortion correction image and the second distortion correction image at the same time, and a spatial alignment image containing the target object is obtained. According to the method and the device, the effective FOV of the image acquisition equipment in the identity recognition field can be optimized while the image alignment effect is ensured.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of the Internet, and are related to but not limited to a spatial alignment method, device, electronic device, storage medium, and program product. Background Art

[0002] Dual-camera image acquisition devices are now common in smart devices, particularly for biometric information collection and recognition, such as face and palm scanning. Accurate biometric recognition requires combining the images captured by both cameras. However, due to issues such as the positioning and assembly of the camera modules' external structures or errors introduced during camera capture, the images captured by both cameras can become misaligned.

[0003] In the related art, when aligning the images collected by the dual cameras, in the face recognition scene, the color image, infrared image, depth image, etc. are usually aligned. Figure 3 In traditional palm-scanning scenarios, the color image and infrared image are usually aligned at fixed positions, or the two images are dynamically aligned based on the depth distance of the PSensor.

[0004] However, the image alignment methods in related technologies all have problems such as poor alignment effect and small effective field of view (FOV), which leads to a small available distance range for palm swiping and poor user palm swiping experience. Summary of the Invention

[0005] The embodiments of the present application provide a spatial alignment method, device, electronic device, storage medium and program product, which can be applied at least in the field of cloud technology or image recognition, and can optimize the effective FOV of image acquisition equipment in the field of identity recognition while ensuring the image alignment effect.

[0006] The technical solution of the embodiment of the present application is implemented as follows:

[0007] An embodiment of the present application provides a spatial alignment method, comprising: acquiring an original image pair to be aligned, the original image pair comprising a first original image containing a target object acquired by a first type of image acquisition device and a second original image containing the target object acquired by a second type of image acquisition device;

[0008] Using a specific cropping ratio, effectively cropping the first original image and the second original image, respectively, to obtain a first cropped image and a second cropped image;

[0009] performing distortion correction processing on the first cropped image and the second cropped image respectively, to obtain a first distortion-corrected image and a second distortion-corrected image respectively;

[0010] Distorted image cropping and spatial alignment are performed simultaneously on the first distortion-corrected image and the second distortion-corrected image to obtain a spatially aligned image containing the target object.

[0011] An embodiment of the present application provides a spatial alignment device, including: an acquisition module, used to acquire an original image pair to be aligned, the original image pair including a first original image containing a target object acquired by a first type of image acquisition device and a second original image containing the target object acquired by a second type of image acquisition device; an image cropping module, used to use a specific cropping ratio to perform effective image cropping on the first original image and the second original image, respectively, to obtain a first cropped image and a second cropped image accordingly; a distortion correction module, used to perform distortion correction processing on the first cropped image and the second cropped image, respectively, to obtain a first distortion-corrected image and a second distortion-corrected image accordingly; and a spatial alignment module, used to simultaneously perform distortion image cropping and spatial alignment on the first distortion-corrected image and the second distortion-corrected image, to obtain a spatially aligned image containing the target object.

[0012] In some embodiments, the first original image and the second original image are images with barrel distortion; the distortion correction module is further used to: perform distortion correction processing on the first cropped image and the second cropped image with the barrel distortion, respectively, to obtain a first initial corrected image and a second initial corrected image with a pincushion shape; use black pixels to fill two arc areas in the first initial corrected image and the second initial corrected image with a pincushion shape, to obtain the first distortion corrected image and the second distortion corrected image; wherein the first distortion corrected image and the second distortion corrected image respectively include: two arc black edge areas formed after the distortion correction processing is performed on the first original image and the second original image with barrel distortion.

[0013] In some embodiments, the spatial alignment module is further used to: adopt a middle alignment method to spatially align the first distortion-corrected image having the circular arc black edge area and the second distortion-corrected image having the circular arc black edge area to obtain an overlapping image after spatial alignment; perform the image cropping on the overlapping image after spatial alignment to obtain a spatially aligned image containing the target object.

[0014] In some embodiments, the spatial alignment module is further used to: determine a first object centerline of the target object in the first distortion-corrected image having the circular arc black border area, and a second object centerline of the target object in the second distortion-corrected image having the circular arc black border area; determine a relative positional relationship between the first distortion-corrected image having the circular arc black border area and the second distortion-corrected image having the circular arc black border area; based on the relative positional relationship, determine a first distance between a first image edge of the first distortion-corrected image having the circular arc black border area and the first object centerline, and a second distance between a second image edge of the second distortion-corrected image having the circular arc black border area and the second object centerline; wherein the first image edge is an edge away from the second distortion-corrected image determined based on the relative positional relationship, and the second image edge is an edge away from the first distortion-corrected image determined based on the relative positional relationship; based on the first distance and the second distance, spatially align the first distortion-corrected image having the circular arc black border area and the second distortion-corrected image having the circular arc black border area to obtain an overlapping image after spatial alignment.

[0015] In some embodiments, the spatial alignment module is also used to: determine the first alignment direction of the first distortion-corrected image having the circular arc black border area and the second alignment direction of the second distortion-corrected image having the circular arc black border area based on the relative position relationship; wherein the first alignment direction is opposite to the second alignment direction; determine the first movement distance of the first distortion-corrected image having the circular arc black border area based on the first distance and the second distance, and determine the second movement distance of the second distortion-corrected image having the circular arc black border area based on the first distance and the second distance; move the first distortion-corrected image having the circular arc black border area along the first alignment direction according to the first movement distance, and move the second distortion-corrected image having the circular arc black border area along the second alignment direction according to the second movement distance to obtain the overlapping image after spatial alignment.

[0016] In some embodiments, the spatial alignment module is also used to: determine a first center line of the first distortion-corrected image having the circular arc black edge area and a second center line of the second distortion-corrected image having the circular arc black edge area; based on the first center line and the second center line, spatially align the first distortion-corrected image having the circular arc black edge area and the second distortion-corrected image having the circular arc black edge area to obtain an overlapping image after spatial alignment.

[0017] In some embodiments, the first original image and the second original image are images with pincushion distortion; the distortion correction module is further used to: perform distortion correction processing on the first cropped image and the second cropped image with the pincushion distortion, respectively, to obtain a first initial corrected image and a second initial corrected image with a barrel shape; use black pixels to fill the four arc regions in the first initial corrected image and the second initial corrected image with a barrel shape, to obtain the first distortion corrected image and the second distortion corrected image; wherein the first distortion corrected image and the second distortion corrected image respectively include: four arc black-edge regions formed after the distortion correction processing is performed on the first original image and the second original image with the pincushion distortion.

[0018] In some embodiments, the specific cropping ratio is an unconventional ratio; the image cropping module is further used to: use the unconventional ratio to perform full-size effective image cropping on the first original image and the second original image, respectively, to obtain the first cropped image and the second cropped image accordingly.

[0019] In some embodiments, the distortion correction module is further used to: obtain a first device internal parameter of the first image acquisition device and a second device internal parameter of the second image acquisition device; obtain a first distortion coefficient of the first image acquisition device from the first device internal parameter, and use the first distortion coefficient to perform distortion correction processing on the first cropped image to obtain the first distortion corrected image; obtain a second distortion coefficient of the second image acquisition device from the second device internal parameter, and use the second distortion coefficient to perform distortion correction processing on the second cropped image to obtain the second distortion corrected image.

[0020] In some embodiments, the distortion correction module is further used to: obtain a preset camera calibration standard value; use a preset calibration fixture to independently calibrate the first image acquisition device and the second image acquisition device in sequence based on the camera calibration standard value, and obtain the first device internal parameter of the first image acquisition device and the second device internal parameter of the second image acquisition device.

[0021] In some embodiments, the first type of image acquisition device is an infrared camera, and the second type of image acquisition device is a color camera; the first original image is an infrared image, and the second original image is a color image.

[0022] An embodiment of the present application provides an electronic device, including: a memory for storing executable instructions; and a processor for implementing the above-mentioned spatial alignment method when executing the executable instructions stored in the memory.

[0023] An embodiment of the present application provides a computer program product, which includes executable instructions stored in a computer-readable storage medium; wherein a processor of an electronic device reads the executable instructions from the computer-readable storage medium and implements the above-mentioned spatial alignment method when executing the executable instructions.

[0024] An embodiment of the present application provides a computer-readable storage medium storing executable instructions for causing a processor to execute the executable instructions to implement the above-mentioned spatial alignment method.

[0025] The embodiments of the present application have the following beneficial effects:

[0026] When performing spatial alignment of the original image pair, the electronic device sequentially performs effective image cropping and distortion correction processing on the first original image and the second original image in the acquired original image pair, thereby obtaining the first distortion-corrected image and the second distortion-corrected image after distortion correction, respectively. Afterwards, the first distortion-corrected image and the second distortion-corrected image are simultaneously subjected to distortion image cropping and spatial alignment, thereby obtaining a spatially aligned image containing the target object. In this way, since the distortion image cropping and spatial alignment are performed simultaneously, it is possible to ensure that when the distortion image is cropped, the non-intersecting portions of the first original image and the second original image are not excessively cropped, that is, the effective FOV in the original image pair is not cropped. Therefore, the method of the embodiment of the present application can optimize the effective FOV between the first type of image acquisition device and the second type of image acquisition device in the field of identity recognition while ensuring that the original image pair has an alignment effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 Schematic diagram of effective FOV and independent FOV provided in an embodiment of the present application;

[0028] Figure 2 This is an optional architectural diagram of a spatial alignment system provided in an embodiment of the present application;

[0029] Figure 3 is a structural diagram of an electronic device provided in an embodiment of the present application;

[0030] Figure 4 This is an optional flowchart of the spatial alignment method provided in an embodiment of the present application;

[0031] Figure 5 This is another optional flowchart of the spatial alignment method provided in an embodiment of the present application;

[0032] Figure 6 This is a schematic diagram of an implementation process of spatial alignment provided in an embodiment of the present application;

[0033] Figure 7 This is a schematic diagram of an implementation principle of spatial alignment using a center-alignment method provided in an embodiment of the present application;

[0034] Figure 8 This is another implementation flow diagram of the spatial alignment provided in an embodiment of the present application;

[0035] Figure 9 This is another optional flowchart of the spatial alignment method provided in the embodiment of the present application;

[0036] Figure 10 This is a flowchart of a method for implementing distortion correction processing provided by an embodiment of the present application;

[0037] Figure 11 This is a schematic diagram of a conventional process for obtaining an effective FOV within a palm brushing area provided in an embodiment of the present application;

[0038] Figure 12 This is a flow chart of a method for optimizing the effective FOV of a camera palm swiping in the palm swiping field provided by an embodiment of the present application;

[0039] Figure 13 2 is a schematic diagram of analyzing and comparing full-size FOV resolution using unconventional ratios in the effective image cropping stage provided by an embodiment of the present application;

[0040] Figure 14 This is a schematic diagram of the problem of uneven vertical alignment of the cameras provided in an embodiment of the present application;

[0041] Figure 15 This is a schematic diagram of the result of random deviation of the camera provided by the embodiment of the present application;

[0042] Figure 16 This is a schematic diagram of lateral loss analysis provided by an embodiment of the present application;

[0043] Figure 17 2 is a comparative diagram of alignment to a color image and alignment to the center provided in an embodiment of the present application;

[0044] Figure 18 This is a schematic diagram comparing the results of alignment in different directions provided by the embodiments of the present application;

[0045] Figure 19 Schematic diagram of an arc region for distortion correction provided by an embodiment of the present application;

[0046] Figure 20 This is a schematic diagram showing the comparison results between the normal process provided in an embodiment of the present application and direct spatial alignment based on the distortion-corrected original image;

[0047] Figure 21This is a schematic diagram of the overall process of the spatial alignment method for optimizing FOV provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solutions and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limiting this application. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0049] In the following description, reference is made to "some embodiments," which describe a subset of all possible embodiments. However, it will be understood that "some embodiments" may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict. Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by those skilled in the art to which the embodiments of this application pertain. The terms used in the embodiments of this application are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0050] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0051] Before explaining the spatial alignment method provided in the embodiment of the present application, the professional terms involved in the embodiment of the present application are first explained.

[0052] (1) Palm area detection: refers to the process of using target detection technology to locate the finger seams and extract the palm print area image from the captured image.

[0053] (2) Palmprint recognition: It refers to identifying the identity information of different users based on the palmprint area image.

[0054] (3) Cross-device registration and identification: This refers to registering and identifying on two devices that are quite different, such as registering on a mobile phone and identifying on an IoT device.

[0055] (4) Color image: refers to a color image captured by a color sensor using natural light imaging. It is generally used for face / palm selection and comparative recognition in face / palm payment.

[0056] (5) Infrared image: refers to an infrared image collected by an infrared sensor using infrared light imaging. It is generally used for liveness detection in face / palm payment.

[0057] (6) Optimization: refers to selecting a set of color images, depth images, and infrared images that meet the preconditions of the liveness detection and contrast recognition algorithms (in the face scanning scenario: color images, depth images, infrared images; in the palm scanning scenario: color images, infrared images). Among them, the color images can be optimized based on indicators such as the face / palm angle, size, centering, and color image clarity; the infrared images can be optimized based on indicators such as infrared image brightness; and the depth images can be optimized based on indicators such as the depth image (face scanning) completeness.

[0058] (7) Effective FOV: refers to the intersection of the field of view of the color camera and the infrared camera (such as Figure 1 middle area 101).

[0059] (8) Independent FOV: refers to the non-intersecting parts of the field of view of the color camera and the infrared camera (such as Figure 1 middle area 102 and area 103).

[0060] (9) Black edge: refers to the portion of the images captured by the color camera and the infrared camera that is filled with black pixels due to spatial alignment, resulting in no objects being captured in the image.

[0061] (10) Spatial alignment: This means that the color camera and the infrared camera are not at the same point in the horizontal direction, so the relative positions of the palms in the two images are different. Spatial alignment is the process of aligning the relative positions of the palms in the two images.

[0062] An embodiment of the present application provides a spatial alignment method, which uses an unconventional ratio (such as full-size resolution) for effective image cropping, aligns to the middle of the two images during spatial alignment, and uses the arc black edges in the distorted original image to optimize the effective FOV of the camera within the palm-swiping area.

[0063] Specifically, in the spatial alignment method provided by the embodiment of the present application, first, the electronic device obtains an original image pair to be aligned, the original image pair including a first original image containing a target object acquired by a first type of image acquisition device and a second original image containing a target object acquired by a second type of image acquisition device; then, using a specific cropping ratio, the first original image and the second original image are respectively effectively cropped to obtain a first cropped image and a second cropped image; then, the first cropped image and the second cropped image are respectively subjected to distortion correction processing to obtain a first distortion-corrected image and a second distortion-corrected image; finally, the first distortion-corrected image and the second distortion-corrected image are simultaneously subjected to distortion image cropping and spatial alignment to obtain a spatially aligned image containing the target object. In this way, since the electronic device simultaneously performs distortion image cropping and spatial alignment, it can ensure that when cropping the distorted image, the non-intersecting portions of the first original image and the second original image will not be excessively cropped, that is, the effective FOV in the original image pair will not be cropped. Therefore, the method of the embodiment of the present application can optimize the effective FOV between the first type of image acquisition device and the second type of image acquisition device in the field of identity recognition while ensuring the alignment effect of the original image pair.

[0064] Here, first, an exemplary application of the spatial alignment device of the embodiment of the present application is described, which is an electronic device for implementing the spatial alignment method. In one implementation, the spatial alignment device (i.e., electronic device) provided in the embodiment of the present application can be implemented as a terminal or as a server. In one implementation, the spatial alignment device provided in the embodiment of the present application can be implemented as a palm-swiping device, a face-swiping device, a laptop computer, a tablet computer, a desktop computer, a mobile phone, a portable music player, a personal digital assistant, a dedicated messaging device, a portable gaming device, an intelligent robot, a smart home appliance, and an intelligent vehicle-mounted device, any terminal with an identity information recognition function or an image data processing function; in another implementation, the spatial alignment device provided in the embodiment of the present application can also be implemented as a server, wherein the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDNs), and big data and artificial intelligence platforms. The terminal and the server can be connected directly or indirectly via wired or wireless communication, which is not limited in the embodiments of the present application. Below, an exemplary application of spatial alignment when implemented as a server will be described.

[0065] See also Figure 2 , Figure 2 It is an optional architectural diagram of the spatial alignment system provided in an embodiment of the present application. The spatial alignment system 10 in the embodiment of the present application includes at least a terminal 100, a network 200 and a server 300. The terminal 100 can be a palm-swiping device or a face-swiping device. Here, the palm-swiping device is taken as an example for explanation. A palm-swiping application is installed on the palm-swiping device, and the palm-swiping application can identify the palm prints of the user collected by the palm-swiping device, thereby realizing the identification of the user's identity information. In the embodiment of the present application, the server 300 can be a server of the palm-swiping application. The server 300 can constitute the spatial alignment device in the embodiment of the present application, that is, the spatial alignment method in the embodiment of the present application is implemented through the server 300. The terminal 100 is connected to the server 300 through the network 200. The network 200 can be a wide area network or a local area network, or a combination of the two.

[0066] See also Figure 2 When a user swipes their palm using a palm-swiping device, the first type of image acquisition device on the palm-swiping device captures a first original image containing the user's palm print (i.e., the target object), and the second type of image acquisition device on the palm-swiping device captures a second original image containing the user's palm print. The terminal 100 then encapsulates the first and second original images into a spatial alignment request and sends the spatial alignment request to the server 300 via the network 200. Subsequently, the server 300 responds to the spatial alignment request by using a specific cropping ratio to perform effective image cropping on the first and second original images, respectively, to obtain first and second cropped images, respectively. The server 300 then performs distortion correction processing on the first and second cropped images, respectively, to obtain first and second distortion-corrected images, respectively. Finally, the server 300 simultaneously performs distortion image cropping and spatial alignment on the first and second distortion-corrected images, to obtain spatially aligned images containing the user's palm print. After obtaining the spatially aligned image, server 300 can return the spatially aligned image to the terminal for display. It can also identify the user based on the spatially aligned image to obtain the user's identity information, and then perform corresponding business processing based on the identity information. For example, for a palm-swipe check-in service, after the user swipes their palm and passes the identity identification and verification, a check-in record for the user is generated; for a palm-swipe payment service, after the user swipes their palm and passes the identity identification and verification, the payment process for the bound account corresponding to the user's identity information is executed.

[0067] In some embodiments, the steps in the spatial alignment method can also be performed by the terminal 100, that is, the palm-swiping device spatially aligns the first and second original images. Specifically, the palm-swiping device uses a specific cropping ratio to effectively crop the first and second original images, respectively, to obtain first and second cropped images. The palm-swiping device then performs distortion correction processing on the first and second cropped images, respectively, to obtain first and second distortion-corrected images. Finally, the palm-swiping device simultaneously performs distortion image cropping and spatial alignment on the first and second distortion-corrected images, to obtain a spatially aligned image containing the user's palm print.

[0068] The spatial alignment method provided in the embodiments of the present application can also be implemented based on a cloud platform and through cloud technology. For example, the server 300 can be a cloud server. The cloud server can perform effective image cropping on the first original image and the second original image, respectively. Alternatively, the cloud server can perform distortion correction processing on the first cropped image and the second cropped image, respectively. Alternatively, the cloud server can perform distortion image cropping and spatial alignment on the first distortion-corrected image and the second distortion-corrected image simultaneously.

[0069] In some embodiments, there may also be a cloud storage, and the original image pair, or the first cropped image and the second cropped image, or the first distortion-corrected image and the second distortion-corrected image, etc. may be stored in the cloud storage, or the final spatially aligned image may be stored in the cloud storage. In this way, when performing actual business processing, the spatially aligned image can be directly obtained from the cloud storage to implement the corresponding business processing flow.

[0070] It's important to note that cloud technology refers to a managed technology that unifies hardware, software, and network resources within a wide or local area network (WAN) to enable data computing, storage, processing, and sharing. Cloud technology is a general term for network, information technology, integration technology, management platform technology, and application technology, all based on the cloud computing business model. It can form a resource pool for on-demand, flexible, and convenient use. Cloud computing technology will become a crucial support. Backend services for technical network systems, such as those for video sites, image sites, and more portals, require significant computing and storage resources. With the rapid development and application of the internet industry, every item will likely have its own unique identifier, requiring transmission to backend systems for logical processing. Data of varying levels will be processed separately, and data from various industries will require robust system support, which can be achieved through cloud computing.

[0071] Figure 3 is a structural diagram of an electronic device provided in an embodiment of the present application, Figure 3 The electronic device shown may be a spatial alignment device, which includes: at least one processor 310, a memory 350, at least one network interface 320, and a user interface 330. The various components in the spatial alignment device are coupled together via a bus system 340. It is understood that the bus system 340 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 340 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, the bus system 340 is not described in detail. Figure 3 Various buses are labeled as bus system 340 .

[0072] The processor 310 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0073] The user interface 330 includes one or more output devices 331 that enable presentation of media content, and one or more input devices 332 .

[0074] The memory 350 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, and the like. The memory 350 may optionally include one or more storage devices physically located away from the processor 310. The memory 350 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 350 described in the embodiments of the present application is intended to include any suitable type of memory. In some embodiments, the memory 350 is capable of storing data to support various operations, examples of which include programs, modules, and data structures, or subsets or supersets thereof, as exemplified below.

[0075] The operating system 351 includes system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic businesses and process hardware-based tasks; a network communication module 352 is used to reach other computing devices via one or more (wired or wireless) network interfaces 320. Exemplary network interfaces 320 include: Bluetooth, Wireless Compatibility Certification (WiFi), and Universal Serial Bus (USB); an input processing module 353 is used to detect one or more user inputs or interactions from one of the one or more input devices 332 and translate the detected inputs or interactions.

[0076] In some embodiments, the apparatus provided in the embodiments of the present application may be implemented in software. Figure 3 A spatial alignment device 354 stored in memory 350 is shown. This spatial alignment device 354 can be a spatial alignment device in an electronic device and can be software in the form of a program or plug-in. The device 354 includes the following software modules: an acquisition module 3541, an image cropping module 3542, a distortion correction module 3543, and a spatial alignment module 3544. These modules are logical and can be arbitrarily combined or further separated according to the functions they implement. The functions of each module are described below.

[0077] In some embodiments, the apparatus provided in the embodiments of the present application may be implemented in hardware. As an example, the apparatus provided in the embodiments of the present application may be a processor in the form of a hardware decoding processor, which is programmed to execute the spatial alignment method provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor may be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0078] The spatial alignment method provided in each embodiment of the present application can be executed by an electronic device, wherein the electronic device can be a server or a terminal, that is, the spatial alignment method provided in each embodiment of the present application can be executed by a server, or by a terminal, or by interaction between a server and a terminal.

[0079] Figure 4This is an optional flow chart of the spatial alignment method provided in the embodiment of the present application. Figure 4 The steps shown are explained as Figure 4 As shown, the execution subject of the spatial alignment method is taken as an example to illustrate that the method includes the following steps S101 to S104:

[0080] Step S101: obtaining a pair of original images to be aligned.

[0081] Here, the original image pair includes a first original image containing the target object acquired by a first type of image acquisition device and a second original image containing the target object acquired by a second type of image acquisition device.

[0082] The first type of image acquisition device and the second type of image acquisition device can be cameras located on both the biometric acquisition device. The first type of image acquisition device is used to capture a first original image of the target object, and the second type of image acquisition device is used to capture a second original image of the target object. In other words, the biometric acquisition device is a dual-camera acquisition device capable of simultaneously capturing two images of the target object. For example, the first type of image acquisition device can be an infrared camera located on the palm scanning device, and the second type of image acquisition device can be a color camera located on the palm scanning device. The infrared camera captures an infrared image containing the user's palm prints, and the color camera captures a color image containing the user's palm prints.

[0083] In the embodiment of the present application, since the biometric acquisition device simultaneously captures two images of the target object, and due to factors such as the design error of the camera and the camera parameters, the position of the target object in the two images will deviate. When using the two images captured by the biometric acquisition device for subsequent business processing, in order to improve the business processing accuracy and reduce the position deviation between the two images, it is necessary to first spatially align the two images to obtain a spatially aligned image, and then perform subsequent business processing based on the spatially aligned image.

[0084] Step S102 : Using a specific cropping ratio, effectively cropping the first original image and the second original image respectively, to obtain a first cropped image and a second cropped image respectively.

[0085] Due to the different shooting angles of the camera and the different distances between the target object and the camera, there will be blank areas of a certain size at the edges of the first original image and the second original image. The blank areas refer to the areas located around the target object but not containing the target object.

[0086] In an embodiment of the present application, after the first original image and the second original image are collected, effective image cropping can be performed on the first original image and the second original image. Effective image cropping refers to cropping the image to retain the valid area related to the specific business in the image and delete the invalid area that is not related to the specific business. For the palm-swiping field, when performing effective image cropping on the first original image and the second original image, the blank area in the first original image and the second original image can be cropped off to retain as much area containing the target object as possible in the middle part of the first original image and the second original image.

[0087] In an embodiment of the present application, a specific cropping ratio can be preset, and then the first original image and the second original image are effectively cropped according to the cropping ratio, thereby obtaining a first cropped image and a second cropped image. During implementation, the specific cropping ratio can be an unconventional ratio, where a conventional ratio can be, for example, a 16:9 ratio, a 4:3 ratio, etc., and an unconventional ratio is a ratio other than a conventional ratio, for example, a full-size ratio, i.e., a 1:1 ratio.

[0088] For example, in the embodiment of the present application, the first and second original images can be effectively cropped using a full-size ratio to obtain first and second cropped images, respectively. This allows for the complete preservation of all information in the first and second original images, thereby preserving the entire FOV of the first and second original images.

[0089] Step S103 : performing distortion correction processing on the first cropped image and the second cropped image respectively, to obtain a first distortion-corrected image and a second distortion-corrected image respectively.

[0090] In image processing, distortion refers to the image deformation caused by technical or physical factors during camera capture. Light entering the camera through a lens is a delicate and complex process. As light passes through the lens, it experiences loss and refraction, resulting in a certain degree of distortion in every optical lens. This distortion can degrade image quality and affect image recognition, analysis, and application. Therefore, distortion correction is necessary.

[0091] Typically, image distortion types include pincushion distortion and barrel distortion. Pincushion distortion typically occurs with telephoto lenses or at the telephoto end of a zoom lens, where straight lines appear to shrink inward, like a pillow. Barrel distortion is a distortion caused by the physical properties and structure of the lens elements in a camera lens, resulting in the image appearing barrel-shaped and expanding. This typically occurs at wide-angle or at the wide-angle end of a zoom lens, where straight lines at the edges of the image expand outward, forming a barrel shape. Barrel distortion is like taping paper to a ball, while pincushion distortion is like taping paper to a bowl.

[0092] The distortion correction process includes at least any one of the following correction methods: a geometric correction method, a correction plate correction method, a method based on adaptive filtering, and a correction method based on a convolution kernel.

[0093] Among them, the geometric correction method is a distortion correction method based on camera intrinsic and extrinsic parameters. This method works by calculating the camera's internal and extrinsic parameters to estimate the transformation matrix required for distortion correction. During implementation, the camera can be calibrated first, by repeatedly photographing a specific calibration object to obtain the camera's internal and extrinsic parameters. These parameters are then used to perform distortion correction. The advantage of the geometric correction method is that it achieves good correction results and relatively high accuracy. The correction plate correction method is a simple and effective distortion correction method. The principle of this method is to first capture an image of a correction plate of known shape, then measure the shape of the correction plate in the image, and finally use the measurement results to perform distortion correction. The advantage of the correction plate correction method is its simplicity of implementation, requiring only a correction plate of known shape, and its relatively high correction accuracy. The adaptive filtering method is a filtering method based on local image features. The idea of ​​this method is to determine the degree of distortion based on local image features and filter the local image features to achieve the purpose of distortion correction. The advantage of the adaptive filtering-based method is that it can adapt to different types and degrees of distortion, and can perform distortion correction processing without a calibration object. The convolution kernel-based correction method is a method based on the transformation matrix. The idea of ​​the convolution kernel-based correction method is to transform the image through the convolution kernel to achieve distortion correction. The convolution kernel is usually a square matrix, where each element in the matrix corresponds to a coefficient in the transformation matrix. The advantage of the convolution kernel-based correction method is that it is simple to implement. It only requires calculating the convolution kernel and the transformation matrix. Moreover, the convolution kernel-based correction method can adapt to a variety of distortion types and degrees of distortion.

[0094] Step S104 : performing distortion image cropping and spatial alignment on the first distortion-corrected image and the second distortion-corrected image simultaneously to obtain a spatially aligned image containing the target object.

[0095] In the embodiment of the present application, since the first cropped image and the second cropped image are respectively subjected to distortion correction processing, after the corresponding first distortion corrected image and the second distortion corrected image are obtained, the first distortion corrected image and the second distortion corrected image are corresponding deformed images. Therefore, when the deformed image is cropped, the effective FOV in the first original image and the second original image may be cropped out. If the distorted image cropping and spatial alignment process of the first distortion corrected image and the second distortion corrected image are performed simultaneously, the effective FOV in the first original image and the second original image can be avoided from being cropped out during the spatial alignment, so that the spatially aligned image retains more effective FOV in the first original image and the second original image.

[0096] In the spatial alignment method provided by the embodiment of the present application, when performing spatial alignment of an original image pair, the electronic device sequentially performs effective image cropping and distortion correction processing on the first original image and the second original image in the acquired original image pair, thereby obtaining a first distortion-corrected image and a second distortion-corrected image after distortion correction, respectively. Thereafter, the first distortion-corrected image and the second distortion-corrected image are simultaneously subjected to distortion image cropping and spatial alignment, thereby obtaining a spatially aligned image containing the target object. In this way, since the distortion image cropping and spatial alignment are performed simultaneously, it is possible to ensure that when the distortion image is cropped, the non-intersecting portions of the first original image and the second original image are not excessively cropped, that is, the effective FOV of the original image pair is not cropped. Therefore, the method of the embodiment of the present application can optimize the effective FOV between the first type of image acquisition device and the second type of image acquisition device in the field of identity recognition while ensuring that the original image pair has an alignment effect.

[0097] Below, taking a palm-swipe device as an example, an application scenario of the spatial alignment method provided in the embodiment of the present application is described. The embodiment of the present application can be applied to at least any of the following exemplary scenarios:

[0098] Scenario 1: A palm-swipe device can be used for clocking in and out. The spatial alignment system includes at least the palm-swipe device and the backend server of the clocking-in system. The palm-swipe device can be a dual-camera device with a first-type image acquisition device and a second-type image acquisition device. The first-type image acquisition device is used to capture infrared images, and the second-type image acquisition device is used to capture color images. When a user clocks in and out with a palm-swipe device, the palm-swipe device captures both an infrared image and a color image containing the user's palm print. The user's identity can be identified based on the infrared and color images.

[0099] To ensure the accuracy of identity recognition, the infrared image and the color image can be spatially aligned to obtain a spatially aligned image containing the user's palm print, which can then be used to identify the user. When spatially aligning the infrared image and the color image, the spatial alignment method provided in the embodiments of the present application can be used.

[0100] Scenario 2: The palm-swiping device can be a collection device for implementing the palm-swiping payment service, and the spatial alignment system includes at least the palm-swiping device and the backend server of the payment application corresponding to the palm-swiping payment service. For example, the payment application can be run on the palm-swiping device, and the palm-swiping device can also collect the user's palm print image, so as to identify the user based on the collected palm print image, determine the user's payment account according to the identification result, and deduct money from the payment account in the payment application. For another example, the palm-swiping device can also be connected to a terminal, on which the payment application can be run. The palm-swiping device collects the user's palm print image and sends the palm print image to the terminal. The terminal sends the user's palm print image to the backend server of the payment application through the payment application running. After the user is identified by the backend server of the payment application, the user's payment account is determined according to the identification result, and deduct money from the payment account in the payment application. Among them, the palm-scanning device can be a device with dual cameras, having a first type of image acquisition device and a second type of image acquisition device. The first type of image acquisition device is used to acquire infrared images, and the second type of image acquisition device is used to acquire color images. That is to say, the palm print images acquired by the palm-scanning device include infrared images and color images.

[0101] Similarly, to ensure the accuracy of identity recognition, the infrared image and the color image can be spatially aligned to obtain a spatially aligned image containing the user's palm print, and the user's identity can be identified using this spatially aligned image. When spatially aligning the infrared image and the color image, the spatial alignment method provided in the embodiments of the present application can be used to achieve this.

[0102] The following will take the above scenario 1 as an example to illustrate the spatial alignment method provided in the embodiment of the present application. Figure 5 This is another optional flow chart of the spatial alignment method provided in the embodiment of the present application, such as Figure 5 As shown, the method includes the following steps S201 to S211:

[0103] Step S201: The palm-swiping device captures a first original image containing a target object through a first type of image acquisition device, and captures a second original image containing a target object through a second type of image acquisition device.

[0104] Here, the first original image and the second original image constitute an original image pair. When the first original image and the second original image are acquired, the first type of image acquisition device and the second type of image acquisition device acquire the images simultaneously.

[0105] In step S202 , the palm-swiping device encapsulates the first original image and the second original image as an original image pair to be aligned into an identity authentication request.

[0106] Step S203: The palm-swiping device sends an identity authentication request to the server.

[0107] Here, the palm-swiping device may have a network module, and the identity authentication request is sent to the server through the network module.

[0108] In some embodiments, the palm-swiping device can also be connected to a terminal running an application for a specific service (e.g., identity authentication). After capturing the first and second original images, the palm-swiping device can send the first and second original images to the terminal. The terminal, running the application, encapsulates the first and second original images as an original image pair to be aligned into an identity authentication request, and then sends the identity authentication request to the server. In some embodiments, the terminal can send the identity authentication request using a protocol such as HTTP or Web Socket.

[0109] Step S204 : In response to the identity authentication request, the server obtains the first original image and the second original image in the pair of original images to be aligned.

[0110] In the embodiment of the present application, the first and second original images may be images exhibiting barrel distortion. Barrel distortion is a distortion phenomenon caused by the physical properties of the lens and the structure of the lens assembly in a camera lens, resulting in the image appearing to expand in a barrel shape. In barrel distortion, straight lines at the edges of the image expand outward, forming a barrel shape.

[0111] In step S205 , the server performs effective image cropping on the first original image and the second original image respectively using a specific cropping ratio, thereby obtaining a first cropped image and a second cropped image respectively.

[0112] In some embodiments, the specific cropping ratio may be an unconventional ratio; using the specific cropping ratio to perform effective image cropping on the first original image and the second original image respectively can be achieved in the following way: using an unconventional ratio to perform full-size effective image cropping on the first original image and the second original image respectively, and obtaining the first cropped image and the second cropped image accordingly.

[0113] In the embodiment of the present application, due to the use of an unconventional ratio, the first original image and the second original image are respectively cropped to full-size effective images, and the first original image and the second original image are images with barrel distortion, so the first cropped image and the second cropped image obtained after the effective image cropping are also images with barrel distortion.

[0114] In step S206 , the server performs distortion correction processing on the first cropped image and the second cropped image having barrel distortion, respectively, to obtain a first initial corrected image and a second initial corrected image having a pincushion shape.

[0115] In an embodiment of the present application, any correction method can be used to perform distortion correction processing on the first cropped image and the second cropped image with barrel distortion, respectively, wherein the correction methods include: geometric correction method, correction plate correction method, adaptive filtering-based method, and convolution kernel-based correction method.

[0116] After the first cropped image and the second cropped image with barrel distortion are respectively subjected to distortion correction processing, the resulting first initial corrected image and the second initial corrected image will have a pillow shape. That is, the left and right edges of the first initial corrected image and the second initial corrected image will shrink toward the middle, like a pillow.

[0117] In step S207 , the server uses black pixels to fill two arc regions in the first initial corrected image and the second initial corrected image having a pincushion shape, to obtain a first distortion-corrected image and a second distortion-corrected image.

[0118] Here, the first distortion-corrected image and the second distortion-corrected image respectively include two arc-shaped black-bordered regions formed after distortion correction processing is performed on the first and second original images having barrel distortion. The arc-shaped regions refer to the inwardly concave regions of the first and second initial corrected images formed in a pincushion shape. These two concave regions can be filled with black pixels, and the concave regions after being filled with black pixels are the aforementioned arc-shaped black-bordered regions.

[0119] It should be noted that the embodiment of the present application illustrates the distortion correction process by correcting only the inward distortion of the left and right edges of the first and second cropped images with barrel distortion. In this case, two symmetrical left and right circular arc regions are formed. After filling these two circular arc regions with black pixels, two arc-shaped black-bordered regions are formed. Of course, in other embodiments, the distortion correction process can also be performed by correcting the inward distortion of all four edges of the first and second cropped images with barrel distortion. In this way, two symmetrical left and right circular arc regions are formed, totaling four arc regions. After filling these four circular arc regions with black pixels, four arc-shaped black-bordered regions are formed.

[0120] In step S208 , the server spatially aligns the first distortion-corrected image having two arc-shaped black edge regions and the second distortion-corrected image having two arc-shaped black edge regions using a center-alignment method to obtain a spatially aligned overlapping image.

[0121] In some embodiments, see Figure 6 , Figure 6 The process of spatial alignment in step S208 is shown, which can be implemented by following steps S2081 to S2084:

[0122] Step S2081 : determining a first object center line of the target object in the first distortion-corrected image having the arc black edge region and a second object center line of the target object in the second distortion-corrected image having the arc black edge region.

[0123] Here, the first object center line is a line passing through the center point of the target object in the first distortion-corrected image and parallel to the left and right sides of the first distortion-corrected image having the circular arc black border area; the second object center line is a line passing through the center point of the target object in the second distortion-corrected image and parallel to the left and right sides of the second distortion-corrected image having the circular arc black border area.

[0124] Step S2082 : determining the relative positional relationship between the first distortion-corrected image having the arc-shaped black edge region and the second distortion-corrected image having the arc-shaped black edge region.

[0125] Here, the relative position relationship refers to the left-right relative relationship between the first distortion-corrected image with the circular arc black border area and the second distortion-corrected image with the circular arc black border area, that is, determining which image of the first distortion-corrected image with the circular arc black border area and the second distortion-corrected image with the circular arc black border area is located on the relative left and which image is located on the relative right.

[0126] Step S2083: Based on the relative position relationship, determine a first distance between a first image edge of the first distortion-corrected image having a circular arc black border area and a center line of the first object, and a second distance between a second image edge of the second distortion-corrected image having a circular arc black border area and a center line of the second object.

[0127] Here, the first image edge is an edge away from the second distortion-corrected image determined based on the relative position relationship, and the second image edge is an edge away from the first distortion-corrected image determined based on the relative position relationship.

[0128] Step S2084 : spatially aligning the first distortion-corrected image with the arc-shaped black border region and the second distortion-corrected image with the arc-shaped black border region based on the first distance and the second distance to obtain a spatially aligned overlapping image.

[0129] Here, a first movement direction and a first movement distance of the first distortion-corrected image with the arc-shaped black border region can be calculated based on the first distance and the second distance, respectively, and a second movement direction and a second movement distance of the second distortion-corrected image with the arc-shaped black border region can be calculated based on the first distance and the second distance. Then, the first distortion-corrected image with the arc-shaped black border region is moved according to the first movement direction and the first movement distance, and the second distortion-corrected image with the arc-shaped black border region is moved according to the second movement direction and the second movement distance, thereby completing spatial alignment of the first distortion-corrected image with the arc-shaped black border region and the second distortion-corrected image with the arc-shaped black border region, and obtaining a spatially aligned overlapping image. The spatially aligned overlapping image is the image of the overlapping area of ​​the first distortion-corrected image with the arc-shaped black border region and the second distortion-corrected image with the arc-shaped black border region after the two images are moved. The image of this overlapping area constitutes the spatially aligned image.

[0130] That is to say, in an embodiment of the present application, spatial alignment of the first distortion-corrected image with the circular arc black border area and the second distortion-corrected image with the circular arc black border area is performed based on the first distance and the second distance, which can be achieved in the following manner: first, based on the relative position relationship, the first alignment direction (i.e., the first movement direction) of the first distortion-corrected image with the circular arc black border area and the second alignment direction (i.e., the second movement direction) of the second distortion-corrected image with the circular arc black border area are determined; wherein the first alignment direction is opposite to the second alignment direction; then, based on the first distance and the second distance, the first movement distance of the first distortion-corrected image with the circular arc black border area is determined, and, based on the first distance and the second distance, the second movement distance of the second distortion-corrected image with the circular arc black border area is determined; finally, the first distortion-corrected image with the circular arc black border area is moved along the first alignment direction according to the first movement distance, and the second distortion-corrected image with the circular arc black border area is moved along the second alignment direction according to the second movement distance, to obtain overlapping images after spatial alignment. That is to say, after determining the first distance and the second distance, it is also necessary to calculate the first moving distance of the first distortion-corrected image with the arc black edge area based on the first distance and the second distance, and calculate the second moving distance of the second distortion-corrected image with the arc black edge area, and then use the first moving distance and the second moving distance to implement the spatial alignment process.

[0131] like Figure 7 The figure is a schematic diagram of the implementation principle of the spatial alignment method provided by the embodiment of the present application. When performing spatial alignment, the two images are aligned by moving left and right. Among them, the target object is Figure 7 The solid line frame corresponds to the first distortion-corrected image with a circular black border area, and the dotted line frame corresponds to the second distortion-corrected image with a circular black border area. Figure 7 The black border area of ​​the arc filled with black pixels is not shown in FIG. Assume that the width of the two images is the same and known, both are D, Figure 7 It represents the result after spatial alignment, so the center line of the first object and the center line of the second object are Figure 7The overlap is represented by the first object centerline (including the overlapping second object centerline) L. The relative positional relationship between the first distortion-corrected image with the arc-shaped black border region and the second distortion-corrected image with the arc-shaped black border region is: the first distortion-corrected image with the arc-shaped black border region is located to the left of the second distortion-corrected image with the arc-shaped black border region. Therefore, during spatial alignment, the first distortion-corrected image with the arc-shaped black border region should be moved to the right, and the second distortion-corrected image with the arc-shaped black border region should be moved to the left. Furthermore, based on the relative positional relationship, the first image edge of the first distortion-corrected image with the arc-shaped black border region can be determined as L1, and the second image edge of the second distortion-corrected image with the arc-shaped black border region can be determined as L2. Therefore, the first distance d1 between the first image edge L1 of the first distortion-corrected image with the arc-shaped black border region and the first object centerline L, and the second distance d2 between the second image edge L2 of the second distortion-corrected image with the arc-shaped black border region and the second object centerline L can be calculated.

[0132] Next, after obtaining the first distance d1 and the second distance d2, when spatially aligning the first distortion-corrected image with the arc black edge region and the second distortion-corrected image with the arc black edge region based on the first distance and the second distance, the first distance d1 and the second distance d2 can be calculated. Figure 7 The length of d22 is equal to D-d1, so we can calculate Figure 7 The length of d11 is equal to D-d2. After calculating d11 and d22, we can continue to calculate the first moving distance d1-d11 of the first distortion-corrected image with the arc black edge area to the right, and the second moving distance d2-d22 of the second distortion-corrected image with the arc black edge area to the left. In this way, the first distortion-corrected image with the arc black edge area and the second distortion-corrected image with the arc black edge area can be spatially aligned based on the calculated first moving distance d1-d11 and second moving distance d2-d22, respectively, to obtain the final spatially aligned overlapping image, that is, Figure 7 The area with hatching.

[0133] In other embodiments, see Figure 8 , Figure 8 The process of spatial alignment in step S208 is shown, and can also be implemented by the following steps S2085 to S2086:

[0134] Step S2085 : Determine a first center line of the first distortion-corrected image having the arc-shaped black edge region and a second center line of the second distortion-corrected image having the arc-shaped black edge region.

[0135] Since the palm is usually located in the middle of the image when swiping the palm (even if it is not exactly in the middle, the deviation to either side will not be too large), spatial alignment is performed with the first center line of the first distortion-corrected image with the circular black border area and the second center line of the second distortion-corrected image with the circular black border area as the alignment targets.

[0136] Here, the first center line is the central axis of the first distortion-corrected image of the entire arc black-border area, and the first center line is parallel to the left and right sides of the first distortion-corrected image of the arc black-border area; the second center line is the central axis of the second distortion-corrected image of the entire arc black-border area, and the second center line is parallel to the left and right sides of the second distortion-corrected image of the arc black-border area.

[0137] Step S2086 : spatially aligning the first distortion-corrected image with the arc-shaped black edge region and the second distortion-corrected image with the arc-shaped black edge region based on the first center line and the second center line to obtain a spatially aligned overlapping image.

[0138] In step S209 , the server crops the spatially aligned overlapping images to obtain a spatially aligned image containing the target object.

[0139] Here, the image cropping may be cropping the overlapping images after spatial alignment, thereby obtaining a spatially aligned image containing the target object.

[0140] In step S210 , the server performs identity recognition on the user based on the spatially aligned images to obtain an identity recognition result.

[0141] Step S211: The server processes the current service based on the identity recognition result.

[0142] The spatial alignment method provided in the embodiments of the present application simultaneously performs distorted image cropping and spatial alignment for first and second original images with barrel distortion. This ensures that, during distorted image cropping, non-intersecting portions of the first and second original images are not excessively cropped. In other words, the effective FOV of the original image pair is not cropped. Therefore, the method of the embodiments of the present application optimizes the effective FOV between the first and second image acquisition devices in the field of identity recognition while ensuring alignment of the original image pair.

[0143] In some embodiments, the first original image and the second original image may also be images with pincushion distortion. The following uses the first original image and the second original image with pincushion distortion as an example to illustrate the spatial alignment method provided in the embodiment of the present application. Figure 9 As shown, the method includes the following steps S301 to S311.

[0144] It should be noted that the implementation process of steps S301 to S311 is the same as that of steps S201 to S211. The only difference is that steps S301 to S311 perform spatial alignment processing on the first original image and the second original image with pincushion distortion, and steps S201 to S211 perform spatial alignment processing on the first original image and the second original image with barrel distortion. Therefore, the embodiment of the present application will not explain the repeated steps in detail. For specific explanations, please refer to the explanations of the corresponding steps in steps S201 to S211.

[0145] Step S301: The palm-swiping device captures a first original image containing a target object through a first type of image acquisition device, and captures a second original image containing a target object through a second type of image acquisition device.

[0146] Step S302: The palm-swiping device encapsulates the first original image and the second original image as an original image pair to be aligned into an identity authentication request.

[0147] Step S303: The palm-swiping device sends an identity authentication request to the server.

[0148] Step S304: In response to the identity authentication request, the server obtains the first original image and the second original image in the pair of original images to be aligned.

[0149] In step S305 , the server performs effective image cropping on the first original image and the second original image respectively using a specific cropping ratio, thereby obtaining a first cropped image and a second cropped image respectively.

[0150] The first original image and the second original image exhibit pincushion distortion; accordingly, the first cropped image and the second cropped image also exhibit pincushion distortion. Pincushion distortion refers to the phenomenon in which the originally straight lines of the target object shrink toward the center, like a pillow.

[0151] Step S306 : The server performs distortion correction processing on the first cropped image and the second cropped image with pincushion distortion, respectively, to obtain a first initial corrected image and a second initial corrected image with barrel shapes.

[0152] In an embodiment of the present application, any correction method can be used to perform distortion correction processing on the first cropped image and the second cropped image with pincushion distortion, respectively, wherein the correction methods include: geometric correction method, correction plate correction method, adaptive filtering-based method, and convolution kernel-based correction method.

[0153] After the distortion correction processing is performed on the first cropped image and the second cropped image with pincushion distortion, the resulting first and second initial corrected images will have a barrel shape. In other words, the left and right edges of the first and second initial corrected images will expand outward like a bucket.

[0154] In step S307 , the server uses black pixels to fill four arc regions in the barrel-shaped first initial corrected image and the second initial corrected image to obtain a first distortion-corrected image and a second distortion-corrected image.

[0155] Here, the first distortion-corrected image and the second distortion-corrected image each include four arc-shaped black-bordered regions formed after distortion correction processing is performed on the first and second original images with pincushion distortion. The four arc-shaped regions are arc-shaped regions located at the four corners of the barrel-shaped image formed after the distortion correction processing is performed on the first and second original images with pincushion distortion. In other words, after the barrel-shaped image is formed after the distortion correction processing is performed on the first and second original images with pincushion distortion, a rectangle can be circumscribed around the barrel-shaped image. In this case, four arc-shaped regions exist outside the barrel-shaped image and within the circumscribed rectangle.

[0156] In the embodiment of the present application, the four arc regions can be filled with black pixels, and the arc regions filled with black pixels are the above-mentioned arc black edge regions.

[0157] It should be noted that the embodiment of the present application uses the example of correcting the outward distortion of only the left and right edges of the first and second cropped images with pincushion distortion as an example to illustrate the distortion correction process. In this case, two pairs of symmetrical arc regions (i.e., four arc regions) are formed. After filling these two pairs of arc regions with black pixels, four arc regions with black edges are formed. Of course, in other embodiments, the distortion correction process can also be correcting the outward distortion of all four edges of the first and second cropped images with pincushion distortion. In this way, two symmetrical pairs of arc regions and two symmetrical pairs of arc regions are formed, totaling four pairs of arc regions, i.e., eight arc regions in total. After filling these eight arc regions with black pixels, eight arc regions with black edges are formed.

[0158] In step S308 , the server spatially aligns the first distortion-corrected image having four arc-shaped black edge regions and the second distortion-corrected image having four arc-shaped black edge regions using a center-alignment method to obtain a spatially aligned overlapping image.

[0159] In step S309 , the server crops the spatially aligned overlapping images to obtain a spatially aligned image containing the target object.

[0160] In step S310 , the server performs identity recognition on the user based on the spatially aligned images to obtain an identity recognition result.

[0161] Step S311: The server processes the current service based on the identity recognition result.

[0162] The spatial alignment method provided in the embodiments of the present application simultaneously performs distorted image cropping and spatial alignment for first and second original images with pincushion distortion. This ensures that, during distorted image cropping, non-intersecting portions of the first and second original images are not excessively cropped. In other words, the effective FOV of the original image pair is not cropped. Therefore, the method of the embodiments of the present application optimizes the effective FOV between the first and second image acquisition devices in the field of identity recognition while ensuring alignment of the original image pair.

[0163] Based on the above embodiments, the embodiments of the present application further provide a method for implementing distortion correction processing, wherein the first cropped image and the second cropped image with barrel distortion are respectively subjected to distortion correction processing, or the first cropped image and the second cropped image with pincushion distortion are respectively subjected to distortion correction processing, both of which can be subjected to distortion correction processing by the implementation method provided in the embodiments of the present application. Figure 10 FIG. 1 is a flow chart of a method for implementing distortion correction processing provided in an embodiment of the present application. Figure 10 As shown, the following steps S401 to S403 are included:

[0164] Step S401: The server obtains a first device internal parameter of a first image acquisition device and a second device internal parameter of a second image acquisition device.

[0165] In some embodiments, the server obtains the first device internal parameter of the first image acquisition device and the second device internal parameter of the second image acquisition device, which can be achieved in the following way: first, the server obtains a preset camera calibration standard value; then, using a preset calibration fixture, the first image acquisition device and the second image acquisition device are independently calibrated in sequence based on the camera calibration standard value, respectively, to obtain the first device internal parameter of the first image acquisition device and the second device internal parameter of the second image acquisition device.

[0166] In the embodiment of the present application, the first type of image acquisition device may be an infrared camera, and the second type of image acquisition device may be a color camera; accordingly, the first original image is an infrared image, and the second original image is a color image.

[0167] In step S402, the server obtains a first distortion coefficient of the first image acquisition device from the internal parameters of the first device, and uses the first distortion coefficient to perform distortion correction processing on the first cropped image to obtain a first distortion-corrected image.

[0168] Here, the first distortion coefficient refers to a coefficient that changes the image when the first image acquisition device captures the image. When performing distortion correction processing, the distortion correction coefficient for the first image acquisition device can be determined based on the first distortion coefficient. The distortion correction coefficient has the same numerical value as the first distortion coefficient and has an opposite sign. That is, if the first distortion correction coefficient is +0.5 (indicating that the image captured by the first image acquisition device is distorted 0.5 times toward a square), the corresponding distortion correction coefficient is -0.5 (indicating that during distortion correction processing, the distortion needs to be corrected 0.5 times in the negative direction).

[0169] In step S403, the server obtains a second distortion coefficient of the second image acquisition device from the internal parameters of the second device, and uses the second distortion coefficient to perform distortion correction processing on the second cropped image to obtain a second distortion-corrected image.

[0170] Here, the second distortion coefficient refers to a coefficient that changes the image when the second image acquisition device acquires the image. During distortion correction processing, a distortion correction coefficient for the second image acquisition device can be determined based on the second distortion coefficient. The distortion correction coefficient has the same magnitude as the second distortion coefficient and an opposite sign.

[0171] The following describes an exemplary application of the embodiments of the present application in a practical application scenario.

[0172] Because in the face recognition scene, the related technology is to use color images, infrared images, depth images, Figure 3 The images are spatially aligned. The palm swiping scenario does not care about the horizontal effective FOV, but is more concerned with the vertical effective FOV. In the palm swiping scenario, the color image and the infrared image are aligned according to a fixed position. The disadvantages of this fixed position alignment are: on the one hand, the spatial alignment effect is poor, and the palms are aligned only at the selected calibration position (for example, in 8CM calibration, only the palms at 8CM are aligned); on the other hand, the horizontal effective FOV is small, resulting in a small range of available distances for palm swiping and a poor user experience. Alternatively, in the palm swiping field, the two images are dynamically aligned based on the depth distance of the PSensor. Although this method can achieve dynamic alignment of the palms (alignment within 5 to 12 cm of the palm swiping), the horizontal effective FOV is small, resulting in a small range of available distances for palm swiping and a poor user experience.

[0173] Based on at least one of the above-mentioned problems existing in the related art, an embodiment of the present application provides a spatial alignment method, which optimizes the horizontal effective FOV of the camera in the palm-brushing area by using effective image cropping (Crop) with full-size resolution, aligning to the middle of the two images during spatial alignment, and using the arc black edges in the barrel-distorted original image.

[0174] Figure 11 This is a flow chart of a conventional palm-brushing method for obtaining an effective FOV in a palm-brushing field provided by an embodiment of the present application, i.e., a spatial alignment method. For the infrared image 111 and the color image 112 in the original image, effective image cropping of the original image is performed in sequence to obtain the original cropped image (including the infrared cropped image and the color cropped image); then, the original cropped image is subjected to distortion correction processing to obtain a distortion-corrected image (including the infrared distortion-corrected image and the color distortion-corrected image); then, the distortion-corrected image is cropped again to obtain a distortion-corrected cropped image (including the infrared distortion-corrected cropped image and the color distortion-corrected cropped image). Finally, the two distortion-corrected cropped images are spatially aligned to obtain a final spatially aligned image, i.e., a final effective FOV image.

[0175] Based on the conventional spatial alignment method, the embodiment of the present application further provides a method for optimizing the effective FOV of the camera palm brushing in the palm brushing field, such as Figure 12 As shown, the process includes the following steps S501 to S505:

[0176] In step S501 , the server uses a full-size FOV resolution with an unconventional ratio in an effective image cropping phase.

[0177] In the embodiment of this application, Figure 13 As shown, it is not recommended to perform effective image cropping (Crop) of regular ratios (for example, 16:9 ratio, 4:3 ratio, etc.) in the effective image cropping and distortion correction stage of the original image, that is, it is not recommended to use Figure 13 The process in line (1) of is used for spatial alignment, otherwise it will cause FOV loss. Figure 13 As shown in row (1) of , a 16:9 ratio effective image cropping is performed in both the original image effective image cropping and distortion correction stages, resulting in a loss of both horizontal and vertical FOV in the final spatially aligned image (with a field of view of 81°×99.6°) relative to the original image (with a field of view of 92°×116°).

[0178] Furthermore, in the embodiment of the present application, it is not recommended to perform effective image cropping of 16:9 ratio only in the distortion correction stage, that is, it is not recommended to use Figure 13 The process in line (2) in the figure above is spatially aligned, otherwise it will cause the FOV loss as described above. Figure 13As shown in row (2) of , an effective image cropping with a ratio of 16:9 is performed during the distortion correction stage of the original image, resulting in a lateral FOV loss in the final spatially aligned image (with a field of view of 81°×116°) relative to the original image (with a field of view of 92°×116°).

[0179] In the embodiment of the present application, it is recommended to perform effective image cropping according to the full-size FOV resolution of the unconventional ratio, that is, it is recommended to use Figure 13 The process in line (3) of is used for spatial alignment, so that the full-size FOV can be obtained. Figure 13 As shown in row (3) of , full-size effective image cropping with unconventional ratios is performed in both the original image effective image cropping and distortion correction stages, thereby ensuring that the final spatially aligned image (field of view is 92°×116°) has no FOV loss relative to the original image (field of view is 92°×116°).

[0180] Step S502: Calibrate each camera separately in the distortion correction stage.

[0181] like Figure 14 As shown in the figure, during factory production, due to factors such as design tolerance, stress, torque, glue characteristics, and gluing process, the camera may not be aligned vertically. Usually, the gold machine calibration method is used to calibrate these unevenness issues. After calibration, a set of parameters is adapted to all devices (i.e., infrared cameras and color cameras). Although this method is simple, it is prone to random deviations, such as Figure 15 As shown. In addition, a buffer space of 14° will be set in the upper layer based on the online sample experience value, which will also cause the available FOV to be damaged. Since the deviation of each device is highly random, a more reasonable solution should be to calibrate each device separately. That is to say, in an embodiment of the present application, a method of calibrating each camera separately is adopted in the distortion correction stage. In this way, a more accurate coordinate system can be provided to the upper layer, thereby removing the buffer space limitation. In the implementation process, a set of calibration fixtures (i.e., preset calibration fixtures) and calibration standards (for example, a set of camera calibration standard values) can be designed to calibrate each camera, thereby obtaining the device internal parameters of each camera used for distortion correction processing in the distortion correction stage, and then the original image can be subjected to distortion correction processing based on these device internal parameters.

[0182] Step S503: The server adopts center alignment in the spatial alignment stage.

[0183] like Figure 16 As shown in FIG. 1 , it is a schematic diagram of the lateral loss analysis provided by the embodiment of the present application. For the infrared camera and the color camera, the fields of view of the two cameras (such as Figure 16If the mid-infrared camera's field of view 161 and the color camera's field of view 162 do not intersect, FOV is lost. However, the available intersection area (i.e., the area where the infrared camera's field of view 161 and the color camera's field of view 162 intersect) varies with the distance between the palm and the camera. Closer distances result in less intersection and greater FOV loss, while greater distances result in greater intersection and less FOV loss. The palm's relative position in the color and infrared images differs, requiring spatial alignment, which also results in FOV loss.

[0184] The distance used in traditional scenes such as faces is basically more than 30 cm. At long distances, FOV has almost no effect, so it is basically aligned with the color image, such as Figure 17 The diagram below shows a comparison of alignment to the color image and alignment to the center. However, palm swiping is usually a close-up scene (the distance between the palm and the camera is about 5 to 8 cm). If the palm is aligned to a certain image, the FOV loss problem will be more obvious (for example, aligning to the color image will cause more loss of the non-intersection part of the infrared image). Therefore, the solution provided by the embodiment of the present application is: both images are aligned to the center, because the palm has the particularity of "the color camera and the infrared camera have the same lens" and "the palm is a plane", so it is feasible to align to the center.

[0185] In step S504 , during the spatial alignment phase, the server performs both distortion correction of the original image and spatial alignment, and uses distortion correction to correct the arc black edges of the barrel-shaped original image.

[0186] Here, after aligning to the middle, we continue to search for optimization space and find that: the blank black edge part of the color image corresponds exactly to the non-intersecting part of the infrared image; similarly, the blank black edge part of the infrared image corresponds exactly to the non-intersecting part of the color image; if we can optimize the blank black edge part of each image, we can optimize the effective FOV. Figure 18 As shown, Figure 18 The (a) part of the figure is the result of alignment to the color image. Figure 18 Part (b) of the figure is the result of alignment to the infrared image. Figure 18 Figure (c) shows the result of aligning the two images toward the center. It can be seen that aligning the two images toward the center optimizes the effective FOV of the two images.

[0187] In the embodiment of the present application, after optimizing the blank black borders of the two images, the effective FOV can be optimized. It is also found that the blank black borders of the two images just correspond to the arc area for distortion correction, such as Figure 19 The arc area 1901 in the arc area is useful FOV except for a small black edge 1902 in the middle (i.e., the arc black edge area). If the blank black edge parts of the two images can be optimized, the effective FOV can be optimized, such as Figure 20The following diagram compares the results of the normal process and direct spatial alignment based on the distortion-corrected original image. Therefore, further exploration of optimizing the black and white margins of the two images leads to the following conclusion: under the normal process, the black and white margins of the two images directly lose FOV; however, combining "distortion correction processing and spatial alignment" achieves the full effective FOV, with only a slight loss of FOV at the arc.

[0188] In summary, the embodiment of the present application further provides an overall process of a spatial alignment method for optimizing FOV, such as Figure 21 (b) process in which, Figure 21 The process (a) in Figure 1 is the overall flow of the spatial alignment method before optimization. The spatial alignment method for optimizing FOV combines distortion correction with spatial alignment, so that the FOV remains almost intact after spatial alignment.

[0189] Step S505: completing effective FOV optimization.

[0190] The spatial alignment method provided in the embodiments of this application can, in a specific case, optimize the effective FOV from 76.8° x 106° in related art methods to 95° x 116°. Furthermore, the palm-brushing distance can be optimized from 6.5-15 cm to 5-15 cm. In other words, a larger effective FOV can be achieved at a wider palm-brushing distance, effectively increasing the effective FOV in the palm-brushing range.

[0191] It is understandable that in the embodiments of the present application, if the content involves user information, such as the user's palm prints and other information, and if it involves data related to user information or corporate information, when the embodiments of the present application are applied to specific products or technologies, it is necessary to obtain user permission or consent, or to blur this information to eliminate the correspondence between this information and the user; and the relevant data collection and processing should be strictly in accordance with the requirements of relevant national laws and regulations when applied in examples, and the informed consent or separate consent of the personal information subject should be obtained, and subsequent data use and processing should be carried out within the scope of authorization of laws and regulations and the personal information subject.

[0192] The following continues to describe the exemplary structure of the spatial alignment device 354 provided in the embodiment of the present application implemented as a software module. In some embodiments, such as Figure 3As shown, the spatial alignment device 354 includes: an acquisition module for acquiring an original image pair to be aligned, wherein the original image pair includes a first original image containing a target object acquired by a first type of image acquisition device and a second original image containing the target object acquired by a second type of image acquisition device; an image cropping module for adopting a specific cropping ratio to perform effective image cropping on the first original image and the second original image, respectively, to obtain a first cropped image and a second cropped image accordingly; a distortion correction module for performing distortion correction processing on the first cropped image and the second cropped image, respectively, to obtain a first distortion corrected image and a second distortion corrected image accordingly; a spatial alignment module for simultaneously performing distortion image cropping and spatial alignment on the first distortion corrected image and the second distortion corrected image, to obtain a spatially aligned image containing the target object.

[0193] In some embodiments, the first original image and the second original image are images with barrel distortion; the distortion correction module is further used to: perform distortion correction processing on the first cropped image and the second cropped image with the barrel distortion, respectively, to obtain a first initial corrected image and a second initial corrected image with a pincushion shape; use black pixels to fill two arc areas in the first initial corrected image and the second initial corrected image with a pincushion shape, to obtain the first distortion corrected image and the second distortion corrected image; wherein the first distortion corrected image and the second distortion corrected image respectively include: two arc black edge areas formed after the distortion correction processing is performed on the first original image and the second original image with barrel distortion.

[0194] In some embodiments, the spatial alignment module is further used to: adopt a middle alignment method to spatially align the first distortion-corrected image having the circular arc black edge area and the second distortion-corrected image having the circular arc black edge area to obtain an overlapping image after spatial alignment; perform the image cropping on the overlapping image after spatial alignment to obtain a spatially aligned image containing the target object.

[0195] In some embodiments, the spatial alignment module is further used to: determine a first object centerline of the target object in the first distortion-corrected image having the circular arc black border area, and a second object centerline of the target object in the second distortion-corrected image having the circular arc black border area; determine a relative positional relationship between the first distortion-corrected image having the circular arc black border area and the second distortion-corrected image having the circular arc black border area; based on the relative positional relationship, determine a first distance between a first image edge of the first distortion-corrected image having the circular arc black border area and the first object centerline, and a second distance between a second image edge of the second distortion-corrected image having the circular arc black border area and the second object centerline; wherein the first image edge is an edge away from the second distortion-corrected image determined based on the relative positional relationship, and the second image edge is an edge away from the first distortion-corrected image determined based on the relative positional relationship; based on the first distance and the second distance, spatially align the first distortion-corrected image having the circular arc black border area and the second distortion-corrected image having the circular arc black border area to obtain an overlapping image after spatial alignment.

[0196] In some embodiments, the spatial alignment module is also used to: determine the first alignment direction of the first distortion-corrected image having the circular arc black border area and the second alignment direction of the second distortion-corrected image having the circular arc black border area based on the relative position relationship; wherein the first alignment direction is opposite to the second alignment direction; determine the first movement distance of the first distortion-corrected image having the circular arc black border area based on the first distance and the second distance, and determine the second movement distance of the second distortion-corrected image having the circular arc black border area based on the first distance and the second distance; move the first distortion-corrected image having the circular arc black border area along the first alignment direction according to the first movement distance, and move the second distortion-corrected image having the circular arc black border area along the second alignment direction according to the second movement distance to obtain the overlapping image after spatial alignment.

[0197] In some embodiments, the spatial alignment module is also used to: determine a first center line of the first distortion-corrected image having the circular arc black edge area and a second center line of the second distortion-corrected image having the circular arc black edge area; based on the first center line and the second center line, spatially align the first distortion-corrected image having the circular arc black edge area and the second distortion-corrected image having the circular arc black edge area to obtain an overlapping image after spatial alignment.

[0198] In some embodiments, the first original image and the second original image are images with pincushion distortion; the distortion correction module is further used to: perform distortion correction processing on the first cropped image and the second cropped image with the pincushion distortion, respectively, to obtain a first initial corrected image and a second initial corrected image with a barrel shape; use black pixels to fill the four arc regions in the first initial corrected image and the second initial corrected image with a barrel shape, to obtain the first distortion corrected image and the second distortion corrected image; wherein the first distortion corrected image and the second distortion corrected image respectively include: four arc black-edge regions formed after the distortion correction processing is performed on the first original image and the second original image with the pincushion distortion.

[0199] In some embodiments, the specific cropping ratio is an unconventional ratio; the image cropping module is further used to: use the unconventional ratio to perform full-size effective image cropping on the first original image and the second original image, respectively, to obtain the first cropped image and the second cropped image accordingly.

[0200] In some embodiments, the distortion correction module is further used to: obtain a first device internal parameter of the first image acquisition device and a second device internal parameter of the second image acquisition device; obtain a first distortion coefficient of the first image acquisition device from the first device internal parameter, and use the first distortion coefficient to perform distortion correction processing on the first cropped image to obtain the first distortion corrected image; obtain a second distortion coefficient of the second image acquisition device from the second device internal parameter, and use the second distortion coefficient to perform distortion correction processing on the second cropped image to obtain the second distortion corrected image.

[0201] In some embodiments, the distortion correction module is further used to: obtain a preset camera calibration standard value; use a preset calibration fixture to independently calibrate the first image acquisition device and the second image acquisition device in sequence based on the camera calibration standard value, and obtain the first device internal parameter of the first image acquisition device and the second device internal parameter of the second image acquisition device.

[0202] In some embodiments, the first type of image acquisition device is an infrared camera, and the second type of image acquisition device is a color camera; the first original image is an infrared image, and the second original image is a color image.

[0203] It should be noted that the description of the device embodiment of the present application is similar to the description of the method embodiment described above, and has similar beneficial effects as the method embodiment, so it will not be repeated. For technical details not disclosed in the device embodiment, please refer to the description of the method embodiment of the present application for understanding.

[0204] The present invention provides a computer program product comprising executable instructions, which are computer instructions stored in a computer-readable storage medium. When a processor of an electronic device reads the executable instructions from the computer-readable storage medium and executes the executable instructions, the electronic device performs the method described in the present invention.

[0205] The embodiment of the present application provides a storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will execute the method provided by the embodiment of the present application, for example, Figure 4 In some embodiments, the storage medium may be a computer-readable storage medium, such as a ferroelectric random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disk, or a compact disk read-only memory (CD-ROM); or various devices including one or any combination of the above memories.

[0206] In some embodiments, executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0207] As an example, the executable instructions may, but need not necessarily, correspond to a file in a file system, may be stored as part of a file storing other programs or data, for example, in one or more scripts in a Hypertext Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions). As an example, the executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located in one location, or on multiple electronic devices distributed in multiple locations and interconnected by a communication network.

[0208] The above description is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, and improvements made within the spirit and scope of the present application are included in the scope of protection of the present application.

Claims

1. A spatial alignment method, characterized in that: The method comprises: Acquire an original image pair to be aligned, the original image pair comprising a first original image containing a target object acquired by a first type of image acquisition device and a second original image containing the target object acquired by a second type of image acquisition device; Using a specific cropping ratio, effectively cropping the first original image and the second original image, respectively, to obtain a first cropped image and a second cropped image; performing distortion correction processing on the first cropped image and the second cropped image respectively, to obtain a first distortion-corrected image and a second distortion-corrected image respectively; Distorted image cropping and spatial alignment are performed simultaneously on the first distortion-corrected image and the second distortion-corrected image to obtain a spatially aligned image containing the target object.

2. The method according to claim 1, characterized in that The first original image and the second original image are images with barrel distortion; The performing distortion correction processing on the first cropped image and the second cropped image respectively to obtain a first distortion-corrected image and a second distortion-corrected image respectively includes: performing distortion correction processing on the first cropped image and the second cropped image having the barrel distortion, respectively, to obtain a first initial corrected image and a second initial corrected image having a pincushion shape; Black pixels are used to fill two arc regions in a first initial corrected image and a second initial corrected image having a pincushion shape, thereby obtaining the first distortion-corrected image and the second distortion-corrected image; wherein the first distortion-corrected image and the second distortion-corrected image respectively include: two arc black-edge regions formed after distortion correction processing is performed on the first original image and the second original image having barrel distortion.

3. The method according to claim 2, characterized in that The step of simultaneously performing distortion image cropping and spatial alignment on the first distortion-corrected image and the second distortion-corrected image to obtain a spatially aligned image containing the target object includes: Using a center alignment method, spatially aligning the first distortion-corrected image having the arc black edge region and the second distortion-corrected image having the arc black edge region to obtain a spatially aligned overlapping image; The image is cropped on the spatially aligned overlapping images to obtain a spatially aligned image containing the target object.

4. The method according to claim 3, characterized in that The method of spatially aligning the first distortion-corrected image having the arc black edge region and the second distortion-corrected image having the arc black edge region by aligning toward the middle to obtain an overlapping image after spatial alignment includes: Determining a first object center line of the target object in the first distortion-corrected image having the circular arc black edge region and a second object center line of the target object in the second distortion-corrected image having the circular arc black edge region; determining a relative positional relationship between the first distortion-corrected image having the arc-shaped black edge region and the second distortion-corrected image having the arc-shaped black edge region; Determining, based on the relative positional relationship, a first distance between a first image edge of the first distortion-corrected image having the arc-shaped black border region and the centerline of the first object, and a second distance between a second image edge of the second distortion-corrected image having the arc-shaped black border region and the centerline of the second object; wherein the first image edge is an edge determined based on the relative positional relationship and away from the second distortion-corrected image, and the second image edge is an edge determined based on the relative positional relationship and away from the first distortion-corrected image; Based on the first distance and the second distance, the first distortion-corrected image having the arc black edge region and the second distortion-corrected image having the arc black edge region are spatially aligned to obtain a spatially aligned overlapping image.

5. The method according to claim 4, characterized in that The step of spatially aligning the first distortion-corrected image having the arc black edge region and the second distortion-corrected image having the arc black edge region based on the first distance and the second distance to obtain a spatially aligned overlapping image includes: Determining, based on the relative positional relationship, a first alignment direction of the first distortion-corrected image having the arc-shaped black border region and a second alignment direction of the second distortion-corrected image having the arc-shaped black border region; wherein the first alignment direction is opposite to the second alignment direction; Determining a first movement distance of a first distortion-corrected image having the arc-shaped black border region based on the first distance and the second distance, and determining a second movement distance of a second distortion-corrected image having the arc-shaped black border region based on the first distance and the second distance; The first distortion-corrected image having the arc black border area is moved along the first alignment direction according to the first movement distance, and the second distortion-corrected image having the arc black border area is moved along the second alignment direction according to the second movement distance to obtain the overlapping image after spatial alignment.

6. The method according to claim 3, characterized in that The method of spatially aligning the first distortion-corrected image having the arc black edge region and the second distortion-corrected image having the arc black edge region by aligning toward the middle to obtain an overlapping image after spatial alignment includes: Determining a first center line of the first distortion-corrected image having the arc-shaped black edge region and a second center line of the second distortion-corrected image having the arc-shaped black edge region; Based on the first center line and the second center line, the first distortion-corrected image having the arc black edge region and the second distortion-corrected image having the arc black edge region are spatially aligned to obtain a spatially aligned overlapping image.

7. The method according to claim 1, characterized in that The first original image and the second original image are images with pincushion distortion; The performing distortion correction processing on the first cropped image and the second cropped image respectively to obtain a first distortion-corrected image and a second distortion-corrected image respectively includes: performing distortion correction processing on the first cropped image and the second cropped image having the pincushion distortion, respectively, to obtain a first initial corrected image and a second initial corrected image having a barrel shape; Black pixels are used to fill four arc regions in the first initial corrected image and the second initial corrected image having a barrel shape, thereby obtaining the first distortion-corrected image and the second distortion-corrected image. The first distortion-corrected image and the second distortion-corrected image respectively include: four arc black-edge regions formed after distortion correction processing is performed on the first original image and the second original image having pincushion distortion.

8. The method according to claim 1, characterized in that The specific cropping ratio is an unconventional ratio; The step of using a specific cropping ratio to perform effective image cropping on the first original image and the second original image, respectively, to obtain a first cropped image and a second cropped image, respectively, includes: The first original image and the second original image are respectively cropped into full-size effective images using the unconventional ratio, thereby obtaining the first cropped image and the second cropped image respectively.

9. The method according to claim 1, characterized in that The performing distortion correction processing on the first cropped image and the second cropped image respectively to obtain a first distortion-corrected image and a second distortion-corrected image respectively includes: Acquire a first device internal parameter of the first image acquisition device and a second device internal parameter of the second image acquisition device; Obtaining a first distortion coefficient of the first image acquisition device from an internal parameter of the first device, and performing distortion correction processing on the first cropped image using the first distortion coefficient to obtain a first distortion-corrected image; A second distortion coefficient of the second image acquisition device is obtained from the internal parameter of the second device, and the second distortion coefficient is used to perform distortion correction processing on the second cropped image to obtain the second distortion-corrected image.

10. The method according to claim 9, characterized in that The obtaining of a first device internal parameter of the first image acquisition device and a second device internal parameter of the second image acquisition device includes: Get the preset camera calibration standard value; Using a preset calibration fixture, the first image acquisition device and the second image acquisition device are independently calibrated in sequence based on the camera calibration standard value, thereby obtaining a first device internal parameter of the first image acquisition device and a second device internal parameter of the second image acquisition device.

11. The method according to any one of claims 1 to 10, characterized in that The first type of image acquisition device is an infrared camera, and the second type of image acquisition device is a color camera; the first original image is an infrared image, and the second original image is a color image.

12. A spatial alignment device, characterized in that: The device comprises: an acquisition module, configured to acquire an original image pair to be aligned, the original image pair comprising a first original image containing a target object acquired by a first type of image acquisition device and a second original image containing the target object acquired by a second type of image acquisition device; an image cropping module, configured to perform effective image cropping on the first original image and the second original image respectively using a specific cropping ratio, thereby obtaining a first cropped image and a second cropped image respectively; a distortion correction module, configured to perform distortion correction processing on the first cropped image and the second cropped image respectively, to obtain a first distortion-corrected image and a second distortion-corrected image respectively; The spatial alignment module is used to simultaneously perform distortion image cropping and spatial alignment on the first distortion-corrected image and the second distortion-corrected image to obtain a spatially aligned image containing the target object.

13. An electronic device, characterized in that: include: a memory for storing executable instructions; A processor, configured to implement the spatial alignment method according to any one of claims 1 to 11 when executing the executable instructions stored in the memory.

14. A computer-readable storage medium, characterized in that Executable instructions are stored, which are used to cause a processor to execute the executable instructions to implement the spatial alignment method described in any one of claims 1 to 11.

15. A computer program product comprising executable instructions stored in a computer-readable storage medium; When the processor of the electronic device reads the executable instructions from the computer-readable storage medium and executes the executable instructions, the spatial alignment method according to any one of claims 1 to 11 is implemented.

Images