Image processing method, device, storage medium and computer program product

Abstract

The present application provides an image processing method, device, storage medium and computer program product, which are applied to various image acquisition scenarios such as cloud technology, artificial intelligence, smart transportation and vehicle-mounted; the image processing method includes: performing image acquisition on the object to be acquired, obtaining the target acquisition distance and the image to be detected; determining the target image area corresponding to the target acquisition distance based on the correspondence between the acquisition distance and the image area, wherein the correspondence between the acquisition distance and the image area represents the maximum image area with the lowest degree of image distortion corresponding to different acquisition distances; detecting the imaging area of ​​the object to be acquired in the image to be detected, obtaining the imaging area to be detected; when the imaging area to be detected is within the target image area, determining the imaging area to be detected as the acquired image of the object to be acquired. Through the present application, the quality of image acquisition can be improved.

Description

Technical Field

[0001] The present application relates to image processing technology in the field of computer applications, and in particular to an image processing method, device, storage medium and computer program product. Background Art

[0002] When collecting images based on the principle of optical imaging, there is a non-linear propagation characteristic, that is, the light is deflected after passing through the lens plane; the non-linear propagation characteristic will affect the distortion of the image. Since the light passing through the lens plane at different angles has different corresponding deflection angles, the distortion is also different. For example, from the center of the lens plane to the surroundings, the deflection angle gradually increases, and the distortion also gradually increases. Therefore, the collected image often has distortion, which affects the image collection quality. Summary of the invention

[0003] The embodiments of the present application provide an image processing method, apparatus, device, computer-readable storage medium, and computer program product, which can improve image acquisition quality.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] The present application provides an image processing method, including:

[0006] Perform image acquisition on the object to be collected to obtain the target acquisition distance and the image to be detected;

[0007] Based on the correspondence between the acquisition distance and the image area, determining the target image area corresponding to the target acquisition distance, wherein the correspondence between the acquisition distance and the image area represents the maximum image area with the lowest degree of image distortion corresponding to different acquisition distances;

[0008] Detecting an imaging area of ​​the object to be collected in the image to be detected to obtain an imaging area to be detected;

[0009] When the imaging area to be detected is located within the target image area, the imaging area to be detected is determined as the captured image of the object to be captured.

[0010] The present application provides an image processing device, including:

[0011] An image acquisition module is used to acquire images of the object to be acquired and obtain the target acquisition distance and the image to be detected;

[0012] A region determination module, configured to determine a target image region corresponding to the target acquisition distance based on a correspondence between the acquisition distance and the image region, wherein the correspondence between the acquisition distance and the image region represents a maximum image region with the lowest degree of image distortion corresponding to different acquisition distances;

[0013] A target detection module, used to detect the imaging area of ​​the object to be collected in the image to be detected, and obtain the imaging area to be detected;

[0014] The image judgment module is used to determine the imaging area to be detected as the acquisition image of the object to be acquired when the imaging area to be detected is located within the target image area.

[0015] In an embodiment of the present application, the image processing device also includes a region calibration module, which is used to perform image acquisition on the first object sample at different acquisition positions at different acquisition distances to obtain a first image sample set corresponding to different acquisition positions at the same acquisition distance; obtain the degree of distortion of each first image sample in the first image sample set, and determine a target image sample set whose degree of distortion is lower than a distortion degree threshold from the first image sample set; determine the maximum area formed by the target image sample set as the image area corresponding to the acquisition distance; and determine the image areas corresponding to different acquisition distances as the correspondence between the acquisition distance and the image area.

[0016] In an embodiment of the present application, the area determination module is also used to determine, based on the target acquisition distance, from the correspondence between the acquisition distance and the image area, an acquisition distance that is the smallest difference from the target acquisition distance and is smaller than the target acquisition distance when the acquisition distance granularity is smaller than the granularity threshold, wherein the interval distance between different acquisitions is positively correlated with the acquisition distance granularity; in the correspondence between the acquisition distance and the image area, the image area corresponding to the acquisition distance that is the smallest difference from the target acquisition distance and is smaller than the target acquisition distance is determined as the target image area.

[0017] In an embodiment of the present application, the area determination module is also used to, when the acquisition distance granularity is greater than or equal to the granularity threshold, determine the two acquisition distances with the smallest difference from the target acquisition distance from the correspondence between the acquisition distances and the image areas, and obtain a first acquisition distance and a second acquisition distance; determine the first image area corresponding to the first acquisition distance and the second image area corresponding to the second acquisition distance from the correspondence between the acquisition distances and the image areas; obtain a first distance difference between the target acquisition distance and the first acquisition distance, and obtain a second distance difference between the target acquisition distance and the second acquisition distance; obtain a first weight that is positively correlated with the first distance difference, and a second weight that is positively correlated with the second distance difference; and determine the target image area based on the fusion result of the first weight and the second image area, and the fusion result of the second weight and the first image area.

[0018] In an embodiment of the present application, the target detection module is used to divide the image to be detected into N grids and determine M bounding boxes corresponding to each grid, wherein N and M are both positive integers; determine the bounding box confidence based on the characteristics of the grids and the characteristics of each of the bounding boxes, wherein the bounding box confidence is determined based on at least one of the possibility that the bounding box includes the imaging area of ​​the object to be collected and the accuracy of the bounding box; and determine the bounding box corresponding to the maximum bounding box confidence as the imaging area to be detected of the object to be collected in the image to be detected.

[0019] In an embodiment of the present application, the image processing device also includes a model training module, which is used to obtain a second image sample carrying an object annotation area, wherein the object annotation area refers to an imaging area of ​​a second object sample in the second image sample; using a detection model to be trained, predicting the imaging area of ​​the second object sample in the second image sample to obtain an object prediction area, wherein the detection model to be trained is a neural network model to be trained for predicting the imaging area of ​​an object; based on the difference between the object prediction area and the object annotation area, training the detection model to be trained to obtain the detection model.

[0020] In an embodiment of the present application, the image judgment module is further used to determine that image acquisition of the object to be acquired has failed when an intersection area between the imaging area to be detected and the target image area is smaller than the imaging area to be detected, or when the imaging area to be detected is located outside the target image area; when image acquisition of the object to be acquired has failed, first acquisition prompt information is generated based on imaging deviation information of the imaging area to be detected deviating from the target image area, wherein the first acquisition prompt information is prompt information for indicating that the imaging area of ​​the object to be acquired is moved to the target image area.

[0021] In an embodiment of the present application, the image judgment module is also used to obtain a circular representation corresponding to the imaging area to be detected to obtain first circle information; obtain a circular representation corresponding to the target image area to obtain second circle information; determine the center distance and radius difference based on the first circle information and the second circle information; when the center distance is less than or equal to the radius difference, determine that the imaging area to be detected is located within the target image area.

[0022] In an embodiment of the present application, the image judgment module is further used to determine that the imaging area to be detected is located within the target image area when an intersection area between the imaging area to be detected and the target image area is the imaging area to be detected.

[0023] In an embodiment of the present application, the image acquisition module is also used to display an authorization control, and in response to an authorization request operation on the authorization control, perform image acquisition on the object to be acquired to obtain the target acquisition distance and the image to be detected.

[0024] In an embodiment of the present application, the image judgment module is further used to compare the acquired image with a standard image corresponding to the object to be acquired, wherein the standard image is an imaging area bound to the object to be acquired, and the standard image is used for information authentication; when the comparison result indicates that the acquired image is similar to the standard image, it is determined that the information authentication is successful; when the information authentication is successful, the target processing corresponding to the authorization request operation is executed, wherein the target processing includes at least one of asset transfer, login and basic information update.

[0025] In an embodiment of the present application, the object to be collected includes at least one of a palm part, an eye part, a lip part, a finger part and a graphic.

[0026] In an embodiment of the present application, the area determination module is also used to determine the target image area corresponding to the target acquisition distance from the correspondence between the acquisition distance and the image area when the target acquisition distance is within the acquisition distance range corresponding to different acquisition distances; and generate second acquisition prompt information when the target acquisition distance is outside the acquisition distance range corresponding to different acquisition distances.

[0027] The present application provides an image processing device, including:

[0028] A memory for storing computer executable instructions;

[0029] The processor is used to implement the image processing method provided in the embodiment of the present application when executing the computer executable instructions stored in the memory.

[0030] An embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to implement the image processing method provided in the embodiment of the present application when executed by a processor.

[0031] An embodiment of the present application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, the image processing method provided by the embodiment of the present application is implemented.

[0032] The embodiments of the present application have at least the following beneficial effects: by presetting the maximum image areas corresponding to different acquisition distances, it is possible to determine the maximum image area with the lowest image distortion corresponding to the acquisition distance of the object to be acquired (called the target image area) during image acquisition, and then, when the imaging area to be detected in the acquired image to be detected is located within the target image area, the imaging area to be detected is determined as the acquisition image of the object to be acquired; in this way, the degree of image distortion of the acquired image is reduced, thereby improving the quality of image acquisition. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a schematic diagram of pinhole imaging;

[0034] Figure 2 is an exemplary image acquisition schematic diagram provided in an embodiment of the present application;

[0035] Figure 3 is a schematic diagram of the architecture of an image processing system provided in an embodiment of the present application;

[0036] Figure 4 The embodiment of this application provides Figure 3 A schematic diagram of the composition structure of the terminal in FIG.

[0037] Figure 5 This is a schematic diagram of the image processing method provided in the embodiment of the present application. Figure 1 ;

[0038] Figure 6 This is a schematic diagram of the image processing method provided in the embodiment of the present application. Figure 2 ;

[0039] Figure 7 This is a schematic diagram of the image processing method provided in the embodiment of the present application. Figure 3 ;

[0040] Figure 8 is a schematic diagram of an exemplary application scenario provided by an embodiment of the present application;

[0041] Fig. 9 is a schematic diagram of an exemplary process of judging a captured palm image provided by an embodiment of the present application;

[0042] Fig.10 is an exemplary target detection schematic diagram provided in an embodiment of the present application;

[0043] Fig.11 is another exemplary target detection schematic diagram provided in an embodiment of the present application;

[0044] Fig.12 is a schematic diagram of an exemplary method of determining a palm area provided in an embodiment of the present application;

[0045] Fig.13 This is another exemplary target detection schematic diagram provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0047] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0048] In the following description, the terms "first\second" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0049] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meaning as those commonly understood by those skilled in the art of the present application. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0050] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0051] 1) Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0052] 2) Machine Learning (ML) is a multi-disciplinary subject involving probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specializes in studying how computers simulate or implement human learning behaviors to acquire new knowledge or skills; reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning usually includes technologies such as artificial neural networks, belief networks, reinforcement learning, transfer learning, and inductive learning.

[0053] 3) Artificial neural network is a mathematical model that imitates the structure and function of biological neural networks. The exemplary structures of artificial neural networks in the embodiments of the present application include graph convolutional networks (Graph Convolutional Network, GCN, a neural network for processing graph-structured data), deep neural networks (Deep Neural Networks, DNN), convolutional neural networks (Convolutional Neural Network, CNN) and recurrent neural networks (Recurrent Neural Network, RNN), neural state machines (Neural State Machine, NSM) and phase function neural networks (Phase-Functioned Neural Network, PFNN), etc. The detection model and the detection model to be trained involved in the embodiments of the present application are models corresponding to artificial neural networks.

[0054] 4) Color image, an image obtained by collecting natural light by a color sensor (Sensor); in the embodiment of the present application, when the image obtained by image acquisition of the object to be acquired is a color image, multiple color images can be screened based on at least one of the imaging angle, imaging size, imaging centering and image resolution of the object to be acquired in the color image to obtain the image to be detected.

[0055] 5) Depth Image: an image obtained by collecting and analyzing speckle structures with an infrared sensor; in three-dimensional (3D) computer graphics and computer vision, a depth image is an image or image channel that includes information about the distance from the surface of a scene object to a viewpoint; it is also called a range image; each pixel of a depth image represents the vertical distance between the plane of the depth camera and the plane of the object being photographed, and is usually represented by 16 bits in millimeters. In the image processing method provided in the embodiment of the present application, it is used for living body recognition, and image processing is performed when the object to be collected is determined to be a living body based on the living body recognition result, and it is used to assist in determining the target collection distance.

[0056] 6) Infrared image, an image obtained by collecting infrared light by an infrared sensor, including thermal information of the imaging object; in the image processing method provided in the embodiment of the present application, it is used for living body recognition, and image processing is performed when it is determined that the object to be collected is a living body based on the living body recognition result; and when the image obtained by image collection of the object to be collected is an infrared image, multiple infrared images can be screened based on the brightness of the infrared image to obtain the image to be detected.

[0057] 7) Region of Interest (ROI), which refers to the largest image region with the lowest degree of image distortion corresponding to different acquisition distances in the embodiment of the present application.

[0058] It should be noted that when the viewing angle (FOV) is less than the viewing angle threshold, the distance between the object to be captured (e.g., face) and the capture device (e.g., 80 to 120 cm) is greater than the first distance threshold, and the image capture process is similar to pinhole imaging, so the distortion of the captured image is low (lower than the distortion threshold), which conforms to the straight-line propagation characteristic; at this time, the ROI of the image capture is the entire imaging range, so the ROI is the same for objects at different capture distances. Figure 1 , Figure 1 is a schematic diagram of pinhole imaging; Figure 1 As shown, light (eg, light 1-1) passes through a pinhole 1-2 and an image 1-5 corresponding to an object 1-4 (eg, the letter "F") is presented on a screen 1-3.

[0059] It should also be noted that when collecting images based on the principle of optical imaging, if the viewing angle is greater than the viewing angle threshold, for example, in close-range (less than the first distance threshold, such as 3 to 15 cm) image collection (for example, image collection of the palm in a palm-swiping scene), there is also a non-linear propagation characteristic, that is, the light is deflected after passing through the lens plane; the non-linear propagation characteristic will affect the distortion of the image. In addition, since the light passing through the lens plane at different angles has different corresponding deflection angles, the distortion is also different. For example, from the center of the lens plane to the surroundings, the deflection angle gradually increases, and the distortion also gradually increases. Therefore, the collected image often has distortion, which affects the image collection quality. See. Figure 2 , Figure 2 is an exemplary image acquisition schematic diagram provided in an embodiment of the present application; Figure 2 As shown, refraction and total reflection occur during image acquisition, wherein refraction and total reflection are divided by a critical ray 2-1; and, when ray 2-2 is refracted through the lens plane, different degrees of deflection occur; and, from the center of the lens plane to the surroundings, the deflection angle gradually increases, and the distortion also gradually increases; and because during image acquisition, the size of the imaging area of ​​the object to be acquired in the acquired image is negatively correlated with the acquisition distance of the object to be acquired, thus, under the same distortion (for example, 5% error ratio), the smaller the imaging area obtained when the acquisition distance is longer, the lower the quality; therefore, the ROI of the fixed acquisition device will affect the quality of image acquisition, the accuracy and efficiency of judgment.

[0060] Based on this, the embodiments of the present application provide an image processing method, apparatus, device, computer-readable storage medium and computer program product, which can improve the quality, judgment accuracy and efficiency of image acquisition. The following describes an exemplary application of the image processing device provided by the embodiments of the present application. The image processing device provided by the embodiments of the present application can be implemented as various types of terminals such as smart phones, smart watches, laptops, tablet computers, desktop computers, smart home appliances, set-top boxes, smart car devices, portable music players, personal digital assistants, dedicated messaging devices, intelligent voice interaction devices, portable gaming devices and smart speakers, and can also be implemented as a server. Below, an exemplary application of the image processing device when it is implemented as a terminal will be described.

[0061] See also Figure 3 , Figure 3 Schematic diagram of the architecture of the image processing system provided in the embodiment of the present application; Figure 3 As shown, to support an image processing application, in the image processing system 100, the terminal 400 (exemplarily showing the terminal 400-1 and the terminal 400-2, referred to as the image processing device) is connected to the server 200 via the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two. In addition, the image processing system 100 also includes a database 500 for providing data support to the server 200; and, Figure 3 What is shown in the figure is a situation where the database 500 is independent of the server 200. In addition, the database 500 can also be integrated in the server 200, which is not limited in the embodiment of the present application.

[0062] Terminal 400 is used to perform image acquisition on the object to be acquired, obtain the target acquisition distance and the image to be detected; determine the target image area corresponding to the target acquisition distance based on the correspondence between the acquisition distance and the image area, wherein the correspondence between the acquisition distance and the image area represents the maximum image area with the lowest degree of image distortion corresponding to different acquisition distances; detect the imaging area of ​​the object to be acquired in the image to be detected, and obtain the imaging area to be detected; when the imaging area to be detected is within the target image area, the imaging area to be detected is determined as the acquisition image of the object to be acquired (such as the "palmprint acquisition" and the acquired image including the palm imaging area displayed by terminal 400-2). It is also used to send the acquisition image to server 200 through network 300, and receive the prompt information of successful target processing sent by server 200 for the acquisition image, and display it on a graphical interface (such as the "transfer successful" and other information displayed on the graphical interface of terminal 400-1). It is also used to send the acquisition image to server 200 through network 300, and receive the prompt information of successful acquisition sent by server 200 for the acquisition image, and display the prompt information on a graphical interface.

[0063] The server 200 is used to receive the captured image sent by the terminal 400 through the network 300, store the corresponding relationship between the captured image and the current account, and send a prompt message of successful capture to the terminal 400 through the network 300. Alternatively, the server 200 is used to receive the captured image sent by the terminal 400 through the network 300, compare the captured image with a standard image corresponding to the object to be captured, wherein the standard image is an imaging area bound to the object to be captured, and the standard image is used for information authentication; when the comparison result indicates that the captured image is similar to the standard image, it is determined that the information authentication is successful; when the information authentication is successful, target processing is performed, wherein the target processing includes at least one of asset transfer, login, and basic information update, and a prompt message of successful target processing is sent to the terminal 400 through the network 300.

[0064] In some embodiments, the server 200 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDN, Content Delivery Network), and basic cloud computing services such as big data and artificial intelligence platforms. The terminal 400 may be a smart phone, a smart watch, a laptop, a tablet computer, a desktop computer, a smart TV, a set-top box, a smart car device, a portable music player, a personal digital assistant, a dedicated messaging device, a portable gaming device, and a smart speaker, but is not limited thereto. The terminal and the server may be directly or indirectly connected by wired or wireless communication, which is not limited in the embodiments of the present application.

[0065] See also Figure 4 , Figure 4 The embodiment of this application provides Figure 3 The schematic diagram of the terminal structure in FIG. Figure 4 The terminal 400 shown includes: at least one processor 410, a memory 450, at least one network interface 420 and a user interface 430. The various components in the terminal 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 440 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 440 is not shown in FIG. Figure 4 Various buses are labeled as bus system 440 .

[0066] Processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0067] The user interface 430 includes one or more output devices 431 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0068] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices that are physically remote from the processor 410.

[0069] The memory 450 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0070] In some embodiments, memory 450 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0071] Operating system 451, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0072] A network communication module 452, used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 include: Bluetooth, Wireless Fidelity (Wi-Fi), and Universal Serial Bus (USB), etc.;

[0073] a presentation module 453 for enabling presentation of information via one or more output devices 431 (e.g., display screen, speaker, etc.) associated with the user interface 430 (e.g., a user interface for operating peripherals and displaying content and information);

[0074] The input processing module 454 is used to detect one or more user inputs or interactions from one of the one or more input devices 432 and translate the detected inputs or interactions.

[0075] In some embodiments, the image processing device provided in the embodiments of the present application can be implemented in a software manner. Figure 4 An image processing device 455 stored in a memory 450 is shown, which may be software in the form of a program or a plug-in, including the following software modules: an image acquisition module 4551, a region determination module 4552, a target detection module 4553, an image judgment module 4554, a region calibration module 4555, and a model training module 4556. These modules are logical, and thus may be arbitrarily combined or further split according to the functions implemented. The functions of each module will be described below.

[0076] In some embodiments, the image processing device provided in the embodiments of the present application can be implemented in hardware. As an example, the image processing device provided in the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the image processing method provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), or other electronic components.

[0077] In some embodiments, the terminal or server can implement the image processing method provided in the embodiments of the present application by running a computer program. For example, the computer program can be a native program or software module in the operating system; it can be a native application (APP, Application), that is, a program that needs to be installed in the operating system to run, such as a payment APP or an instant messaging APP; it can also be a small program, that is, a program that can be run only by downloading it to a browser environment; it can also be a small program that can be embedded in any APP. In short, the above-mentioned computer program can be an application, module or plug-in in any form.

[0078] Below, the image processing method provided by the embodiment of the present application will be described in combination with the exemplary application and implementation of the image processing device provided by the embodiment of the present application. In addition, the image processing method provided by the embodiment of the present application is applied to various image acquisition scenarios such as cloud technology, artificial intelligence, smart transportation and vehicle-mounted.

[0079] See also Figure 5 , Figure 5 This is a schematic diagram of the image processing method provided in the embodiment of the present application. Figure 1 , will combine Figure 5 The steps shown are explained.

[0080] Step 501: perform image acquisition on the object to be acquired to obtain the target acquisition distance and the image to be detected.

[0081] In an embodiment of the present application, the image processing device includes an image acquisition device for acquiring images, or the image acquisition device is independent of the image processing device, and the image processing device can control the image acquisition device to acquire images; thus, when the image processing device receives an image acquisition operation triggered by a user or receives an image acquisition request sent by other devices, in response to the image acquisition operation or the image acquisition request, the image acquisition device performs image acquisition on the object to be acquired; at this time, the target acquisition distance and the image to be detected are obtained.

[0082] It should be noted that the object to be collected includes at least one of the palm part, the eye part, the lip part, the finger part and the graphic part; that is, the object to be collected can be a biological part used to determine biological characteristics (such as palm prints, eye prints, mouth shape and fingerprints); of course, the object to be collected can also be graphics (such as QR codes, etc.) and other information, which is not limited in the embodiments of the present application. The target collection distance represents the vertical distance between the object to be collected and the graphic collection device, which can be obtained by a distance sensor, and the distance sensor can obtain at least one distance. When at least one distance is one distance, the one distance is the target collection distance; when at least one distance is multiple distances, the target collection distance can be any one of the at least one distance, the minimum distance of the at least one distance, the maximum distance of the at least one distance, the average distance of the at least one distance, etc., which is not limited in the embodiments of the present application. The image to be detected is an image obtained by capturing the image of the object to be captured, including the imaging area of ​​the object to be captured; and the image to be detected can be screened out from at least one image captured by the image capture device, and the screening conditions include at least one of the presence or absence of the imaging area of ​​the object to be captured, the image clarity, the locality of the imaging area of ​​the object to be captured, and the inclination of the imaging area of ​​the object to be captured.

[0083] Step 502: Based on the correspondence between the acquisition distance and the image area, determine the target image area corresponding to the target acquisition distance.

[0084] In the embodiment of the present application, the image processing device is provided with a correspondence between the acquisition distance and the image area, or the image processing device can obtain the correspondence between the acquisition distance and the image area from other devices or received information; the correspondence between the acquisition distance and the image area is used to determine the image area corresponding to each acquisition distance, and one image area is the ROI of the image acquisition device corresponding to one acquisition distance. Thus, the image processing device can determine the target image area corresponding to the target acquisition distance based on the correspondence between the acquisition distance and the image area; here, the image processing device can directly select the target image area corresponding to the target acquisition distance from the correspondence between the acquisition distance and the image area; it can also select multiple image areas that best match the target acquisition distance from the correspondence between the acquisition distance and the image area, and then calculate the target image area corresponding to the target acquisition distance based on the selected multiple image areas; and so on, the embodiment of the present application does not limit this.

[0085] It should be noted that the image to be detected collected by the image processing device is distorted, wherein the distortion is positively correlated with at least one of the target acquisition distance, the viewing angle of the image acquisition device, and the angle at which the object to be acquired deviates from the center of the image acquisition device. Therefore, the embodiment of the present application pre-sets the maximum image area with the lowest degree of image distortion corresponding to different acquisition distances, that is, the corresponding relationship between the acquisition distance and the image area; wherein each image area represents the area of ​​the image within the distortion range that can be received at the acquisition distance corresponding to the image area. Here, in the corresponding relationship between the acquisition distance and the image area, the image area is negatively correlated with the acquisition distance.

[0086] Step 503: Detect the imaging area of ​​the object to be collected in the image to be detected to obtain the imaging area to be detected.

[0087] In an embodiment of the present application, the image processing device can directly compare the image to be detected with the target image area to determine whether the image acquisition of the object to be acquired is successful; it can also perform target detection on the image to be detected to obtain the imaging area to be detected, and then determine whether the image acquisition of the object to be acquired is successful based on the comparison between the imaging area to be detected and the target image area; and so on, the embodiment of the present application is not limited to this; wherein the two comparison processes are similar.

[0088] It should be noted that there is no particular order in which step 503 and step 502 are executed; the imaging area to be detected is the imaging area of ​​the object to be collected in the image to be detected; for example, when the object to be collected is the palm, the imaging area to be detected is the palm area.

[0089] Step 504: When the imaging area to be detected is located within the target image area, the imaging area to be detected is determined as the captured image of the object to be captured.

[0090] In an embodiment of the present application, the image processing device compares the imaging area to be detected with the target image area to determine whether the imaging area to be detected is located within the target image area; if it is determined that the imaging area to be detected is located within the target image area, it is determined that the image acquisition of the object to be acquired is successful, so that the image processing device determines the imaging area to be detected as the acquired image of the object to be acquired. If it is determined that the imaging area to be detected is outside the target image area, or a part of the imaging area to be detected is located within the target image area and a part of the imaging area to be detected is located outside the target image area, it is determined that the image acquisition of the object to be acquired has failed, so that the image processing device ends the image acquisition or continues the image acquisition when the position of the object to be acquired changes.

[0091] It should be noted that, when the intersection area between the imaging area to be detected and the target image area is the imaging area to be detected, the image processing device determines that the imaging area to be detected is located within the target image area.

[0092] It can be understood that by pre-setting the maximum image areas corresponding to different acquisition distances, it is possible to determine the maximum image area with the lowest image distortion corresponding to the acquisition distance of the object to be acquired during image acquisition, and then, when the imaging area to be detected in the acquired image to be detected is located within the target image area, the imaging area to be detected is determined as the acquisition image of the object to be acquired; in this way, the degree of image distortion of the acquired image is reduced, thereby improving the quality of image acquisition.

[0093] See also Figure 6 , Figure 6 This is a schematic diagram of the image processing method provided in the embodiment of the present application. Figure 2 ;like Figure 6 As shown, in the embodiment of the present application, steps 505 to 508 are also included before step 502; that is, before the image processing device determines the target image area corresponding to the target acquisition distance based on the correspondence between the acquisition distance and the image area, the image processing method also includes steps 505 to 508, and each step is explained below.

[0094] Step 505: perform image acquisition on the first object sample at different acquisition positions at different acquisition distances to obtain a first image sample set corresponding to different acquisition positions at the same acquisition distance.

[0095] In an embodiment of the present application, the image processing device first determines a plurality of different acquisition distances, and the plurality of different acquisition distances may be spaced apart by the same distance or by different distances, which is not limited in the embodiment of the present application; here, the image processing device performs image acquisition on first object samples located at different acquisition positions corresponding to each acquisition distance, so that for each acquisition distance, a set of first image samples corresponding to a plurality of different acquisition positions will be obtained; wherein each first image sample in the set of first image samples corresponds to an acquisition position at the acquisition distance.

[0096] It should be noted that when the interval distances between multiple different acquisition distances are different, the interval distances can be random, or can be determined by being divided into a first distance range, a second distance range (for example, the most commonly used distance range for image acquisition) and a third distance range according to the distance from the image acquisition device, wherein the distance within the first distance range is smaller than the second distance range, and the distance within the second distance range is smaller than the distance within the third distance range. At this time, the distance interval within the second distance range can be smaller than the distance intervals within the first distance range and the third distance range.

[0097] It should also be noted that, before step 505, the image processing device may also perform statistics on the object sizes corresponding to different first object samples to obtain the target object size; thus, step 505 includes: the image processing device performs image acquisition on the first object samples of the target object size at different acquisition positions at different acquisition distances to obtain a set of first image samples corresponding to different acquisition positions at the same acquisition distance.

[0098] Step 506: Obtain the distortion degree of each first image sample in the first image sample set, and determine a target image sample set whose distortion degree is lower than a distortion degree threshold from the first image sample set.

[0099] In an embodiment of the present application, the degree of distortion of each first image sample in the first image sample set obtained by the image processing device is determined based on at least one of the image clarity and the degree of deformation of the first image sample. In addition, a distortion degree threshold is set in the image processing device, or the image processing device can obtain the distortion degree threshold from other devices or can obtain the distortion degree threshold from received information, and the distortion degree threshold represents the maximum distortion degree corresponding to the subsequent processing of the image. Here, the image processing device compares the distortion degree of each first image sample with the distortion degree threshold, and obtains all first image samples whose distortion degree is lower than the distortion degree threshold from the first image sample set, thereby obtaining the target image sample set.

[0100] Step 507: determine the maximum area formed by the target image sample set as the image area corresponding to the acquisition distance.

[0101] In an embodiment of the present application, the image processing device obtains the maximum area formed by all the first image samples in the target image sample set, and thus obtains the image area corresponding to the acquisition distance; thus, for different acquisition distances, image areas corresponding to different acquisition distances are obtained.

[0102] Step 508: Determine the image regions corresponding to different acquisition distances as the corresponding relationship between the acquisition distance and the image region.

[0103] In the embodiment of the present application, the image areas corresponding to the different acquisition distances respectively obtained by the image processing device are the corresponding relationship between the acquisition distance and the image area.

[0104] It can be understood that by collecting the first object sample at different collection positions at different collection distances, and combining the distortion degree threshold to realize calibration of the image area at different collection distances, data support is provided for the judgment of image collection, thereby improving the efficiency and quality of image collection.

[0105] In the embodiment of the present application, step 502 may be implemented through step 5021 and step 5022 (not shown in the figure); that is, the image processing device determines the target image area corresponding to the target acquisition distance based on the correspondence between the acquisition distance and the image area, including step 5021 and step 5022, and each step is described below.

[0106] Step 5021: When the acquisition distance granularity is less than the granularity threshold, based on the target acquisition distance, determine the acquisition distance that is the smallest difference from the target acquisition distance and is less than the target acquisition distance from the corresponding relationship between the acquisition distance and the image area.

[0107] In the embodiment of the present application, when the interval distances between different acquisition distances are the same, the acquisition distance granularity is positively correlated with the interval distance; and a granularity threshold is set in the image processing device, or the image processing device can obtain the granularity threshold from other devices or received information, and the granularity threshold is used to measure the size of the acquisition distance granularity corresponding to different acquisition distances. Thus, the image processing device compares the acquisition distance granularity with the granularity threshold, and when it is determined that the acquisition distance granularity is less than the granularity threshold, it indicates that the acquisition distance granularity is small; at this time, the image processing device determines the acquisition distance that is the smallest difference from the target acquisition distance and is less than the target acquisition distance from the corresponding relationship between the acquisition distance and the image area.

[0108] Step 5022: In the correspondence between the acquisition distance and the image area, the image area corresponding to the acquisition distance that has the smallest difference from the target acquisition distance and is smaller than the target acquisition distance is determined as the target image area.

[0109] In an embodiment of the present application, when the image processing device determines that the acquisition distance granularity is less than the granularity threshold, it can also determine the acquisition distance with the smallest difference from the target acquisition distance from the correspondence between the acquisition distance and the image area, and in the correspondence between the acquisition distance and the image area, the image area corresponding to the smallest difference from the target acquisition distance is determined as the target image area.

[0110] It is understandable that when the acquisition distance granularity is small, the image processing device directly determines the target image area from the correspondence between the acquisition distance and the image area, which can improve the acquisition efficiency of the target image area while ensuring accuracy, thereby improving the efficiency of image acquisition.

[0111] In the embodiment of the present application, step 502 may be implemented through step 5023 and step 5027 (not shown in the figure); that is, the image processing device determines the target image area corresponding to the target acquisition distance based on the correspondence between the acquisition distance and the image area, including step 5023 and step 5027, and each step is described below.

[0112] Step 5023: When the acquisition distance granularity is greater than or equal to the granularity threshold, based on the target acquisition distance, from the correspondence between the acquisition distance and the image area, determine the two acquisition distances with the smallest difference from the target acquisition distance to obtain the first acquisition distance and the second acquisition distance.

[0113] In the embodiment of the present application, the image processing device compares the acquisition distance granularity with the granularity threshold. When it is determined that the acquisition distance granularity is greater than or equal to the granularity threshold, it indicates that the acquisition distance granularity is large; at this time, the image processing device can obtain the target image area by calculating the acquired multiple image areas that match the target acquisition distance. Therefore, the image processing device first determines the two acquisition distances with the smallest difference from the target acquisition distance from the corresponding relationship between the acquisition distance and the image area, and obtains the first acquisition distance and the second acquisition distance; it is easy to know that when the first acquisition distance is less than the second acquisition distance, the first acquisition distance to the second acquisition distance represents the minimum distance range determined in the corresponding relationship between the acquisition distance and the image area based on the target acquisition distance.

[0114] Step 5024: Determine, from the correspondence between the acquisition distance and the image area, a first image area corresponding to the first acquisition distance and a second image area corresponding to the second acquisition distance.

[0115] In an embodiment of the present application, the image processing device obtains the image area corresponding to the first acquisition distance from the correspondence between the acquisition distance and the image area, thus obtaining the first image area, and obtains the image area corresponding to the second acquisition distance, thus obtaining the second image area.

[0116] Step 5025: Obtain a first distance difference between the target acquisition distance and the first acquisition distance, and obtain a second distance difference between the target acquisition distance and the second acquisition distance.

[0117] In the embodiment of the present application, the image processing device obtains the difference between the target acquisition distance and the first acquisition distance, and thus obtains the first distance difference; the image processing device obtains the difference between the target acquisition distance and the second acquisition distance, and thus obtains the second distance difference. Here, the first distance difference and the second distance difference can both be scalars.

[0118] Step 5026: Obtain a first weight that is positively correlated with the first distance difference and a second weight that is positively correlated with the second distance difference.

[0119] In an embodiment of the present application, the image processing device determines the weight of the second image area based on the first distance difference, and determines the weight of the first image area based on the second distance difference; here, the first weight corresponding to the second image area determined by the image processing device is positively correlated with the first distance difference, and the second weight corresponding to the first image area determined is positively correlated with the second distance difference.

[0120] Step 5027: Determine the target image area based on the fusion result of the first weight and the second image area, and the fusion result of the second weight and the first image area.

[0121] In an embodiment of the present application, the image processing device fuses the first weight with the second image area, and fuses the second weight with the first image area, and then integrates the two fusion results into a target image area; that is, the image processing device uses the first weight and the second weight to perform weighted integration of the second image area and the first image area, and thus obtains the target image area.

[0122] It can be understood that the image processing device first determines the distance range corresponding to the target acquisition distance from the correspondence between the acquisition distance and the image area, and then determines the target image area corresponding to the target acquisition distance based on the image area range corresponding to the distance range, so that the target image area is calculated through context information, which can improve the accuracy of the target image area and thus improve the accuracy of image acquisition.

[0123] In the embodiment of the present application, step 503 can be implemented through steps 5031 to 5033 (not shown in the figure); that is, the image processing device detects the imaging area of ​​the object to be collected in the image to be detected, and obtains the imaging area to be detected, including steps 5031 to 5033, and each step is described below.

[0124] Step 5031: Divide the image to be detected into N grids, and determine M bounding boxes corresponding to each grid.

[0125] In the embodiment of the present application, the image processing device divides the image to be detected into grids, thus obtaining N (for example, 49) grids; then, the image processing device determines M (for example, 2) bounding boxes in the image to be detected for each grid, each bounding box being an imaging area of ​​the object to be collected estimated based on the grid. Wherein, N and M are both positive integers.

[0126] Step 5032: Determine the bounding box confidence by combining the features of the grid and the features of each bounding box.

[0127] It should be noted that the image processing device extracts features from the grid and each bounding box respectively, and combines the features of the grid with the features of each bounding box to determine the bounding box confidence of the bounding box. The bounding box confidence is determined based on at least one of the possibility that the bounding box includes the imaging area of ​​the object to be captured and the accuracy of the bounding box.

[0128] Step 5033: determine the bounding box corresponding to the maximum bounding box confidence as the imaging area to be detected of the object to be collected in the image to be detected.

[0129] In the embodiment of the present application, after the image processing device obtains the bounding box confidence corresponding to each bounding box, it also obtains all bounding box confidences corresponding to all bounding boxes; thus, the image processing device selects the maximum bounding box confidence from all bounding box confidences, and determines the bounding box corresponding to the maximum bounding box confidence as the imaging area to be detected of the object to be collected in the image to be detected.

[0130] In an embodiment of the present application, in step 503, the image processing device detects the imaging area of ​​the object to be collected in the image to be detected, and the process of obtaining the imaging area to be detected can be implemented by a detection model, and the detection model is used for target detection, that is, for detecting the imaging area of ​​the object to be collected. Here, the detection model is obtained by the following steps: the image processing device obtains a second image sample carrying an object annotation area, wherein the object annotation area refers to the imaging area of ​​the second object sample in the second image sample; the detection model to be trained is used to predict the imaging area of ​​the second object sample in the second image sample to obtain the object prediction area, wherein the detection model to be trained is a neural network model to be trained for predicting the imaging area of ​​the object; based on the difference between the object prediction area and the object annotation area, the detection model to be trained is trained to obtain the detection model.

[0131] It should be noted that when the image processing device is training the detection model to be trained, it performs back propagation in the detection model to be trained based on the difference between the object prediction area and the object annotation area to adjust the model parameters of the detection model to be trained until the training end condition is reached, the training is ended, and the currently trained detection model to be trained is determined as the detection model; thereafter, the image processing device can also optimize the detection model based on the acquired new training samples to improve the generalization ability of the detection model. Among them, the training end condition can be reaching the accuracy index threshold, or reaching the training number threshold, or reaching the training duration threshold, or a combination of the above, etc., which is not limited in the embodiments of the present application.

[0132] See also Figure 7 , Figure 7 This is a schematic diagram of the image processing method provided in the embodiment of the present application. Figure 3 ;like Figure 7 As shown, in the embodiment of the present application, step 503 also includes step 509 and step 510; that is, the image processing device detects the imaging area of ​​the object to be collected in the image to be detected, and after obtaining the imaging area to be detected, the image processing method also includes step 509 and step 510, and each step is explained below.

[0133] Step 509: When the intersection area between the imaging area to be detected and the target image area is smaller than the imaging area to be detected, or when the imaging area to be detected is outside the target image area, it is determined that image acquisition of the object to be acquired has failed.

[0134] It should be noted that if part of the imaging area to be detected is located within the target image area and part of the imaging area to be detected is located outside the target image area, that is, the intersection area between the imaging area to be detected and the target image area is smaller than the imaging area to be detected, at this time, the image processing device determines that the image acquisition of the object to be acquired has failed, and thus, the image acquisition device ends the image acquisition or continues the image acquisition when the position of the object to be acquired changes.

[0135] Step 510: When image acquisition of the object to be acquired fails, first acquisition prompt information is generated based on imaging deviation information of the imaging area to be detected deviating from the target image area.

[0136] In an embodiment of the present application, the position change of the object to be collected can be achieved through first collection prompt information, wherein the first collection prompt information is generated based on imaging deviation information of the imaging area to be detected deviating from the target image area when image collection of the object to be collected fails, and is used to indicate the prompt information that the imaging area of ​​the object to be collected moves to the target image area; for example, move 1 cm to the left, etc.

[0137] It can be understood that the first acquisition prompt information is generated by using the imaging deviation information of the imaging area to be detected deviating from the target image area, so that the position of the object to be acquired changes, which can improve the success rate of the second image acquisition.

[0138] In an embodiment of the present application, the image processing device can determine whether the imaging area to be detected is located within the target image area based on whether the intersection area of ​​the imaging area to be detected and the target image area is the imaging area to be detected, or it can determine whether the imaging area to be detected is located within the target image area based on the following method, including: obtaining a circular representation corresponding to the imaging area to be detected to obtain first circle information; obtaining a circular representation corresponding to the target image area to obtain second circle information; based on the first circle information and the second circle information, determining the center distance and the radius difference; when the center distance is less than or equal to the radius difference, determining that the imaging area to be detected is located within the target image area.

[0139] It should be noted that the image processing device may also determine that the imaging area to be detected is not located within the target image area when determining that the center distance is greater than the radius difference, thereby determining that the image acquisition has failed.

[0140] In an embodiment of the present application, in step 501, the image processing device performs image capture on the object to be captured to obtain a target capture distance and an image to be detected, including: the image processing device displays an authorization control, responds to an authorization request operation on the authorization control, performs image capture on the object to be captured, and obtains a target capture distance and an image to be detected.

[0141] It should be noted that the authorization control is used to trigger the image acquisition of the object to be acquired, such as the "palm swipe" button, the "palmprint acquisition" button, and the like.

[0142] In an embodiment of the present application, in step 504, when the imaging area to be detected is located within the target image area, after the image processing device determines the imaging area to be detected as a captured image of the object to be captured, the image processing method further includes: the image processing device compares the captured image with a standard image corresponding to the object to be captured, wherein the standard image is an imaging area bound to the object to be captured, and the standard image is used for information authentication; when the comparison result indicates that the captured image is similar to the standard image, it is determined that the information authentication is successful; when the information authentication is successful, target processing corresponding to the authorization request operation is performed, wherein the target processing includes at least one of asset transfer, login, and basic information update.

[0143] In an embodiment of the present application, before the image processing device determines the target image area corresponding to the target acquisition distance based on the correspondence between the acquisition distance and the image area in step 502, the image processing method also includes: when the target acquisition distance is within the acquisition distance range corresponding to different acquisition distances, the image processing device determines the target image area corresponding to the target acquisition distance from the correspondence between the acquisition distance and the image area; when the target acquisition distance is outside the acquisition distance range corresponding to different acquisition distances, the image processing device generates second acquisition prompt information.

[0144] It should be noted that after the image processing device obtains the target acquisition distance, it compares the target acquisition distance with the acquisition distance ranges corresponding to different acquisition distances. When the target acquisition distance is within the acquisition distance range, it indicates that the target acquisition distance is reasonable, and only then is the image to be detected judged; when the target acquisition distance is outside the acquisition distance range, it indicates that the target acquisition distance is unreasonable, and the target acquisition distance needs to be adjusted. Therefore, the image processing device generates a second acquisition prompt information based on the difference between the target acquisition distance and the acquisition distance range.

[0145] Next, an exemplary application of the embodiment of the present application in a practical application scenario will be described. The exemplary application describes the process of capturing an image of the palm and implementing asset transfer in a palm swiping scenario.

[0146] See also Figure 8 , Figure 8 is a schematic diagram of an exemplary application scenario provided by an embodiment of the present application; Figure 8 As shown, the terminal (referred to as the image processing device) controls the palm-swiping device 8-2 (referred to as the image acquisition device) to perform image acquisition in response to the palm-swiping request 8-1, so as to acquire an image of the palm part 8-3; then the acquired palm image (referred to as the image to be detected) is detected by the image processing method provided in the embodiment of the present application to determine whether the image acquisition is successful or the asset transfer is successful.

[0147] It should be noted that the palm-swiping request 8-1 can be generated in response to a user operation (called an authorization request operation) or a request received from another device; the palm-swiping device can be integrated into the terminal or independent of the terminal. In addition, successful image acquisition can refer to successful image acquisition during the asset transfer process or to successful image acquisition when the palm print information of a specified account is pre-stored; and successful asset transfer refers to the case where the acquired image is determined to be successful, and the acquired image is compared with the pre-stored image, and the asset transfer is performed when the palm print information matches according to the comparison result.

[0148] The following describes the process of determining whether the captured palm image is successful.

[0149] See also Fig. 9 , Fig. 9 is a schematic diagram of an exemplary process of judging a collected palm image provided by an embodiment of the present application; Fig. 9 As shown, the exemplary process of determining whether the captured palm image is successful includes steps 901 to 910, and each step is described below.

[0150] Step 901: Calibrate the region of interest of the palm-swiping device.

[0151] It should be noted that the terminal controls the palm scanning device to collect images of standard palms at different distances (called different collection distances) (for example, 1 cm intervals) and different positions.

[0152] Here, since the palm brush device imaging is centrally symmetrical, the region of interest corresponding to each calibrated distance can be represented by the center point coordinates (x, y) and the radius r. For example, when the calibration results obtained are the ROIs corresponding to the distances of 1, 2, ..., 19 and 20 cm, they are recorded as O1 to O2. 20 , where each O i ∈[O1,O 20 ] is the center point of the i-th ROI (x i ,y i ) and radius (r i )express.

[0153] Step 902: Collect palm images.

[0154] Step 903: Determine the current acquisition distance (referred to as the target acquisition distance).

[0155] It should be noted that during the image acquisition process, the terminal controls the palm-brush device to collect distance data, that is, the terminal controls the palm-brush device to collect distance data through a distance sensor (for example, the Psensor interface). At this time, the collected distance data includes multiple distances, which is a group of distances; the average value of this group of distances is calculated to obtain the current acquisition distance.

[0156] Step 904: Determine whether the current acquisition distance is within the distance range corresponding to the calibration result. If yes, execute step 905; if not, execute step 910.

[0157] For example, determine whether the current collection distance is within 1 to 20 centimeters.

[0158] Step 905: determine the calibration area range (called the first image area and the second image area) corresponding to the current acquisition distance from the calibration result (called the correspondence between the acquisition distance and the image area), and determine the area of ​​interest (called the target image area) of the current acquisition distance based on the calibration area range.

[0159] It should be noted that the terminal determines the minimum distance range [L, R] (called the first collection distance and the second collection distance) of the current collection distance from the calibration result, where L and R are two adjacent distances in the calibration result, and there is a distance interval between L and R; for example, L∈[1, 19], R=L+1; and when the current collection distance I is 3.5, L is 3 and R is 4. Then, the calibration area range [O L , O R Here, when O L is (x L ,y L , r L ), O R is (x R ,y R , r R ), the area of ​​interest O of the current acquisition distance I (x I ,y I , r I ) as shown in equations (1) to (3).

[0160]

[0161] in, is called the first weight, It is called the second weight.

[0162] Step 906: Detect the palm area of ​​the palm image.

[0163] Step 907: Determine the palm area based on the detection result.

[0164] See also Fig.10 , Fig.10 is an exemplary target detection schematic diagram provided in an embodiment of the present application; Fig.10 As shown, the palm image is first adjusted to a specified size (e.g., 448*448) to obtain image 10-1; then image 10-1 is input into a prediction model 10-2 (e.g., CNN model, GoogLeNet model); finally, the output result of the prediction model 10-2 is the palm area 10-3.

[0165] The processing procedure of the prediction model 10-2 is described below.

[0166] See also Fig.11 , Fig.11 is another exemplary target detection schematic diagram provided in an embodiment of the present application; Fig.11 As shown, the prediction model 10-2 divides the image 10-1 into S*S(N) grids to obtain image 11-1; M bounding boxes are predicted for each grid, as shown in image 11-2; wherein each bounding box includes a center position (u, v), a size (w, h) and a confidence level (p). In addition, each grid is also used to predict the probability that the grid belongs to C (1 in the palm payment scenario) specified categories, as shown in image 11-3; thus, the confidence level of each bounding box is a fusion of the probability of the category corresponding to the grid and the accuracy of the bounding box. Here, the bounding box 11-4 with the highest confidence level is the detection result, and the area within the bounding box 11-4 with the highest confidence level is the palm area 10-3; wherein the detection result is expressed as (u I , v I , w I ,h I ), thus, the corresponding circumscribed circle radius r′ can be obtained, as shown in formula (4).

[0167]

[0168] Thus, the circumscribed circle corresponding to the palm area 10-3 is expressed as ((u I +w I ) / 2,(v I +h I ) / 2, r′), when u′=(u I +w I ) / 2, v′=(v I +h I ) / 2, the circumscribed circle is simplified to (u′, v′, r′).

[0169] In addition, when there is no palm imaging area in the grid, the confidence is 0. If there is, the confidence is equal to the fusion value of the category probability and the intersection area, where the intersection area refers to the intersection of the bounding box and the actual palm border.

[0170] See also Fig.12 , Fig.12 is a schematic diagram of an exemplary palm area determination provided by an embodiment of the present application; Fig.12 As shown, in the palm area 10-3 determined in the image 10-1, the position point 12-2 is the center of the circumscribed circle of the detection result (u′, v′), and the position point 12-1 is (u I , v I ).

[0171] See also Fig.13 , Fig.13is another exemplary target detection schematic diagram provided in the embodiment of the present application; Fig.13 As shown, the prediction model 10-2 includes a feature extraction module 13-1 and a prediction module 13-2. Among them, the submodule 13-11 in the feature extraction module 13-1 includes a convolution layer (7*7*64--2) and a maximum pooling layer (2*2--2), which is used to process the input data 13-31 ((448*448*3(7*7)) is the image 10-1) to obtain the feature 13-32 ((112*112*192(3*3)); the submodule 13-12 in the feature extraction module 13-1 includes a convolution layer (3*3*192) and a maximum pooling layer (2*2--2), which is used to process the feature 13-32 to obtain the feature 13-33 ((56*56 *256(3*3)); the submodule 13-13 in the feature extraction module 13-1 includes a convolution layer (1*1*128, 3*3*256, 1*1*256, 3*3*512) and a maximum pooling layer (2*2--2), which is used to process the feature 13-33 to obtain the feature 13-34 ((28*28*512(3*3)); the submodule 13-14 in the feature extraction module 13-1 includes a convolution layer ((1*1*256, 3*3*512)*4, 1*1*512, 3*3*1024) and a maximum pooling layer (2*2--2), which is used Feature 13-34 is processed to obtain feature 13-35 ((14*14*1024(3*3)); submodule 13-15 in feature extraction module 13-1 includes convolution layer ((1*1*512, 3*3*1024)*2, 3*3*1024, 3*3*1024--2), which is used to process feature 13-35 and obtain feature 13-36 ((7*7*1024(3*3)); submodule 13-16 in feature extraction module 13-1 includes convolution layer ((3*3*1024)*2), which is used to process feature 13-36. Feature 13-37 (7*7*1024) is obtained. Submodule 13-21 in prediction module 13-2 includes a fully connected layer for processing feature 13-37 to obtain feature 13-38 (4096); submodule 13-22 in prediction module 13-2 includes a fully connected layer for processing feature 13-38 to obtain feature 13-39 (7*7*11, 7*7 is the number of grids N, 11=(2*5+1), 2 is the number of bounding boxes corresponding to each grid, and 1 is the number of target categories (palm category)). Here, feature 13-39 is used to determine the detection result.

[0172] Step 908 , determine whether the palm area is within the region of interest of the current acquisition distance; if yes, execute step 909 , if not, execute step 910 .

[0173] Refer to formula (5), which describes the palm area (u′, v′, r′) in the region of interest (x I ,y I , r I ) within the representation.

[0174]

[0175] Step 909: Determine whether the image acquisition is successful.

[0176] Step 910: Determine that image acquisition fails.

[0177] It can be understood that in the palm swiping scenario, by pre-calibrating the ROIs corresponding to multiple distances, the target ROI (called the target image area) corresponding to the current acquisition distance can be dynamically calculated based on the pre-calibrated results; and then, by determining whether the detected palm area is within the target ROI, the image acquisition result can be determined quickly and accurately.

[0178] The following is a description of an exemplary structure of the image processing device 455 provided in the embodiment of the present application implemented as a software module. In some embodiments, Figure 4 As shown, the software modules stored in the image processing device 455 of the memory 450 may include:

[0179] The image acquisition module 4551 is used to acquire images of the object to be acquired and obtain the target acquisition distance and the image to be detected;

[0180] The region determination module 4552 is used to determine the target image region corresponding to the target acquisition distance based on the correspondence between the acquisition distance and the image region, wherein the correspondence between the acquisition distance and the image region represents the maximum image region with the lowest degree of image distortion corresponding to different acquisition distances;

[0181] The target detection module 4553 is used to detect the imaging area of ​​the object to be collected in the image to be detected to obtain the imaging area to be detected;

[0182] The image determination module 4554 is used to determine the imaging area to be detected as the acquisition image of the object to be acquired when the imaging area to be detected is located within the target image area.

[0183] In an embodiment of the present application, the image processing device 455 also includes an area calibration module 4555, which is used to perform image acquisition on the first object sample at different acquisition positions at different acquisition distances to obtain a first image sample set corresponding to different acquisition positions at the same acquisition distance; obtain the degree of distortion of each first image sample in the first image sample set, and determine a target image sample set whose degree of distortion is lower than a distortion degree threshold from the first image sample set; determine the maximum area formed by the target image sample set as the image area corresponding to the acquisition distance; and determine the image areas corresponding to different acquisition distances as the correspondence between the acquisition distance and the image area.

[0184] In an embodiment of the present application, the area determination module 4552 is also used to determine, based on the target acquisition distance, from the correspondence between the acquisition distance and the image area, an acquisition distance that is the smallest difference from the target acquisition distance and is smaller than the target acquisition distance when the acquisition distance granularity is smaller than the granularity threshold, wherein the interval distance between different acquisitions is positively correlated with the acquisition distance granularity; in the correspondence between the acquisition distance and the image area, the image area corresponding to the acquisition distance that is the smallest difference from the target acquisition distance and is smaller than the target acquisition distance is determined as the target image area.

[0185] In an embodiment of the present application, the area determination module 4552 is also used to, when the acquisition distance granularity is greater than or equal to the granularity threshold, determine the two acquisition distances with the smallest difference from the target acquisition distance from the correspondence between the acquisition distances and the image areas, and obtain a first acquisition distance and a second acquisition distance; determine the first image area corresponding to the first acquisition distance and the second image area corresponding to the second acquisition distance from the correspondence between the acquisition distances and the image areas; obtain a first distance difference between the target acquisition distance and the first acquisition distance, and obtain a second distance difference between the target acquisition distance and the second acquisition distance; obtain a first weight that is positively correlated with the first distance difference, and a second weight that is positively correlated with the second distance difference; and determine the target image area based on the fusion result of the first weight and the second image area, and the fusion result of the second weight and the first image area.

[0186] In an embodiment of the present application, the target detection module 4553 is used to divide the image to be detected into N grids and determine M bounding boxes corresponding to each grid, wherein N and M are both positive integers; determine the bounding box confidence based on the characteristics of the grids and the characteristics of each of the bounding boxes, wherein the bounding box confidence is determined based on at least one of the possibility that the bounding box includes the imaging area of ​​the object to be collected and the accuracy of the bounding box; and determine the bounding box corresponding to the maximum bounding box confidence as the imaging area to be detected of the object to be collected in the image to be detected.

[0187] In an embodiment of the present application, the image processing device 455 also includes a model training module 4556, which is used to obtain a second image sample carrying an object annotation area, wherein the object annotation area refers to an imaging area of ​​a second object sample in the second image sample; using a detection model to be trained, predicting the imaging area of ​​the second object sample in the second image sample to obtain an object prediction area, wherein the detection model to be trained is a neural network model to be trained for predicting the imaging area of ​​an object; based on the difference between the object prediction area and the object annotation area, training the detection model to be trained to obtain the detection model.

[0188] In the embodiment of the present application, the image judgment module 4554 is also used to determine that the image acquisition of the object to be acquired has failed when the intersection area between the imaging area to be detected and the target image area is smaller than the imaging area to be detected, or when the imaging area to be detected is located outside the target image area; when the image acquisition of the object to be acquired has failed, based on the imaging deviation information of the imaging area to be detected deviating from the target image area, generate first acquisition prompt information, wherein the first acquisition prompt information is used to indicate prompt information that the imaging area of ​​the object to be acquired is moved to the target image area.

[0189] In an embodiment of the present application, the image judgment module 4554 is also used to obtain a circular representation corresponding to the imaging area to be detected to obtain first circle information; obtain a circular representation corresponding to the target image area to obtain second circle information; determine the center distance and radius difference based on the first circle information and the second circle information; when the center distance is less than or equal to the radius difference, determine that the imaging area to be detected is located within the target image area.

[0190] In the embodiment of the present application, the image judgment module 4554 is also used to determine that the imaging area to be detected is located within the target image area when the intersection area between the imaging area to be detected and the target image area is the imaging area to be detected.

[0191] In the embodiment of the present application, the image acquisition module 4551 is also used to display an authorization control, and in response to an authorization request operation on the authorization control, perform image acquisition on the object to be acquired to obtain the target acquisition distance and the image to be detected.

[0192] In an embodiment of the present application, the image judgment module 4554 is further used to compare the acquired image with a standard image corresponding to the object to be acquired, wherein the standard image is an imaging area bound to the object to be acquired, and the standard image is used for information authentication; when the comparison result indicates that the acquired image is similar to the standard image, it is determined that the information authentication is successful; when the information authentication is successful, the target processing corresponding to the authorization request operation is performed, wherein the target processing includes at least one of asset transfer, login and basic information update.

[0193] In an embodiment of the present application, the object to be collected includes at least one of a palm part, an eye part, a lip part, a finger part and a graphic.

[0194] In an embodiment of the present application, the area determination module 4552 is also used to determine the target image area corresponding to the target acquisition distance from the correspondence between the acquisition distance and the image area when the target acquisition distance is within the acquisition distance range corresponding to different acquisition distances; and to generate second acquisition prompt information when the target acquisition distance is outside the acquisition distance range corresponding to different acquisition distances.

[0195] The embodiment of the present application provides a computer program product or a computer program, which includes computer executable instructions, and the computer executable instructions are stored in a computer-readable storage medium. The processor of the electronic device (referred to as the image processing device) reads the computer executable instructions from the computer-readable storage medium, and the processor executes the computer executable instructions, so that the electronic device performs the above-mentioned image processing method of the embodiment of the present application.

[0196] The present application embodiment provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, the processor will be caused to execute the image processing method provided by the present application embodiment, for example, Figure 5 The image processing method is shown.

[0197] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EP ROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or it may be various devices including one or any combination of the above memories.

[0198] In some embodiments, computer executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0199] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file storing other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0200] As an example, computer-executable instructions may be deployed to be executed on one electronic device (in this case, the one electronic device is an image processing device), or on multiple electronic devices located at one location (in this case, the multiple electronic devices located at one location are image processing devices), or on multiple electronic devices distributed at multiple locations and interconnected by a communication network (in this case, the multiple electronic devices distributed at multiple locations and interconnected by a communication network are image processing devices).

[0201] It is understandable that in the embodiments of the present application, related data such as images to be detected are involved. When the embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0202] To summarize, the embodiment of the present application pre-sets the maximum image areas corresponding to different acquisition distances, so that when performing image acquisition, the maximum image area with the lowest image distortion corresponding to the acquisition distance of the object to be acquired can be determined, and then when the imaging area to be detected in the acquired image to be detected is located within the target image area, the imaging area to be detected is determined as the acquisition image of the object to be acquired; in this way, the degree of image distortion of the acquired image is reduced, and therefore the quality of image acquisition can be improved.

[0203] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Perform image acquisition on the object to be acquired to obtain a target acquisition distance and an image to be detected, wherein the target acquisition distance represents a vertical distance between the object to be acquired and the image acquisition device; When the acquisition distance granularity is less than the granularity threshold, based on the target acquisition distance, determine the acquisition distance that is the smallest difference from the target acquisition distance and is less than the target acquisition distance from the corresponding relationship between the acquisition distance and the image area; Determine the image area corresponding to the acquisition distance that has the smallest difference from the target acquisition distance and is smaller than the target acquisition distance as the target image area, wherein the interval distance between different acquisition distances is positively correlated with the acquisition distance granularity, and the corresponding relationship between the acquisition distance and the image area represents the maximum image area with the lowest degree of image distortion corresponding to different acquisition distances; Detecting an imaging area of ​​the object to be collected in the image to be detected to obtain an imaging area to be detected; When the imaging area to be detected is located within the target image area, the imaging area to be detected is determined as the captured image of the object to be captured.

2. The method according to claim 1, characterized in that When the acquisition distance granularity is less than the granularity threshold, based on the target acquisition distance, before determining the acquisition distance that is the smallest difference from the target acquisition distance and is less than the target acquisition distance from the corresponding relationship between the acquisition distance and the image area, the method further includes: Performing image acquisition on the first object sample at different acquisition positions at different acquisition distances to obtain a first image sample set corresponding to different acquisition positions at the same acquisition distance; Acquire the distortion degree of each first image sample in the first image sample set, and determine a target image sample set whose distortion degree is lower than a distortion degree threshold from the first image sample set; Determine the maximum area formed by the target image sample set as the image area corresponding to the acquisition distance; The image regions corresponding to the different acquisition distances are respectively determined as the corresponding relationship between the acquisition distance and the image region.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: When the acquisition distance granularity is greater than or equal to the granularity threshold, determining two acquisition distances with the smallest difference from the target acquisition distance from the corresponding relationship between the acquisition distance and the image area, and obtaining a first acquisition distance and a second acquisition distance; Determine, from the correspondence between the acquisition distances and the image regions, a first image region corresponding to the first acquisition distance and a second image region corresponding to the second acquisition distance; Acquire a first distance difference between the target acquisition distance and the first acquisition distance, and acquire a second distance difference between the target acquisition distance and the second acquisition distance; Acquire a first weight that is positively correlated with the first distance difference, and a second weight that is positively correlated with the second distance difference; The target image region is determined based on a fusion result of the first weight and the second image region and a fusion result of the second weight and the first image region.

4. The method according to claim 1 or 2, characterized in that: The detecting the imaging area of ​​the object to be collected in the image to be detected to obtain the imaging area to be detected includes: Divide the image to be detected into N grids, and determine M bounding boxes corresponding to each grid, where N and M are both positive integers; Determining a bounding box confidence in combination with the features of the grid and the features of each of the bounding boxes, wherein the bounding box confidence is determined based on at least one of a possibility that the bounding box includes the imaging region of the object to be acquired and an accuracy of the bounding box; The bounding box corresponding to the maximum bounding box confidence is determined as the imaging area to be detected of the object to be collected in the image to be detected.

5. The method according to claim 4, characterized in that The detecting of the imaging area of ​​the object to be collected in the image to be detected to obtain the imaging area to be detected is achieved by a detection model, and the detection model is obtained by the following steps: Acquire a second image sample carrying an object annotation region, wherein the object annotation region refers to an imaging region of a second object sample in the second image sample; Using the detection model to be trained, predicting the imaging area of ​​the second object sample in the second image sample to obtain an object prediction area, wherein the detection model to be trained is a neural network model to be trained for predicting the imaging area of ​​the object; Based on the difference between the object prediction area and the object annotation area, the to-be-trained detection model is trained to obtain the detection model.

6. The method according to claim 1 or 2, characterized in that: After detecting the imaging area of ​​the object to be collected in the image to be detected to obtain the imaging area to be detected, the method further includes: When the intersection area between the imaging area to be detected and the target image area is smaller than the imaging area to be detected, or when the imaging area to be detected is outside the target image area, it is determined that the image acquisition of the object to be acquired has failed; When image acquisition of the object to be acquired fails, first acquisition prompt information is generated based on imaging deviation information of the imaging area to be detected deviating from the target image area, wherein the first acquisition prompt information is prompt information for indicating that the imaging area of ​​the object to be acquired moves to the target image area.

7. The method according to claim 1 or 2, characterized in that: After detecting the imaging area of ​​the object to be collected in the image to be detected to obtain the imaging area to be detected, and before determining the imaging area to be detected as the collection image of the object to be collected when the imaging area to be detected is located within the target image area, the method further includes: Acquire a circular representation corresponding to the imaging area to be detected to obtain first circle information; Obtaining a circular representation corresponding to the target image area to obtain second circle information; Determine a center distance and a radius difference based on the first circle information and the second circle information; When the circle center distance is less than or equal to the radius difference, it is determined that the imaging area to be detected is located within the target image area.

8. The method according to claim 1 or 2, characterized in that: After detecting the imaging area of ​​the object to be collected in the image to be detected to obtain the imaging area to be detected, and before determining the imaging area to be detected as the collection image of the object to be collected when the imaging area to be detected is located within the target image area, the method further includes: When the intersection area between the imaging area to be detected and the target image area is the imaging area to be detected, it is determined that the imaging area to be detected is located within the target image area.

9. The method according to claim 1 or 2, characterized in that: The method of performing image acquisition on the object to be acquired to obtain the target acquisition distance and the image to be detected includes: Displaying an authorization control, and in response to an authorization request operation on the authorization control, performing image acquisition on the object to be acquired to obtain the target acquisition distance and the image to be detected; When the imaging area to be detected is located within the target image area, after determining the imaging area to be detected as the captured image of the object to be captured, the method further includes: Comparing the collected image with a standard image corresponding to the object to be collected, wherein the standard image is an imaging area bound to the object to be collected, and the standard image is used for information authentication; When the comparison result indicates that the collected image is similar to the standard image, it is determined that the information authentication is successful; When the information authentication succeeds, a target process corresponding to the authorization request operation is executed, wherein the target process includes at least one of asset transfer, login, and basic information update.

10. The method according to claim 1 or 2, characterized in that: The object to be collected includes at least one of a palm part, an eye part, a lip part, a finger part and a figure.

11. The method according to claim 1 or 2, characterized in that: When the acquisition distance granularity is less than the granularity threshold, based on the target acquisition distance, before determining the acquisition distance that is the smallest difference from the target acquisition distance and is less than the target acquisition distance from the corresponding relationship between the acquisition distance and the image area, the method further includes: When the target acquisition distance is within the acquisition distance range corresponding to different acquisition distances, determining the target image area corresponding to the target acquisition distance from the corresponding relationship between the acquisition distance and the image area; When the target collection distance is outside the collection distance range corresponding to the different collection distances, second collection prompt information is generated.

12. An image processing device, characterized in that: The image processing device comprises: A memory for storing computer executable instructions; A processor, configured to implement the image processing method according to any one of claims 1 to 11 when executing the computer executable instructions stored in the memory.

13. A computer-readable storage medium storing computer-executable instructions, characterized in that: The computer executable instructions are used to implement the image processing method according to any one of claims 1 to 11 when executed by a processor.

14. A computer program product comprising a computer program or computer executable instructions, characterized in that When the computer program or computer executable instruction is executed by a processor, the image processing method according to any one of claims 1 to 11 is implemented.

15. An image processing device, characterized in that: The image processing device comprises: An image acquisition module is used to acquire an image of the object to be acquired, and obtain a target acquisition distance and an image to be detected, wherein the target acquisition distance represents a vertical distance between the object to be acquired and the image acquisition device; A region determination module, configured to determine, based on the target acquisition distance and from the corresponding relationship between the acquisition distance and the image region, a collection distance that is the smallest in difference from the target acquisition distance and is smaller than the target acquisition distance when the acquisition distance granularity is smaller than the granularity threshold; Determine the image area corresponding to the acquisition distance that has the smallest difference from the target acquisition distance and is smaller than the target acquisition distance as the target image area, wherein the interval distance between different acquisition distances is positively correlated with the acquisition distance granularity, and the corresponding relationship between the acquisition distance and the image area represents the maximum image area with the lowest degree of image distortion corresponding to different acquisition distances; A target detection module, used to detect the imaging area of ​​the object to be collected in the image to be detected, and obtain the imaging area to be detected; The image judgment module is used to determine the imaging area to be detected as the acquisition image of the object to be acquired when the imaging area to be detected is located within the target image area.

Images