Financial user identity recognition method and device, electronic equipment and storage medium

Abstract

The invention discloses a financial user identity recognition method and device, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence and biological recognition or other related fields, and the method comprises the steps that a first image and a second image of a target financial user are obtained, the first image is a user whole body image collected by a target teller machine, and the second image is a user whole body image collected by a target teller machine; the second image is a certificate image uploaded by the target financial user through a target teller machine; positioning a face region of the target financial user in the first image by adopting a target detection algorithm, and extracting a face feature vector in the face region; performing text recognition processing on the second image to obtain an identity information text of the target financial user; and performing associated identity recognition based on the facial feature vector and the identity information text to obtain a recognition result, and returning the recognition result to the target teller machine for financial service processing. Through the method and the device, the technical problem of low user identity recognition accuracy of a financial institution in related technologies is solved.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and biometric technology or other related fields, and in particular to a method and device for identifying the identity of a financial user, an electronic device, and a storage medium. Background Art

[0002] Customer identity verification in current financial business processing directly affects the security and efficiency of financial transactions. Existing identity recognition technologies have the following flaws:

[0003] 1. Slow recognition speed: Traditional identity recognition methods rely primarily on manual review and input or single image processing technology. These methods are slow to achieve the real-time response required by counters or self-service devices. This leads to long customer wait times, especially during peak business hours, which reduces the service quality and customer satisfaction of financial institutions.

[0004] 2. Poor recognition accuracy. Existing technologies, whether image-based or text-based, can suffer from inaccurate results due to factors such as ambient lighting, camera quality, and document wear. For example, OCR technology significantly reduces recognition rates when processing documents with blurred handwriting or complex layouts, increasing the likelihood of errors.

[0005] 3. Security issues. Traditional customer information identification methods are vulnerable to identity forgery or data tampering. Financial institutions find it difficult to verify the authenticity of customer information in a short period of time, making it impossible to effectively prevent financial fraud.

[0006] 4. Information fusion is difficult. In existing technologies, facial feature recognition and ID information extraction are two independent processes. There is a lack of an effective information fusion mechanism, which makes it difficult to ensure the consistency of recognition results, affecting the efficiency and reliability of the entire identity verification process.

[0007] In summary, existing financial user identity recognition technologies are unable to balance recognition accuracy and security while ensuring recognition speed, especially when processing large amounts of customer data and facing identity authentication challenges in complex environments. There is still a problem of low accuracy.

[0008] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0009] The main purpose of this application is to provide a method and device for identifying a financial user, an electronic device, and a storage medium, so as to at least solve the technical problem of low accuracy in identifying user identities by financial institutions in related technologies.

[0010] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a method for identifying an identity of a financial user is provided, the method comprising: obtaining a first image and a second image of a target financial user, wherein the first image is a full-body image of the user collected by a target ATM, and the second image is an ID image uploaded by the target financial user through the target ATM; using a target detection algorithm to locate the facial area of ​​the target financial user in the first image, and extracting a facial feature vector in the facial area; performing text recognition processing on the second image to obtain the identity information text of the target financial user; performing associated identity recognition based on the facial feature vector and the identity information text to obtain an identification result, and returning the identification result to the target ATM for financial business processing.

[0011] Furthermore, after obtaining the first image and the second image of the target financial user, the method also includes: denoising the first image based on a Gaussian blur algorithm, and enhancing the image contrast using a histogram equalization algorithm to obtain an optimized first image; performing grayscale conversion on the second image, and enhancing the edge features of the document through an edge detection algorithm, and then correcting image deformation using perspective transformation to obtain an optimized second image.

[0012] Furthermore, the step of using a target detection algorithm to locate the facial area of ​​the target financial user in the first image includes: inputting the first image into a target detection model and outputting a detection result, wherein the target detection model is a pre-trained convolutional neural network model based on the target detection algorithm, and the detection result records N bounding boxes and the confidence corresponding to each bounding box, where N is a positive integer; screening M target bounding boxes corresponding to M confidences greater than a first preset threshold, and determining M candidate facial regions based on the M bounding boxes, where M is a positive integer less than or equal to N; performing overlapping detection box filtering processing on the candidate facial regions to obtain R facial regions, and determining the bounding box coordinates of the R facial regions, where R is a positive integer less than or equal to M.

[0013] Furthermore, the training step of the target detection model includes: obtaining a first training data set containing T financial user image samples, preprocessing all financial user image samples in the first training data set, wherein the preprocessing includes: Gaussian blur denoising, histogram equalization image enhancement and facial area labeling, and T is a positive integer; using a multi-task loss function and the preprocessed first training data set to train the initial target detection model, wherein the multi-task loss function includes a confidence loss function, a classification loss function and a bounding box regression loss function; obtaining a model evaluation index during the training process, and terminating the training when the model evaluation index does not improve for K consecutive preset cycles, to obtain the target detection model, wherein K is a preset positive integer.

[0014] Furthermore, the step of extracting facial feature vectors from the facial areas includes: for each facial area, performing facial key point detection within the facial area to obtain a detection result, wherein the preset facial key points include at least the facial features; aligning the facial areas based on the facial key points recorded in the detection result to obtain a facial image of the target financial user; inputting the facial image into a feature extraction model to output the facial feature vector, wherein the feature extraction model is a feature encoder obtained by pre-training a deep convolutional neural network, and is used to map the input facial image into a fixed-dimensional vector.

[0015] Furthermore, the step of performing associated identity recognition based on the facial feature vector and the identity information text to obtain an identification result includes: querying registered financial users in a pre-stored database and obtaining the registration feature vector and registration identity information corresponding to the registered financial user; calculating the identity credibility score of the target financial user based on the facial feature vector, the identity information text, the registration feature vector and the registration identity information; setting the identification result as credible when the identity credibility score is greater than or equal to a second preset threshold; or setting the identification result as uncredible when the identity credibility score is less than the second preset threshold.

[0016] Furthermore, the step of calculating the identity credibility score of the target financial user based on the facial feature vector, the identity information text, the registration feature vector and the registered identity information includes: comparing the similarity between the facial feature vector and the registration feature vector and calculating a first score; verifying the consistency between the identity information text and the registered identity information and calculating a second score; and performing a weighted calculation on the first score and the second score to obtain the identity credibility score.

[0017] In order to achieve the above-mentioned purpose, according to another aspect of the present application, a financial user identity recognition device is also provided, which includes: an acquisition unit, used to acquire a first image and a second image of a target financial user, wherein the first image is a full-body image of the user collected by a target ATM, and the second image is an ID card image uploaded by the target financial user through the target ATM; an extraction unit, used to locate the facial area of ​​the target financial user in the first image using a target detection algorithm, and extract the facial feature vector in the facial area; a first recognition unit, used to perform text recognition processing on the second image to obtain the identity information text of the target financial user; a second recognition unit, used to perform associated identity recognition based on the facial feature vector and the identity information text to obtain a recognition result, and return the recognition result to the target ATM for financial business processing.

[0018] Furthermore, the financial user identity recognition device also includes: a preprocessing unit, which includes: a first optimization module, which is used to, after obtaining the first image and the second image of the target financial user, denoise the first image based on a Gaussian blur algorithm, and enhance the image contrast using a histogram equalization algorithm to obtain an optimized first image; a second optimization module, which is used to perform grayscale conversion on the second image, enhance the edge features of the document through an edge detection algorithm, and then correct the image deformation using perspective transformation to obtain an optimized second image.

[0019] Furthermore, the extraction unit includes: a first input module, used to input the first image into a target detection model and output a detection result, wherein the target detection model is a pre-trained convolutional neural network model based on a target detection algorithm, and the detection result records N bounding boxes and the confidence corresponding to each bounding box, where N is a positive integer; a screening module, used to screen M target bounding boxes corresponding to M confidences greater than a first preset threshold, and determine M candidate facial regions based on the M bounding boxes, where M is a positive integer less than or equal to N; and a filtering module, used to perform overlapping detection frame filtering processing on the candidate facial regions to obtain R facial regions, and determine the bounding box coordinates of the R facial regions, where R is a positive integer less than or equal to M.

[0020] Furthermore, the financial user identity recognition device also includes: a training unit, used to execute the training steps of the target detection model, the training unit including: an acquisition module, used to obtain a first training data set containing T financial user image samples, and preprocess all financial user image samples in the first training data set, wherein the preprocessing includes: Gaussian blur denoising, histogram equalization image enhancement and facial area labeling, and T is a positive integer; a training module, used to train the initial target detection model using a multi-task loss function and the preprocessed first training data set, wherein the multi-task loss function includes a confidence loss function, a classification loss function and a bounding box regression loss function; a termination module, used to obtain a model evaluation index during the training process, and terminate the training when the model evaluation index does not improve for K consecutive preset cycles, so as to obtain the target detection model, wherein K is a preset positive integer.

[0021] Furthermore, the extraction unit also includes: a detection module, which is used to perform facial key point detection in each facial area to obtain a detection result, wherein the preset facial key points include at least five facial features; an alignment module, which is used to align the facial areas based on the facial key points recorded in the detection result to obtain the facial image of the target financial user; a second input module, which is used to input the facial image into a feature extraction model and output the facial feature vector, wherein the feature extraction model is a feature encoder obtained by pre-training based on a deep convolutional neural network, which is used to map the input facial image into a fixed-dimensional vector.

[0022] Furthermore, the second identification unit includes: a query module, used to query registered financial users in a pre-stored database and obtain the registration feature vector and registration identity information corresponding to the registered financial user; a calculation module, used to calculate the identity credibility score of the target financial user based on the facial feature vector, the identity information text, the registration feature vector and the registration identity information; a first setting module, used to set the recognition result as credible when the identity credibility score is greater than or equal to a second preset threshold; and a second setting module, used to set the recognition result as untrustworthy when the identity credibility score is less than the second preset threshold.

[0023] Furthermore, the calculation module includes: a first calculation submodule, used to compare the similarity between the facial feature vector and the registered feature vector, and calculate a first score; a second calculation submodule, used to verify the consistency between the identity information text and the registered identity information, and calculate a second score; a third calculation submodule, used to perform weighted calculation on the first score and the second score to obtain the identity credibility score.

[0024] In order to achieve the above-mentioned purpose, according to another aspect of the present application, a computer-readable storage medium is also provided, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned financial user identity identification methods.

[0025] In order to achieve the above-mentioned purpose, according to another aspect of the present application, an electronic device is also provided, including one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the identity identification method of the financial user described in any one of the above-mentioned items.

[0026] In order to achieve the above-mentioned purpose, according to another aspect of the present application, a computer program product is further provided, comprising computer instructions, wherein when the computer instructions are executed by a processor, the steps of any one of the above-mentioned methods for identifying the identity of a financial user are implemented.

[0027] The present invention proposes a method for identifying the identity of a financial user. The method first obtains a first image and a second image of a target financial user, wherein the first image is a full-body image of the user collected by a target ATM, and the second image is an ID image uploaded by the target financial user by the target ATM. Then, a target detection algorithm is used to locate the facial area of ​​the target financial user in the first image, and a facial feature vector in the facial area is extracted. Then, text recognition processing is performed on the second image to obtain the identity information text of the target financial user. Finally, identity association is performed based on the facial feature vector and the identity information text to obtain an identification result, and the identification result is returned to the target ATM for financial business processing.

[0028] In the present invention, deep learning is combined with computer vision. By integrating the user's full-body image collected in real time by the target ATM and the ID image uploaded by the user, the purpose of accurately locating and extracting the user's facial features and efficiently identifying the ID information is achieved, thereby realizing a highly automated, real-time, safe and reliable financial user identity authentication technical effect. Specifically, the present invention uses a target detection algorithm to accurately locate the facial area in the user's full-body image, and then extracts the facial feature vector through a deep neural network. At the same time, optical character recognition (OCR) processing is performed on the ID image uploaded by the user to accurately and quickly obtain the identity information text; by combining the facial feature vector with the identity information text for deep correlation analysis and comparison, multi-dimensional verification of the user's identity is achieved, which significantly improves the accuracy and security of recognition, thereby solving the technical problem of low accuracy of user identity recognition by financial institutions in related technologies.

[0029] In addition, the present invention also feeds back the recognition results to the target ATM in real time, which not only accelerates the processing flow of financial services, but also greatly improves the user's service experience. It provides strong technical support for financial institutions to build a safe, efficient and intelligent user identity recognition system, effectively avoiding the subjective errors and inefficiency of manual review, and overcoming the limitations of a single recognition method in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0031] Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing a method for identifying a financial user is shown;

[0032] Figure 2 is a flow chart of an optional method for identifying a financial user according to an embodiment of the present invention;

[0033] Figure 3 is an architectural diagram of an optional customer information identification system according to an embodiment of the present invention;

[0034] Figure 4 is a flow chart of an optional customer information identification method according to an embodiment of the present invention;

[0035] Figure 5 is a schematic diagram of an optional financial user identity recognition device according to an embodiment of the present invention;

[0036] Figure 6 The present invention is a block diagram of an electronic device for executing a method for identifying a financial user according to an embodiment of the present invention. DETAILED DESCRIPTION

[0037] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0038] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0039] To facilitate those skilled in the art to understand the present invention, some of the terms or nouns involved in the embodiments of the present invention are explained below:

[0040] Deep learning (DL) is a branch of machine learning that simulates how the human brain processes information by building multi-layer neural network models to automatically abstract and extract features from complex data. In this paper, deep learning is used to train the YOLO object detection algorithm to accurately identify and extract facial features in target images.

[0041] YOLO (You Only Look Once) is a real-time object detection algorithm that treats object detection as a regression problem and generates bounding boxes and class probabilities directly from the input image. It is fast, accurate, and performs well in real time. In this paper, the YOLO algorithm is used to quickly locate facial regions in full-body images of users, providing a foundation for subsequent facial feature extraction.

[0042] Optical Character Recognition (OCR) is a method for converting text in an image into an editable and searchable text format. It is widely used in automatic text recognition and data entry. In the context of this invention, OCR is used to recognize text information in document images, such as names and ID numbers, to automatically extract identity information.

[0043] Non-Maximum Suppression (NMS) is a technique used to reduce overlapping bounding boxes in the output of object detection algorithms, ensuring that each object is detected only once. In this paper, NMS is used to filter out the bounding boxes most likely to contain a face from the multiple bounding boxes output by YOLO, improving the accuracy of face detection.

[0044] It should be noted that the financial user identity identification method and device in this application can be used in the field of artificial intelligence and biometric technology to quickly and accurately identify financial users, and can also be used in any field outside the field of artificial intelligence and biometric technology to quickly and accurately identify financial users. This application does not limit the application field of the financial user identity identification method and device.

[0045] It should be noted that the relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, processing, transmission, provision, disclosure, use and processing of the relevant data comply with the laws, regulations and standards of the relevant regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse. For example, an interface is set up between this system and the relevant users or institutions. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or institution through the interface, and obtain relevant information after receiving the consent information fed back by the aforementioned user or institution.

[0046] The information collection (for example, user voice, video, and text collection) and analysis operations involved in this application have provided users with corresponding operation entrances when they are executed, allowing users to choose to agree or reject the automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered.

[0047] The following embodiments of the present invention can be applied to various systems, applications, and devices requiring software security vulnerability detection and source code testing, enabling efficient and secure identity recognition solutions based on deep learning. This invention uses an object detection algorithm to precisely locate the facial region within a user's full-body image and then extracts facial feature vectors from that region, effectively capturing the user's unique facial details. Furthermore, optical character recognition technology is used to analyze uploaded ID images, enabling automatic and accurate reading of text information.

[0048] The present invention also accurately verifies user identity through real-time monitoring and intelligent comparison mechanisms, and automatically processes the collected images. The testing process is efficient and fast, and identity confirmation is rapid and accurate, thus achieving fast, accurate and secure identification of financial user identities.

[0049] The present invention will be described in detail below with reference to various embodiments.

[0050] Example 1

[0051] According to an embodiment of the present invention, an embodiment of a method for identifying an identity of a financial user is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0052] The financial user identity identification method embodiment provided in the first embodiment of the present invention can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 The hardware structure block diagram of a computer terminal (or mobile device) for implementing a method for identifying a financial user is shown. Figure 1 As shown, the computer terminal 10 (or mobile device) may include one or more (illustrated as 102a, 102b, ..., 102n in the figure) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0053] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuitry may be a single independent processing module, or may be incorporated in whole or in part into any of the other components of the computer terminal 10 (or mobile device). As described in the embodiments of the present application, the data processing circuitry serves as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0054] Memory 104 can be used to store software programs and modules for application software, such as the program instructions / data storage device corresponding to the financial user identification method in the embodiments of the present application. Processor 102 executes the software programs and modules stored in memory 104 to perform various functional applications and data processing, thereby implementing the aforementioned financial user identification method. Memory 104 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, memory 104 may further include memory remotely located from processor 102, which can be connected to computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0055] The transmission device 106 is configured to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by the communications provider of the computer terminal 10. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is configured to communicate with the Internet wirelessly.

[0056] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0057] Under the above operating environment, the present invention provides Figure 2 The financial user identity recognition method shown in the figure is implemented by an intelligent identity authentication system, which combines deep learning and computer vision technology and is used in user identity rapid verification scenarios of financial institutions. In particular, it solves the problems of low accuracy, slow processing speed and poor user experience in financial user identity recognition in complex environments. By integrating the user's full-body image collected in real time by the target ATM and the ID image uploaded by the user, the specific implementation steps include: image acquisition, image preprocessing, facial area positioning and feature extraction based on the YOLO algorithm, text information OCR recognition processing of the ID image, information fusion comparison and verification, and result display and feedback, so as to achieve the purpose of automated, high-precision and real-time user identity authentication.

[0058] The embodiment of the present invention is described in detail below with reference to various specific steps.

[0059] Figure 2is a flow chart of an optional method for identifying a financial user according to an embodiment of the present invention. Figure 2 As shown, the method includes the following steps:

[0060] Step S201 : obtaining a first image and a second image of a target financial user, wherein the first image is a full-body image of the user captured by a target ATM, and the second image is an ID image uploaded by the target financial user via the target ATM.

[0061] Specifically, target financial users refer to individuals who apply for services or conduct transactions at financial institutions, including but not limited to depositors, credit card applicants, loan customers, etc. Users need to pass identity verification when interacting with financial institutions to ensure the security and legitimacy of transactions.

[0062] The first image refers to a user image captured in real time by a high-resolution camera installed on the target ATM, which includes the user's entire body so that the system can fully capture the user's appearance features, especially the facial area.

[0063] In an optional embodiment, the acquisition process needs to ensure image clarity, with an image resolution of 1920x1080 and an acquisition frequency of 30 frames per second, so as to capture the user's dynamic information. The image is used for subsequent facial detection and feature extraction and is one of the key data sources for achieving target financial user identity identification.

[0064] The second image refers to an ID image, such as an ID card, passport, or driver's license, that the target financial user proactively uploads to the target ATM. The user uses the ATM's user interface to upload the front, back, or key pages of the ID for OCR (Optical Character Recognition) processing. The clarity and completeness of the ID image affect the recognition rate of OCR technology. Embodiments of the present invention utilize OCR technology to extract textual information from ID images, including but not limited to name, ID number, and date of birth, and combine this information with the user's facial features for comprehensive identity verification.

[0065] The present invention can realize automatic, high-precision and real-time verification of the target financial user's identity by integrating the first image and the second image, significantly improving the service efficiency and user experience of financial institutions while ensuring the security and legitimacy of transactions.

[0066] Optionally, in the identity recognition method for financial users provided in an embodiment of the present invention, after obtaining the first image and the second image of the target financial user, the method further includes: denoising the first image based on a Gaussian blur algorithm, and enhancing the image contrast using a histogram equalization algorithm to obtain an optimized first image; performing grayscale conversion on the second image, and strengthening the edge features of the document through an edge detection algorithm, and then correcting the image deformation using perspective transformation to obtain an optimized second image.

[0067] In an embodiment of the present invention, a Gaussian blur algorithm can be used to denoise the first image to improve the recognition accuracy of the target financial user's facial features. It should be noted that the Gaussian blur algorithm is a low-pass filter that effectively reduces random noise in an image while preserving the image's key features. Applying Gaussian blur can smooth the image and suppress high-frequency noise generated by lighting variations or camera sensors, thereby improving the accuracy of subsequent facial detection and feature extraction.

[0068] It should also be noted that histogram equalization is an image processing technique that can adjust the brightness distribution of an image and enhance its contrast and details. In an embodiment of the present invention, histogram equalization is further applied to the optimized first image to highlight the details of the facial area, making it clearer and more discernible. Histogram equalization can automatically stretch the dynamic range of the image, ensuring that facial features are fully displayed even in low-light or overly bright conditions. The contrast-enhanced image helps improve the accuracy and stability of the target detection algorithm, especially in situations where facial features change slightly or when wearing glasses.

[0069] Furthermore, grayscale conversion is the process of converting a color image into a grayscale image. In the embodiments of the present invention, grayscale conversion of the captured second image (i.e., the document image) is performed to simplify the image data and reduce the interference of color information on OCR recognition. Grayscale images have smaller data volumes and faster processing speeds. Furthermore, color is not a critical factor for character recognition, and grayscale images are sufficient to analyze the target information. Grayscale conversion allows for greater focus on character outline recognition, improving OCR accuracy and speed.

[0070] Furthermore, edge detection algorithms can identify and highlight edge information within an image. In embodiments of the present invention, the application of edge detection algorithms can clearly identify the boundaries of a document. This enhanced edge feature helps ensure accurate cropping of the document image, reduces recognition errors caused by image tilt or deformation, and improves the recognition effectiveness of OCR technology.

[0071] Furthermore, perspective transformation is an image processing technique used to correct for perspective distortion in images, making tilted surfaces appear closer to the front. In embodiments of the present invention, perspective transformation correction is performed on document images to eliminate distortion caused by the shooting angle, restoring the text and images on the document to their normal upright state. This minimizes changes in shape and size of the text and images, thereby reducing the likelihood of misreading and missed readings and significantly improving OCR recognition rates.

[0072] In summary, the preprocessing steps in the embodiments of the present invention not only solve the technical challenges of financial user identity identification, but also provide financial institutions and financial institutions with a more intelligent, efficient and user-friendly identity identification solution by improving efficiency, enhancing security, improving user experience and simplifying operational processes.

[0073] Step S202: Using a target detection algorithm to locate the facial region of the target financial user in the first image, and extracting a facial feature vector in the facial region.

[0074] Specifically, the object detection algorithm in this embodiment uses computer vision techniques to identify and locate multiple facial regions in an image, while also assigning a category label to each facial region. Compared to traditional pixel-by-pixel classification or sliding window detection methods, this object detection algorithm can significantly reduce detection time and improve facial detection accuracy.

[0075] This invention uses deep learning models, specifically convolutional neural networks (CNNs), to learn and recognize facial patterns from images, outputting bounding boxes with confidence scores to delineate and segment the facial regions of target financial users. The object detection algorithm maintains stable performance in complex environments (e.g., varying lighting, backgrounds, and facial angles).

[0076] The facial area is the portion successfully located by the target detection algorithm in the first image (i.e., the full-body image of the user), and includes all visible features of the target financial user's face.

[0077] To ensure the accuracy and efficiency of subsequent processing, a series of preprocessing steps can be performed on the facial area, including resizing, contrast enhancement, and face alignment, so as to accurately extract facial features from the image. Only based on accurately framed facial images can effective feature vector extraction be performed, thereby achieving high-precision identity authentication.

[0078] Another thing that needs to be explained is that the facial feature vector is a series of numerical values ​​extracted from the located facial area by a pre-trained deep learning model, representing the geometric structure, texture details and shape features of the face. This vector has a fixed dimension, such as 128 or 512 dimensions, and can encode unique information of the face for subsequent comparison and verification. In the present invention, the extraction of facial feature vectors is an automated process that utilizes the powerful feature learning capabilities of deep neural networks, and automatically extracts key features from facial images through multi-level convolution and pooling operations. These features are then compressed into a compact vector representation through a fully connected layer. The feature vector obtained in this way can not only effectively distinguish different individuals, but also maintain a high recognition rate in the case of changes in facial expressions, wearing glasses or masks, etc.

[0079] Optionally, in the financial user identity recognition method provided by an embodiment of the present invention, the step of using a target detection algorithm to locate the facial area of ​​the target financial user in the first image includes: inputting the first image into a target detection model and outputting a detection result, wherein the target detection model is a pre-trained convolutional neural network model based on the target detection algorithm, and the detection result records N bounding boxes and the confidence corresponding to each bounding box, where N is a positive integer; screening M target bounding boxes corresponding to M confidences greater than a first preset threshold, and determining M candidate facial areas based on the M bounding boxes, where M is a positive integer less than or equal to N; performing overlapping detection box filtering processing on the candidate facial areas to obtain R facial areas, and determining the bounding box coordinates of the R facial areas, where R is a positive integer less than or equal to M.

[0080] Optionally, inputting the first image into the target detection model involves passing the captured first image of the target financial user to a pre-trained target detection model. This model is a convolutional neural network (CNN) trained on a large-scale facial dataset and based on a target detection algorithm. The model's task is to find and mark all potential facial regions in the input image and output detection results, which include multiple bounding boxes and a corresponding confidence score for each bounding box. A bounding box is a rectangle defined by (x, y) coordinates that frames the location of a face in the image; the confidence score reflects the model's confidence in the detection result, that is, the probability that the detected bounding box is indeed a face.

[0081] Furthermore, the detection results may include multiple bounding boxes because the model may detect multiple faces in the image, or even false positives of non-face areas. To ensure the accuracy of positioning, it is necessary to screen out those bounding boxes with confidence levels higher than a first preset threshold, filter out false positives with low confidence levels, and retain the most likely face detection results. The candidate facial areas screened out are considered valid candidates containing the target financial user's face.

[0082] Furthermore, there may be partial overlap in the bounding boxes of the filtered candidate facial regions. In order to obtain the most accurate facial regions, these candidate regions need to be filtered for overlapping detection boxes. Usually, a non-maximum suppression algorithm is used to retain the bounding boxes with the highest confidence and non-overlapping according to the confidence and the degree of overlap of the bounding boxes. This ensures that each facial region is independent and most likely to contain a face, avoiding repeated or erroneous facial information extraction.

[0083] The facial region positioning process in the embodiment of the present invention not only solves the technical difficulties of achieving high-precision facial recognition in complex environments, but also achieves additional beneficial effects such as improving user experience, optimizing resource utilization, and strengthening privacy protection.

[0084] Optionally, in the financial user identity identification method provided in an embodiment of the present invention, the target detection model training step includes: obtaining a first training data set containing T financial user image samples, preprocessing all financial user image samples in the first training data set, wherein the preprocessing includes: Gaussian blur denoising, histogram equalization image enhancement and facial area labeling, and T is a positive integer; using a multi-task loss function and the preprocessed first training data set to train the initial target detection model, wherein the multi-task loss function includes a confidence loss function, a classification loss function and a bounding box regression loss function; obtaining a model evaluation index during the training process, and terminating the training when the model evaluation index has not improved for K consecutive preset cycles, to obtain a target detection model, wherein K is a preset positive integer.

[0085] In an optional embodiment, before training the target detection model, a first training dataset containing multiple financial user image samples needs to be prepared. These financial user image samples should cover various lighting conditions, facial expressions, postures, and backgrounds to ensure the robustness of the model in practical applications.

[0086] Preprocessing all financial user image samples in the first training dataset is a key step in optimizing image quality and improving training results. Preprocessing includes the following three main steps: Gaussian blur denoising, histogram equalization image enhancement, and facial region labeling. Specifically:

[0087] Applying Gaussian blur can smooth images, reduce random noise, and improve image clarity and feature detection accuracy. Histogram equalization can enhance image contrast, making facial features more prominent, significantly improving the model's recognition capabilities, especially in low-light conditions. Marking the exact location of the face in the preprocessed image as annotation information during training can help the model learn facial features and boundaries.

[0088] Furthermore, a multi-task loss function is used when training the target detection model, comprehensively considering multiple aspects of target detection, including: confidence loss function, classification loss function and bounding box regression loss function. Specifically, the confidence loss function is used to measure the accuracy of whether the bounding box predicted by the model contains the target; the classification loss function is used to evaluate the model's classification accuracy of the target within the bounding box, that is, whether it can correctly identify it as a face; the bounding box regression loss function is used to optimize the position and size of the bounding box to ensure that the model can accurately frame the facial area.

[0089] Furthermore, during the training process, model evaluation indicators are regularly calculated to monitor the training status and performance of the model, including but not limited to: mean average precision (mAP) or the value of the loss function. If these evaluation indicators do not improve within a specified preset period, training is stopped to avoid overfitting the training data set. This can ensure that the model stops training after reaching the optimal performance, avoid wasting computing resources, and maintain the generalization ability of the model.

[0090] In an embodiment of the present invention, the target detection model trained by using a multi-task loss function and a fully preprocessed training data set can more accurately locate and identify facial areas, and maintain stable performance even in complex environments; the diversity of preprocessing steps and samples ensures that the model can adapt to various changes in lighting and facial posture, enhancing the robustness and adaptability in actual application scenarios; by setting a training termination condition for a specified preset period, unnecessary overtraining is avoided, thereby optimizing training efficiency and saving computing resources; during the training process, facial area labels are used as supervisory information to help the model learn how to recognize and locate faces without exposing the user's sensitive information, thereby improving privacy protection and security during data processing.

[0091] Through the above training steps, the target detection model of the embodiment of the present invention not only solves the technical problem of accurate facial recognition in complex environments, but also has additional beneficial effects such as enhancing robustness, optimizing training efficiency and improving data security.

[0092] Optionally, in the method for identifying financial users provided in an embodiment of the present invention, the step of extracting facial feature vectors from facial areas includes: for each facial area, performing facial key point detection within the facial area to obtain a detection result, wherein the preset facial key points include at least the facial features; aligning the facial areas based on the facial key points recorded in the detection result to obtain a facial image of the target financial user; inputting the facial image into a feature extraction model and outputting a facial feature vector, wherein the feature extraction model is a feature encoder obtained by pre-training a deep convolutional neural network, and is used to map the input facial image into a fixed-dimensional vector.

[0093] It should be noted that facial landmark detection within a facial region involves performing facial landmark detection on each facial region selected by the target detection algorithm to locate prominent facial features such as the eyes, nose, mouth, and eyebrows. Facial landmarks include at least the five facial features, namely the eyes, nose, mouth, left ear, and right ear. The detection process utilizes a deep learning model (such as a regression-based model) to accurately output the location of each landmark in the image, representing the location as (x, y) coordinates.

[0094] The purpose of aligning facial regions based on detected facial key points is to eliminate image distortion caused by head pose, expression changes, or occlusion, ensuring that the facial image input to the feature extraction model has a standardized pose and size. Alignment can be achieved in various ways, such as using an affine transformation to adjust the image's perspective and scale, or constructing an alignment template based on the relative positions of key points. The facial image is then mapped onto this alignment template to obtain the target financial user's facial image.

[0095] The aligned facial image is passed as input to the feature extraction model. The feature extraction model in the embodiment of the present invention is a feature encoder based on a deep convolutional neural network (DCNN), which is used to convert the input facial image into a fixed-dimensional vector, namely a facial feature vector. The facial feature vector contains abstract facial features such as face shape, eye distance, nose shape, etc., which are highly discriminative for identity recognition. The feature extraction model is pre-trained on a large amount of facial data, so it can learn common facial feature expressions.

[0096] The embodiment of the present invention can ensure that the extracted facial feature vectors maintain high consistency and distinctiveness even in the case of changeable head postures or different expressions through facial key point detection and alignment processing, thereby significantly improving the accuracy of identity recognition; the deep learning architecture of the feature extraction model can automatically learn and extract complex features from facial images, and maintain a high recognition rate even in complex environments such as lighting changes and facial occlusion, thereby enhancing the robustness of the model and its generalization ability for unknown data; the facial feature vector provides a standardized and numerical expression of facial features, which facilitates subsequent comparison and matching, simplifies the identity authentication process, and improves processing efficiency; the fixed-dimensional facial feature vector greatly reduces the storage and computing requirements compared to the original image data, and can efficiently process and recognize facial information even on resource-limited devices, thereby lowering the hardware threshold of the financial user identity recognition system.

[0097] Step S203: Perform text recognition processing on the second image to obtain the identity information text of the target financial user.

[0098] Specifically, text recognition processing refers to the optical character recognition (OCR) process performed on the second image. The purpose is to accurately read text information from the ID image, including but not limited to key identification information such as name, ID number, address, etc. Text recognition processing relies on OCR technology to handle complex backgrounds and font styles, and can maintain a high recognition rate even when the image is blurred, at an angular angle, or in poor lighting.

[0099] In some specific implementation scenarios, when processing the second image, the image is first preprocessed (e.g., grayscale conversion, binarization, perspective correction, etc.) to eliminate interference factors and optimize conditions for character recognition. Next, the processed image is analyzed using a trained OCR model to convert the text in the image into text format.

[0100] In other specific implementation scenarios, in order to ensure the accuracy of the recognition results, the text can also be post-processed, such as verifying the format of key fields through regular expressions, removing unnecessary blank characters, and correcting possible recognition errors.

[0101] Furthermore, the identity information text refers to the text description of the target financial user's identity extracted from the second image through text recognition processing, including all data that can clearly identify a person's identity, such as name, ID number, issuing authority, validity period, etc.

[0102] In this embodiment of the present invention, the text successfully recognized from the ID image is integrated into the system's workflow and used, along with the facial feature vector obtained in the first step, to verify the identity of the target financial user. The accuracy and completeness of the identity information text directly impacts the effectiveness and security of the entire identity verification process.

[0103] The present invention can effectively integrate the target financial user's biometric features (facial features) and ID information through the above text recognition processing and identity information text acquisition, greatly improving the efficiency of financial institution services, while also strengthening the protection of personal identity information and improving the security of financial transactions.

[0104] Step S204: perform associated identity recognition based on the facial feature vector and the identity information text to obtain a recognition result, and return the recognition result to the target ATM for financial business processing.

[0105] Specifically, linked identity recognition involves matching and verifying the facial feature vector extracted from the first image with the identity information text obtained through text recognition processing of the second image. Specifically, the database is first searched for records matching the identity information text to find the corresponding registered user; then, the facial feature vector of the registered user stored in the database is compared with the facial feature vector extracted in real time, and a similarity metric (such as Euclidean distance, cosine similarity, etc.) can be used to determine the degree of match between the two; if the similarity of the facial feature vectors exceeds a set threshold and the key fields in the identity information text (such as name, ID number) are consistent with the database record, then the identity of the target financial user is considered to be confirmed; otherwise, further manual review or other auxiliary verification methods are required.

[0106] The identification result is a formal statement of the target financial user's identity verification, including information on whether the identification was successful and other relevant verification details. After the linked identity is identified, the identification result is generated, detailing the key steps and decision-making basis of the identification process.

[0107] For example, a recognition result might include information such as "Facial features match database records by X%, document information is consistent with registration information," or "Facial features do not match records, manual review recommended." Successful recognition results immediately return a "Recognition Successful" message along with the user's detailed identity information, facilitating further financial transactions at the target ATM. Failures also return a detailed reason for failure, allowing the operator to take appropriate action.

[0108] Furthermore, financial service processing encompasses a range of services provided by financial institutions to their customers, including account opening, transfers, loan applications, etc. In the context of embodiments of the present invention, once the identity of the target financial user is confirmed through the associated identity recognition step, the target ATM will receive this positive recognition result and immediately initiate the financial service processing flow for the user. Without manual intervention, the ATM can directly access the user's account information, credit history, etc., simplifying the service application process and speeding up the service processing.

[0109] The present invention can significantly improve the convenience and security of financial institution services by implementing the above-mentioned associated identity recognition process, and reduce the risks of operational errors and abnormal transactions.

[0110] Optionally, in the method for identifying financial users provided in an embodiment of the present invention, the step of performing associated identity identification based on facial feature vectors and identity information text to obtain an identification result includes: querying registered financial users in a pre-stored database and obtaining the registration feature vectors and registration identity information corresponding to the registered financial users; calculating the identity credibility score of the target financial user based on the facial feature vectors, identity information text, registration feature vectors and registration identity information; setting the identification result as credible if the identity credibility score is greater than or equal to a second preset threshold; or setting the identification result as uncredible if the identity credibility score is less than the second preset threshold.

[0111] In a specific implementation scenario, after capturing the facial feature vector and identity information text of the target financial user, the pre-stored database is accessed to search for registered financial user records that match the identity information text. It should be noted that the pre-stored database is used to store detailed information of all previously registered financial users, including but not limited to facial feature vectors and identity information (such as name, ID type and number), so as to facilitate rapid location of the corresponding registration information of the target financial user in the database through query.

[0112] The target financial user's identity credibility score is calculated based on the degree of match between the facial feature vector and the identity information text, specifically including the following two aspects of comparison: facial feature vector similarity calculation and identity information text consistency check.

[0113] Among them, the similarity calculation of facial feature vectors refers to comparing the facial feature vector of the target financial user with the registered feature vector corresponding to the registered financial user using methods such as cosine similarity or Euclidean distance, and obtaining the cosine similarity value or Euclidean distance value as the similarity of the facial feature vectors. A high similarity score means that the facial features match well, which can increase the credibility score.

[0114] The consistency check of the identity information text refers to checking whether the identity information text is completely consistent with all key fields of the registered identity information. String matching or regular expressions can be used to perform verification checks.

[0115] Furthermore, a weighted summation can be used to calculate the identity credibility score. Because facial features provide biometric evidence, the similarity of facial feature vectors carries a higher weight (e.g., 0.7), while the consistency of identity information text provides document-level verification and can have a lower weight (e.g., 0.3). Combining these two aspects of information yields an objective score that reflects the authenticity of the target financial user's identity.

[0116] Finally, the credibility of the identification result is determined based on the calculated identity credibility score. If the score is above a second preset threshold, the target financial user's identity is deemed credible and subsequent financial transactions can proceed. Conversely, if the score is below the second preset threshold, the identification result is marked as untrustworthy, triggering an alert or manual review to ensure that unauthorized users cannot conduct financial transactions.

[0117] Optionally, in the financial user identity identification method provided in an embodiment of the present invention, the step of calculating the identity credibility score of the target financial user based on the facial feature vector, identity information text, registration feature vector and registered identity information includes: comparing the similarity between the facial feature vector and the registration feature vector, and calculating a first score; verifying the consistency between the identity information text and the registered identity information, and calculating a second score; and performing weighted calculation on the first score and the second score to obtain the identity credibility score.

[0118] The embodiment of the present invention not only solves the technical problem of financial user identity identification through the above-mentioned detailed steps, but also has additional beneficial effects such as improving security, realizing automation and real-time verification, reducing operating costs and improving user experience.

[0119] Through the above steps S201 to S204, a first image and a second image of the target financial user can be first obtained, wherein the first image is a full-body image of the user collected by the target ATM, and the second image is an ID image uploaded by the target financial user through the target ATM. Then, a target detection algorithm is used to locate the facial area of ​​the target financial user in the first image, and the facial feature vector in the facial area is extracted. Then, text recognition processing is performed on the second image to obtain the identity information text of the target financial user. Finally, identity association is performed based on the facial feature vector and the identity information text to obtain a recognition result, and the recognition result is returned to the target ATM for financial business processing.

[0120] In an embodiment of the present invention, deep learning is combined with computer vision to achieve the purpose of accurately locating and extracting user facial features and efficiently identifying document information by integrating the user's full-body image collected in real time by the target ATM and the ID image uploaded by the user, thereby realizing a highly automated, real-time, safe and reliable financial user identity authentication technical effect. Specifically, the embodiment of the present invention uses a target detection algorithm to accurately locate the facial area in the user's full-body image, and then extracts the facial feature vector through a deep neural network. At the same time, optical character recognition (OCR) processing is performed on the ID image uploaded by the user to accurately and quickly obtain the identity information text; by combining the facial feature vector with the identity information text for deep correlation analysis and comparison, multi-dimensional verification of the user's identity is achieved, which significantly improves the accuracy and security of recognition, thereby solving the technical problem of low accuracy of user identity recognition by financial institutions in related technologies.

[0121] In addition, the embodiment of the present invention also feeds back the recognition results to the target ATM in real time, which not only accelerates the processing flow of financial services, but also greatly improves the user's service experience. It provides strong technical support for financial institutions to build a safe, efficient and intelligent user identity recognition system, effectively avoiding the subjective errors and inefficiency of manual review, and overcoming the limitations of a single recognition method in complex environments.

[0122] The present invention will be described below in conjunction with another specific embodiment.

[0123] The embodiment of the present invention proposes a customer information recognition method based on the YOLO target detection algorithm. By combining deep learning and image processing technology, it can realize automatic and real-time recognition of customer information of financial institutions, effectively improving the accuracy, speed and security of recognition.

[0124] Figure 3 is an architectural diagram of an optional customer information identification system according to an embodiment of the present invention. Figure 3 As shown, the system includes the following modules:

[0125] Data Acquisition Module: This module is responsible for acquiring real-time customer image data through a camera. The camera should be able to operate under various lighting and environmental conditions to ensure image quality. Multiple sensors (such as infrared sensors) can be incorporated into the data acquisition process to improve recognition accuracy.

[0126] Image preprocessing module: After image acquisition, image preprocessing is first performed, including image denoising, contrast enhancement, and image resizing. In addition, face detection and key point positioning are required to extract the customer's facial features.

[0127] Object Detection Module: This module uses the YOLO algorithm for object detection. YOLO is a real-time object detection algorithm based on convolutional neural networks that can quickly locate and classify objects in images. This module identifies the customer's facial features and extracts relevant information, such as the location of the eyes, nose, and mouth.

[0128] Information recognition module: Based on the target detection results, it further extracts the customer's identity information, including name, ID number and other relevant data. It can be combined with OCR (optical character recognition) technology to identify text information on the document to enhance the comprehensiveness of recognition.

[0129] Database Management Module: This module stores identified customer information in a database for subsequent query and analysis. This module also interacts with existing financial institutions' systems to ensure real-time updates.

[0130] Result Display Module: After identification is complete, customer information is displayed to financial institution staff in real time through the user interface, facilitating subsequent business processing. It also provides information integrity verification to ensure data accuracy and reliability.

[0131] Figure 4 is a flow chart of an optional customer information identification method according to an embodiment of the present invention. Figure 4 As shown, the method includes the following steps:

[0132] 1. Customer Arrival and Data Collection: When a customer enters a financial institution, a camera automatically captures their image. The system monitors customer arrival in real time to ensure a prompt response.

[0133] 2. Image preprocessing: The acquired images are processed by denoising and enhancement to improve the accuracy of subsequent recognition.

[0134] 3. Object Detection: The preprocessed image is fed into the YOLO model for object detection. The model outputs the detected object location (bounding box) and classification label.

[0135] 4. Facial Region Localization: Based on the object detection results (bounding box coordinates and confidence scores) output by YOLO, an image of the customer's facial region is captured. Overlapping detection frames are filtered using a non-maximum suppression algorithm to ensure that the facial region with the highest confidence score is selected as input.

[0136] 5. Facial Feature Extraction: Detect facial key points, achieve facial alignment, and normalize pose. A pre-trained deep learning model is used to input the aligned facial image and extract a 128 / 512-dimensional feature vector. This vector uniquely represents the customer's facial information.

[0137] 6. ID Information OCR Extraction: First, the YOLO algorithm detects the ID card's position and captures the ID area image from the original ID card image captured by the camera. Next, the ID card area is grayscaled, binarized, and perspective corrected to eliminate skew and shadow interference. Finally, an OCR engine is used to recognize the ID card's text information. Finally, regular expressions are used to verify the format of key fields, remove spaces and invalid characters, and correct recognition errors.

[0138] 7. Information Fusion and Verification: The facial feature vector is associated with the text information extracted by OCR (such as name and ID number), and the consistency is verified through database query and comparison. If a match is successful, it is marked as "Recognition Completed"; if it fails, a manual review process is triggered.

[0139] 8. Data Storage and Management: After verification, the customer's identity information and image data are stored in the database to achieve centralized information management. At the same time, the financial institution's customer information database is updated for subsequent business processing and analysis.

[0140] 9. Result display and feedback: The recognition results, including the customer's name, ID number, and other necessary information, are displayed on the user interface. Real-time feedback is provided to financial institution staff to speed up business processing.

[0141] Regarding some implementation details, it is important to note that Gaussian Blur is used for image denoising. The core of Gaussian Blur is to smooth the image using a convolution kernel generated by a Gaussian function, thereby suppressing noise. The steps for implementing Gaussian Blur image denoising are as follows: ① Boundary processing: Fill the image edges (such as mirroring, repeating, or zero-padding) to avoid loss of boundary information during convolution; ② Separability acceleration: Utilizing the separability of the Gaussian kernel, the two-dimensional convolution is decomposed into two one-dimensional convolutions (horizontal + vertical), reducing computational complexity; ③ Multi-channel processing: Gaussian Blur is applied to each channel of the color image (such as RGB) separately.

[0142] In other implementation details, when aligning faces, it is necessary to locate key points and align the detected faces to ensure the consistency of feature extraction.

[0143] The distribution of 68 key points on the face includes: ① Chin contour (17 points): evenly distributed along the jawline from the left to the right end of the chin; ② Right eyebrow (5 points): from the outside to the inside of the right eyebrow, covering the eyebrow contour; ③ Left eyebrow (5 points): from the outside to the inside of the left eyebrow, symmetrically distributed; ④ Nose bridge and nose tip (9 points): from the root of the nose to the tip of the nose on the vertical line of the nose bridge, as well as from the left to the right sides of the nose; ⑤ Right eye (6 points): the outer corner and inner corner of the right eye and the contours of the upper and lower eyelids; ⑥ Left eye (6 points): the outer corner and inner corner of the left eye and the contours of the upper and lower eyelids; ⑦ Outer lip contour (12 points): upper lip and lower lip, surrounding the outer edge of the lips; ⑧ Inner lip contour (8 points): the boundary line between the upper and lower lips, which can capture the details of lip movements.

[0144] The three steps of positioning are as follows: ①P-Net: Generate candidate boxes and preliminary 5 points (eye centers, nose tip, and mouth corners); ②R-Net: Refine the candidate boxes and output accurate 5 points; ③O-Net: Finally output 68-point coordinates and face confidence.

[0145] After obtaining the coordinates of the 68 key points, the key point transformation matrix can be calculated to normalize them.

[0146] The beneficial effects of the embodiments of the present invention include: high recognition accuracy, strong real-time performance, flexible application scenarios, and reduced labor costs, which brings a good user experience and significantly improves the processing efficiency and security of financial services.

[0147] The present invention is described below in conjunction with another optional embodiment.

[0148] Example 2

[0149] An embodiment of the present invention also provides an identity identification device for a financial user. It should be noted that the identity identification device for a financial user in an embodiment of the present invention includes multiple implementation units, which can be used to execute the identity identification method for a financial user provided in the above-mentioned embodiment one, and each implementation unit corresponds to each implementation step in the above-mentioned embodiment one.

[0150] Figure 5 is a schematic diagram of an optional financial user identification device according to an embodiment of the present invention, such as Figure 5 As shown, the device may include: an acquisition unit 51 , an extraction unit 52 , a first recognition unit 53 , and a second recognition unit 54 .

[0151] The acquisition unit 51 is configured to acquire a first image and a second image of the target financial user, wherein the first image is a full-body image of the user captured by the target ATM, and the second image is an ID image uploaded by the target financial user through the target ATM.

[0152] The extraction unit 52 is configured to locate the facial region of the target financial user in the first image using a target detection algorithm, and extract a facial feature vector from the facial region.

[0153] The first recognition unit 53 is configured to perform text recognition processing on the second image to obtain the identity information text of the target financial user.

[0154] The second recognition unit 54 is used to perform associated identity recognition based on the facial feature vector and the identity information text, obtain a recognition result, and return the recognition result to the target ATM for financial business processing.

[0155] The above-mentioned financial user identity recognition device can first obtain a first image and a second image of the target financial user through the acquisition unit 51, wherein the first image is a full-body image of the user collected by the target ATM, and the second image is an ID image uploaded by the target financial user through the target ATM. Then, the extraction unit 52 uses a target detection algorithm to locate the facial area of ​​the target financial user in the first image and extract the facial feature vector in the facial area. Then, the first recognition unit 53 performs text recognition processing on the second image to obtain the identity information text of the target financial user. Finally, the second recognition unit 54 performs associated identity recognition based on the facial feature vector and the identity information text to obtain a recognition result, and returns the recognition result to the target ATM for financial business processing.

[0156] In an embodiment of the present invention, deep learning is combined with computer vision to achieve the purpose of accurately locating and extracting user facial features and efficiently identifying document information by integrating the user's full-body image collected in real time by the target ATM and the ID image uploaded by the user, thereby realizing a highly automated, real-time, safe and reliable financial user identity authentication technical effect. Specifically, the embodiment of the present invention uses a target detection algorithm to accurately locate the facial area in the user's full-body image, and then extracts the facial feature vector through a deep neural network. At the same time, optical character recognition (OCR) processing is performed on the ID image uploaded by the user to accurately and quickly obtain the identity information text; by combining the facial feature vector with the identity information text for deep correlation analysis and comparison, multi-dimensional verification of the user's identity is achieved, which significantly improves the accuracy and security of recognition, thereby solving the technical problem of low accuracy of user identity recognition by financial institutions in related technologies.

[0157] In addition, the embodiment of the present invention also feeds back the recognition results to the target ATM in real time, which not only accelerates the processing flow of financial services, but also greatly improves the user's service experience. It provides strong technical support for financial institutions to build a safe, efficient and intelligent user identity recognition system, effectively avoiding the subjective errors and inefficiency of manual review, and overcoming the limitations of a single recognition method in complex environments.

[0158] Furthermore, the identity recognition device for financial users also includes: a preprocessing unit, the preprocessing unit includes: a first optimization module, which is used to, after obtaining the first image and the second image of the target financial user, denoise the first image based on a Gaussian blur algorithm, and enhance the image contrast using a histogram equalization algorithm to obtain an optimized first image; a second optimization module, which is used to perform grayscale conversion on the second image, enhance the edge features of the document through an edge detection algorithm, and then correct the image deformation using perspective transformation to obtain an optimized second image.

[0159] Furthermore, the extraction unit includes: a first input module, used to input the first image into a target detection model and output a detection result, wherein the target detection model is a pre-trained convolutional neural network model based on a target detection algorithm, and the detection result records N bounding boxes and the confidence corresponding to each bounding box, where N is a positive integer; a screening module, used to screen M target bounding boxes corresponding to M confidences greater than a first preset threshold, and determine M candidate facial regions based on the M bounding boxes, wherein M is a positive integer less than or equal to N; a filtering module, used to perform overlapping detection frame filtering processing on the candidate facial regions to obtain R facial regions, and determine the bounding box coordinates of the R facial regions, wherein R is a positive integer less than or equal to M.

[0160] Furthermore, the financial user identity recognition device also includes: a training unit, which is used to execute the training steps of the target detection model, and the training unit includes: an acquisition module, which is used to obtain a first training data set containing T financial user image samples, and preprocess all financial user image samples in the first training data set, wherein the preprocessing includes: Gaussian blur denoising, histogram equalization image enhancement and facial area labeling, and T is a positive integer; a training module, which is used to train the initial target detection model using a multi-task loss function and the preprocessed first training data set, wherein the multi-task loss function includes a confidence loss function, a classification loss function and a bounding box regression loss function; a termination module, which is used to obtain a model evaluation index during the training process, and terminate the training when the model evaluation index has not improved for K consecutive preset cycles, to obtain a target detection model, wherein K is a preset positive integer.

[0161] Furthermore, the extraction unit also includes: a detection module, which is used to perform facial key point detection in each facial area to obtain a detection result, wherein the preset facial key points include at least the five facial features; an alignment module, which is used to align the facial area based on the facial key points recorded in the detection result to obtain a facial image of the target financial user; a second input module, which is used to input the facial image into a feature extraction model and output a facial feature vector, wherein the feature extraction model is a feature encoder obtained by pre-training based on a deep convolutional neural network, and is used to map the input facial image into a fixed-dimensional vector.

[0162] Furthermore, the second identification unit includes: a query module, used to query registered financial users in a pre-stored database and obtain the registration feature vector and registration identity information corresponding to the registered financial user; a calculation module, used to calculate the identity credibility score of the target financial user based on the facial feature vector, identity information text, registration feature vector and registration identity information; a first setting module, used to set the recognition result as credible when the identity credibility score is greater than or equal to a second preset threshold; and a second setting module, used to set the recognition result as untrustworthy when the identity credibility score is less than the second preset threshold.

[0163] Furthermore, the calculation module includes: a first calculation submodule, used to compare the similarity between the facial feature vector and the registered feature vector, and calculate a first score; a second calculation submodule, used to verify the consistency between the identity information text and the registered identity information, and calculate a second score; a third calculation submodule, used to perform weighted calculation on the first score and the second score to obtain an identity credibility score.

[0164] It should be noted that the acquisition unit 51, extraction unit 52, first identification unit 53, and second identification unit 54 correspond to steps S201 to S204 in the first embodiment. The examples and application scenarios implemented by the above units and corresponding steps are the same, but are not limited to the contents disclosed in the first embodiment. It should be noted that the above modules or units can be hardware components or software components stored in a memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n). The above modules or units can also be part of a device and can be run in the computer terminal 10 provided in the first embodiment.

[0165] The present invention is described below in conjunction with another optional embodiment.

[0166] Example 3

[0167] An embodiment of the present invention may further provide an electronic device, Figure 6 is a structural block diagram of an electronic device for executing a method for identifying a financial user according to an embodiment of the present invention. Figure 6 As shown, the electronic device may include: one or more ( Figure 6 Only one is shown) processor 602, memory 604, storage controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0168] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the financial user identification method and device in the embodiments of the present application. The processor executes the software programs and modules stored in the memory to perform various functional applications and data processing, thereby implementing the aforementioned financial user identification method. The memory can include high-speed random access memory (RAM) and can also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory can further include memory remotely located from the processor, and such remote memory can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0169] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain a first image and a second image of the target financial user, wherein the first image is a full-body image of the user collected by the target ATM, and the second image is an ID image uploaded by the target financial user through the target ATM; use a target detection algorithm to locate the facial area of ​​the target financial user in the first image and extract the facial feature vector in the facial area; perform text recognition processing on the second image to obtain the identity information text of the target financial user; perform associated identity recognition based on the facial feature vector and the identity information text to obtain a recognition result, and return the recognition result to the target ATM for financial business processing.

[0170] The processor can also call the information and application programs stored in the memory through the transmission device to perform the following steps: denoising the first image based on the Gaussian blur algorithm, and enhancing the image contrast using the histogram equalization algorithm to obtain an optimized first image; performing grayscale conversion on the second image, and strengthening the edge features of the document through the edge detection algorithm, and then using perspective transformation to correct image deformation to obtain an optimized second image.

[0171] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: input the first image into the target detection model and output the detection result, wherein the target detection model is a pre-trained convolutional neural network model based on the target detection algorithm, and the detection result records N bounding boxes and the confidence corresponding to each bounding box, where N is a positive integer; screen M target bounding boxes corresponding to M confidences greater than a first preset threshold, and determine M candidate facial areas based on the M bounding boxes, where M is a positive integer less than or equal to N; perform overlapping detection frame filtering on the candidate facial areas to obtain R facial areas, and determine the bounding box coordinates of the R facial areas, where R is a positive integer less than or equal to M.

[0172] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: obtain a first training data set containing T financial user image samples, preprocess all financial user image samples in the first training data set, wherein the preprocessing includes: Gaussian blur denoising, histogram equalization image enhancement and facial area marking, and T is a positive integer; use a multi-task loss function and the preprocessed first training data set to train the initial target detection model, wherein the multi-task loss function includes a confidence loss function, a classification loss function and a bounding box regression loss function; obtain a model evaluation index during the training process, and terminate the training when the model evaluation index does not improve for K consecutive preset cycles to obtain a target detection model, wherein K is a preset positive integer.

[0173] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: for each facial area, perform facial key point detection in the facial area to obtain a detection result, wherein the preset facial key points include at least the facial features; align the facial area based on the facial key points recorded in the detection result to obtain a facial image of the target financial user; input the facial image into a feature extraction model and output a facial feature vector, wherein the feature extraction model is a feature encoder obtained by pre-training a deep convolutional neural network, which is used to map the input facial image into a fixed-dimensional vector.

[0174] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: query the registered financial users in the pre-stored database and obtain the registration feature vector and registration identity information corresponding to the registered financial user; calculate the identity credibility score of the target financial user based on the facial feature vector, identity information text, registration feature vector and registration identity information; if the identity credibility score is greater than or equal to a second preset threshold, set the recognition result to be credible; or, if the identity credibility score is less than the second preset threshold, set the recognition result to be untrustworthy.

[0175] The processor can also call the information and application stored in the memory through the transmission device to perform the following steps: the step of calculating the identity credibility score of the target financial user based on the facial feature vector, the identity information text, the registration feature vector and the registered identity information, including: comparing the similarity between the facial feature vector and the registration feature vector, and calculating a first score; verifying the consistency between the identity information text and the registered identity information, and calculating a second score; performing weighted calculation on the first score and the second score to obtain the identity credibility score.

[0176] An embodiment of the present invention provides an identity recognition solution for financial users. By combining deep learning with computer vision, the solution achieves the goal of accurately locating and extracting user facial features and efficiently identifying ID information by integrating a full-body image of the user captured in real time by a target ATM with an ID image proactively uploaded by the user. This achieves a highly automated, real-time, secure, and reliable financial user identity verification technology. Specifically, the embodiment of the present invention utilizes a target detection algorithm to accurately locate the facial region in the full-body image of the user, and then extracts facial feature vectors through a deep neural network. At the same time, optical character recognition (OCR) is performed on the ID image uploaded by the user to accurately and quickly obtain the identity information text. By combining the facial feature vectors with the identity information text for deep correlation analysis and comparison, multi-dimensional verification of the user's identity is achieved, significantly improving the accuracy and security of identification, and thereby resolving the technical problem of low accuracy in user identity identification by financial institutions in related technologies.

[0177] It can be understood by those skilled in the art that Figure 6 The structure shown is for illustration only, and the electronic device may also be a terminal device such as a smart phone, a tablet computer, a PDA, a mobile Internet device (MID), or a PAD. Figure 6 It does not limit the structure of the above electronic device. For example, the electronic device may also include Figure 6 More or fewer components (such as network interfaces, display devices, etc.) shown in, or with Figure 6 Different configurations shown.

[0178] A person skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0179] The present invention is described below in conjunction with another optional embodiment.

[0180] Example 4

[0181] The embodiment of the present invention further provides a computer-readable storage medium. Optionally, in the embodiment of the present invention, the computer-readable storage medium can be used to store the program code executed by the financial user identity identification method provided in the first embodiment.

[0182] Optionally, in an embodiment of the present invention, the above-mentioned storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0183] An embodiment of the present invention also provides a computer program product, which, when executed on a data processing device, is suitable for executing the steps of a method for identifying a financial user: obtaining a first image and a second image of a target financial user, wherein the first image is a full-body image of the user collected by a target ATM, and the second image is an ID image uploaded by the target financial user through the target ATM; using a target detection algorithm to locate the facial area of ​​the target financial user in the first image, and extracting a facial feature vector in the facial area; performing text recognition processing on the second image to obtain the identity information text of the target financial user; performing associated identity recognition based on the facial feature vector and the identity information text to obtain a recognition result, and returning the recognition result to the target ATM for financial business processing.

[0184] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0185] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0186] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0187] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0188] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0189] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0190] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for identifying a financial user, characterized in that: include: Acquire a first image and a second image of a target financial user, wherein the first image is a full-body image of the user captured by a target ATM, and the second image is an ID image uploaded by the target financial user by the target ATM; Using a target detection algorithm to locate a facial region of the target financial user in the first image, and extracting a facial feature vector from the facial region; Performing text recognition processing on the second image to obtain the identity information text of the target financial user; An associated identity recognition is performed based on the facial feature vector and the identity information text to obtain a recognition result, and the recognition result is returned to the target teller machine for financial business processing.

2. The identity recognition method according to claim 1, characterized in that: After acquiring the first image and the second image of the target financial user, the method further includes: Performing denoising on the first image based on a Gaussian blur algorithm and enhancing image contrast using a histogram equalization algorithm to obtain an optimized first image; The second image is subjected to grayscale conversion processing, and the edge features of the document are enhanced through an edge detection algorithm. The image deformation is then corrected using perspective transformation to obtain an optimized second image.

3. The identity recognition method according to claim 1, characterized in that: The step of locating the facial region of the target financial user in the first image using a target detection algorithm comprises: Inputting the first image into a target detection model and outputting a detection result, wherein the target detection model is a pre-trained convolutional neural network model based on a target detection algorithm, and the detection result records N bounding boxes and a confidence score corresponding to each bounding box, where N is a positive integer; Screening M target bounding boxes corresponding to M confidence levels greater than a first preset threshold, and determining M candidate facial regions based on the M bounding boxes, where M is a positive integer less than or equal to N; Overlapping detection frame filtering is performed on the candidate facial regions to obtain R facial regions, and bounding box coordinates of the R facial regions are determined, where R is a positive integer less than or equal to M.

4. The identity recognition method according to claim 3, characterized in that: The training steps of the target detection model include: Obtain a first training dataset comprising T financial user image samples, and perform preprocessing on all financial user image samples in the first training dataset, wherein the preprocessing includes: Gaussian blur denoising, histogram equalization image enhancement, and facial region labeling, where T is a positive integer; Training an initial object detection model using a multi-task loss function and the preprocessed first training data set, wherein the multi-task loss function includes a confidence loss function, a classification loss function, and a bounding box regression loss function; A model evaluation index is obtained during the training process, and the training is terminated when the model evaluation index does not improve for K consecutive preset cycles to obtain the target detection model, where K is a preset positive integer.

5. The identity recognition method according to claim 3, characterized in that: The step of extracting the facial feature vector in the facial area comprises: For each of the facial regions, facial key points detection is performed within the facial region to obtain a detection result, wherein the preset facial key points include at least facial features; Performing alignment processing on the facial region based on the facial key points recorded in the detection result to obtain a facial image of the target financial user; The facial image is input into a feature extraction model, and the facial feature vector is output, wherein the feature extraction model is a feature encoder obtained by pre-training based on a deep convolutional neural network, and is used to map the input facial image into a fixed-dimensional vector.

6. The identity recognition method according to claim 1, characterized in that: The step of performing associated identity recognition based on the facial feature vector and the identity information text to obtain a recognition result includes: Querying a pre-stored database for registered financial users and obtaining a registration feature vector and registration identity information corresponding to the registered financial user; Calculating an identity credibility score of the target financial user based on the facial feature vector, the identity information text, the registration feature vector, and the registered identity information; If the identity credibility score is greater than or equal to a second preset threshold, the recognition result is set as credible; or, When the identity credibility score is less than the second preset threshold, the recognition result is set as untrustworthy.

7. The identity recognition method according to claim 6, characterized in that: The step of calculating the identity credibility score of the target financial user based on the facial feature vector, the identity information text, the registration feature vector, and the registered identity information includes: comparing the similarity between the facial feature vector and the registered feature vector, and calculating a first score; Verifying the consistency between the identity information text and the registered identity information, and calculating a second score; The first score and the second score are weightedly calculated to obtain the identity credibility score.

8. A financial user identification device, characterized in that: include: an acquisition unit, configured to acquire a first image and a second image of a target financial user, wherein the first image is a full-body image of the user captured by a target ATM, and the second image is an ID image uploaded by the target financial user by the target ATM; an extraction unit, configured to locate a facial region of the target financial user in the first image using a target detection algorithm, and extract a facial feature vector from the facial region; a first recognition unit, configured to perform text recognition processing on the second image to obtain the identity information text of the target financial user; The second recognition unit is used to perform associated identity recognition based on the facial feature vector and the identity information text, obtain a recognition result, and return the recognition result to the target ATM for financial business processing.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the financial user identity identification method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: It includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the financial user identity identification method described in any one of claims 1 to 7.

11. A computer program product, characterized in that The method comprises computer instructions, wherein when the computer instructions are executed by a processor, the steps of the method for identifying the identity of a financial user according to any one of claims 1 to 7 are implemented.

Images