Image recognition method, device, equipment and computer readable storage medium

Abstract

The present application provides an image recognition method, device, equipment and computer-readable storage medium; the method includes: obtaining an image to be recognized, wherein the image to be recognized includes a first region image; determining a first binarization threshold, binarizing the image to be recognized based on the first binarization threshold, and obtaining a first binarized image; determining a second binarization threshold, binarizing the image to be recognized based on the second binarization threshold, and obtaining a second binarized image, wherein the second binarization threshold is less than the first binarization threshold; determining the position information of the first region image based on the first binarization image, and fusing the first binarization image and the second binarization image based on the position information to obtain a target binarized image; and recognizing the target binarization image to obtain a recognition result. Through the present application, the accuracy of image recognition can be improved.

Description

Technical Field

[0001] The present application relates to image processing technology, and in particular to an image recognition method, device, equipment and computer-readable storage medium. Background Art

[0002] With the development of mobile terminals and Internet technology, mobile payment has become the payment method used by most people, and mobile payment has evolved from the previous card payment to the currently widely used scan code payment, biometric recognition payment, etc. When performing biometric recognition payment, in order to increase the light brightness, a fill light device can be set on the code scanning device to improve the success rate of biometric recognition payment. At this time, when using the two-dimensional identification code displayed on the mobile terminal for scanning payment, the two-dimensional identification code will be abnormally exposed due to the reflection of the mirror surface. After the traditional binarization algorithm is processed, the content of the two-dimensional identification code is missing and cannot be recognized. Summary of the invention

[0003] The embodiments of the present application provide an image recognition method, device, and computer-readable storage medium, which can improve recognition accuracy.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] The present application provides an image recognition method, including:

[0006] Acquire an image to be identified, wherein the image to be identified includes a first region image, and the brightness of the first region image is higher than the brightness of the second region image;

[0007] Determine a first binarization threshold, and perform binarization processing on the image to be identified based on the first binarization threshold to obtain a first binarized image;

[0008] Determine a second binarization threshold, and perform binarization processing on the image to be identified based on the second binarization threshold to obtain a second binarized image, wherein the second binarization threshold is less than the first binarization threshold;

[0009] Determining the position information of the first region image based on the first binarized image, and fusing the first binarized image with the second binarized image based on the position information to obtain a target binarized image;

[0010] The target binary image is recognized to obtain a recognition result.

[0011] The present application provides an image recognition device, including:

[0012] A first acquisition module, used for acquiring an image to be identified, wherein the image to be identified includes a first region image, and the brightness of the first region image is higher than the brightness of the second region image;

[0013] A first binarization module, used to determine a first binarization threshold, and perform binarization processing on the image to be identified based on the first binarization threshold to obtain a first binarized image;

[0014] A second binarization module is used to determine a second binarization threshold, and perform binarization processing on the image to be identified based on the second binarization threshold to obtain a second binarized image, wherein the second binarization threshold is less than the first binarization threshold;

[0015] An image fusion module, used for determining the position information of the first region image based on the first binarized image, and fusing the first binarized image with the second binarized image based on the position information to obtain a target binarized image;

[0016] The image recognition module is used to recognize the target binary image and obtain a recognition result.

[0017] In some embodiments, the first binarization module is further used to:

[0018] When the image to be identified is a color image, grayscale processing is performed on the image to be identified to obtain a grayscale image;

[0019] Based on the grayscale image, determining the maximum grayscale value, the minimum grayscale value and the number of pixels corresponding to each grayscale value;

[0020] Determine a plurality of target grayscale intervals based on the maximum grayscale value and the minimum grayscale value;

[0021] Determine the total number of pixels corresponding to each target grayscale interval based on the number of pixels corresponding to each grayscale value;

[0022] A first binarization threshold is determined based on the total number of pixels corresponding to each target grayscale interval.

[0023] In some embodiments, the first binarization module is further used to:

[0024] Get the number of grayscale intervals used to divide the grayscale interval;

[0025] Dividing a plurality of grayscale intervals from the maximum grayscale value to the minimum grayscale value according to the number of grayscale intervals;

[0026] determining the grayscale interval as the target grayscale interval; or,

[0027] A plurality of target grayscale intervals are selected from the plurality of grayscale intervals according to preset interval intervals.

[0028] In some embodiments, the first binarization module is further used to:

[0029] Sorting the target grayscale intervals from large to small according to the maximum grayscale value of the interval to obtain sorted target grayscale intervals;

[0030] Based on the sorted target grayscale intervals, determining an i-th difference between the total number of pixels corresponding to the i+1-th target grayscale interval and the total number of pixels corresponding to the i-th target grayscale interval;

[0031] When the i-th difference is greater than a preset difference threshold, the minimum grayscale value corresponding to the i-th target grayscale interval is determined as a first binarization threshold, where i is an integer between 1 and (N-1), and N is the total number of target grayscale intervals.

[0032] In some embodiments, the second binarization module is further used to:

[0033] Determining each grayscale value from the minimum grayscale value to the maximum grayscale value as a candidate threshold value in sequence;

[0034] Determine a plurality of foreground pixels and a plurality of background pixels corresponding to each candidate threshold;

[0035] Determine the inter-class variance corresponding to each candidate threshold based on the grayscale value corresponding to each foreground pixel and the grayscale value corresponding to each background pixel;

[0036] The candidate threshold corresponding to the maximum inter-class variance is determined as the second binarization threshold.

[0037] In some embodiments, the second binarization module is further used to:

[0038] Determine the foreground pixel ratio and the background pixel ratio based on the total number of foreground pixels and the total number of background pixels.

[0039] Determine a foreground grayscale average value based on the grayscale value corresponding to each foreground pixel and the total number of foreground pixels;

[0040] Determine the background grayscale average value based on the grayscale value corresponding to each background pixel and the total number of background pixels;

[0041] Determining an image grayscale average value of the grayscale image;

[0042] The inter-class variance corresponding to the candidate threshold is determined based on the image average, the foreground pixel ratio, the background pixel ratio, the foreground grayscale average, and the background grayscale average.

[0043] In some embodiments, the image fusion module is further used to:

[0044] Performing filtering on the first binary image to obtain a filtered binary image;

[0045] Obtaining a set of white points in the filtered binary image;

[0046] Fitting the region parameters corresponding to the first region image based on the white point set;

[0047] Position information of the first area image is determined based on the area parameter and the white point set.

[0048] In some embodiments, the image fusion module is further used to:

[0049] Obtaining width information corresponding to the first area image;

[0050] generating a binary reference image based on the region parameter, the white point set and the width information, wherein the pixel points in the first region image in the binary reference image are white points, and the pixel points outside the first region image are black points;

[0051] Determine the coordinate information of each pixel point in the first area image based on the binary reference image;

[0052] The coordinate information of each pixel point is determined as the position information of the first area image.

[0053] In some embodiments, the image fusion module is further used to:

[0054] Obtaining coordinate information of the j-th pixel point in the second binary image, where j=1, 2, ..., M, and M is the total number of pixels in the second binary image;

[0055] When it is determined that the jth pixel is located in the first regional image based on the coordinate information of the jth pixel and the binary reference image, obtaining a first pixel value of the jth pixel in the first binary image;

[0056] Replacing the pixel value of the j-th pixel in the second binary image with the first pixel value;

[0057] When it is determined that the j-th pixel point is outside the first regional image, the pixel value of the j-th pixel point is kept unchanged,

[0058] The processed second binarized image is determined as a target binarized image.

[0059] In some embodiments, when the image to be recognized includes an identification code to be recognized, the image recognition module is further used to:

[0060] Performing identification code positioning processing on the target binary image to obtain position information of the identification code;

[0061] Acquire an identification code image from the target binary image based on the position information of the identification code;

[0062] The identification code image is identified using a preset decoding algorithm to obtain an identification result.

[0063] An embodiment of the present application provides an electronic device, including:

[0064] A memory for storing executable instructions;

[0065] The processor is used to implement the image recognition method provided in the embodiment of the present application when executing the executable instructions stored in the memory.

[0066] An embodiment of the present application provides a computer-readable storage medium storing executable instructions for causing a processor to execute and implement the image recognition method provided in the embodiment of the present application.

[0067] An embodiment of the present application provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the image recognition method provided in the embodiment of the present application.

[0068] The embodiments of the present application have the following beneficial effects:

[0069] After acquiring the image to be recognized including the first region image, firstly, a first binarization threshold is determined, and the image to be recognized is binarized based on the first binarization threshold to obtain a first binarized image, and the first region image is distinguished from the second region image by the first binarization threshold, and then a second binarization threshold is determined, and the image to be recognized is binarized based on the second binarization threshold to obtain a second binarized image, and the second binarization threshold is less than the first binarization threshold, and then the position information of the first region image is determined based on the first binarization image, and the first binarization image and the second binarization image are fused based on the position information to obtain a target binarized image, and then the target binarized image is recognized to obtain a recognition result. Since the image to be recognized includes the first region image, if only the lower second binarization threshold is used to recognize the image to be recognized, If the image of the first region is binarized, the image will be binarized into white dots, resulting in loss of details in the image of the first region. Therefore, in the embodiment of the present application, a higher first binarization threshold is preferentially used to binarize the image to be identified, so as to distinguish the first region image in the image to be identified, and the details in the first region image can be retained. Then, a lower second binarization threshold is used to binarize the image to be identified, so as to binarize other regions except the first region image (at this time, the first region image will be binarized into white dots). Finally, the first binarized image and the second binarized image are fused based on the position information of the first region image to obtain a target binarized image, so that the target binarized image includes the detail information of the first region image and the second region image, thereby ensuring the integrity of the target binarized image. Then, the target binarized image is used for image recognition, which can improve the recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1A A schematic diagram of the implementation process of image recognition for payment in the related art;

[0071] Figure 1B The color image collected in the related technology and the image after the color image is binarized;

[0072] Figure 2A is a schematic diagram of the network architecture of the image recognition system 100 provided in an embodiment of the present application;

[0073] Figure 2B is another schematic diagram of a network architecture of the image recognition system 100 provided in an embodiment of the present application;

[0074] Figure 3 is a schematic diagram of the structure of the server 400 provided in an embodiment of the present application;

[0075] Figure 4 This is a schematic diagram of an implementation flow of the image recognition method provided in an embodiment of the present application;

[0076] Figure 5 A schematic diagram of a process for implementing fusion processing of a first binary image and a second binary image provided in an embodiment of the present application;

[0077] Figure 6 A schematic diagram of another implementation flow of the image recognition method provided in an embodiment of the present application;

[0078] Figure 7 A schematic diagram of another implementation flow of the image recognition method provided in the embodiment of the present application;

[0079] Fig. 8A is a grayscale image obtained by graying a color image;

[0080] Figure 8B Is the grayscale histogram corresponding to the grayscale image;

[0081] Figure 8C is a schematic diagram of a binary image obtained after the first binarization process;

[0082] Fig.8D A schematic diagram of an image obtained after median filtering is performed on the binary image obtained after the first binarization process;

[0083] Fig. 9 is a schematic diagram of a binarized image obtained after the second binarization process;

[0084] Fig.10 Schematic diagram of the completed ellipse. DETAILED DESCRIPTION

[0085] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0086] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0087] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0088] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0089] In order to better understand the image recognition method provided in the embodiment of the present application, the image recognition algorithm used for payment in the related art and its existing shortcomings are first explained. Figure 1A The following is a schematic diagram of the implementation process of image recognition for payment in the related art. Figure 1A As shown, the process includes:

[0090] Step S001, collecting color images.

[0091] Step S002, pre-processing the collected color image.

[0092] The preprocessing here at least includes noise processing and binarization processing.

[0093] Step S003, two-dimensional identification code positioning.

[0094] Step S004, decoding the two-dimensional identification code.

[0095] Since the 2D code is usually a black code with a white background, it is easy to filter out interference information through binarization to highlight the content of the 2D code. However, in some actual uses, if the code scanning device has an annular fill light, the 2D code image will be incomplete due to the influence of the aperture after the 2D code image is binarized, and the 2D code cannot be located or parsed. Figure 1B is a color image collected in the related art and an image after the color image is binarized, wherein: Figure 1B In the figure, 011 is the collected color image, and 012 is the image after binarization of the color image. It can be seen from 012 that the two-dimensional identification code in the binarized image is partially incomplete, which may easily cause recognition failure.

[0096] Based on this, the embodiments of the present application provide an image recognition method, apparatus, device and computer-readable storage medium, which can improve the recognition success rate. The exemplary application of the electronic device provided by the embodiments of the present application is described below. The electronic device provided by the embodiments of the present application can be implemented as various types of user terminals such as laptops, tablet computers, desktop computers, set-top boxes, mobile devices (e.g., mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices), and can also be implemented as servers. Below, an exemplary application when the device is implemented as a server will be described.

[0097] See also Figure 2A , Figure 2A is a schematic diagram of the network architecture of the image recognition system 100 provided in the embodiment of the present application, such as Figure 2A As shown, the network architecture includes a terminal 200, a network 300 and a server 400, wherein the terminal 200 is connected to the server 400 via the network 300, and the network 300 may be a wide area network or a local area network, or a combination of the two.

[0098] The terminal 200 obtains an image to be identified, which may be an image acquired in real time by the terminal through its own image acquisition device, an image stored in the local storage space of the terminal, or an image sent to the terminal 200 by other terminals. In the embodiment of the present application, the source of the image to be identified is not limited. The image to be identified includes a first area image, which may be caused by the strong light emitted by the fill light device when the image to be identified is acquired. The terminal 200 sends an image recognition request to the server 400 through the network 300. The image recognition request carries an image to be recognized. After receiving the image recognition request, the server 400 obtains the image to be recognized, determines a first binarization threshold, and performs binarization processing on the image to be recognized based on the first binarization threshold to obtain a first binarized image; determines a second binarization threshold, and performs binarization processing on the image to be recognized based on the second binarization threshold to obtain a second binarized image, wherein the second binarization threshold is less than the first binarization threshold; determines the position information of the first area image based on the first binarization image, and performs fusion processing on the first binarization image and the second binarization image based on the position information to obtain a target binarized image; recognizes the target binarization image to obtain a recognition result, and the server 400 returns the recognition result to the terminal 200.

[0099] In an embodiment of the present application, the image to be identified is binarized twice using a higher first binarization threshold and a lower second binarization threshold to obtain a first binarized image and a second binarized image, respectively, and the first binarized image retains the details in the first region image, and the second binarized image retains the details in the second region image (at this time, the first region image will be binarized into white dots). Finally, the first binarized image and the second binarized image are fused based on the position information of the first region image to obtain a target binarized image, so that the target binarized image includes the detail information of the first region image and the second region image, thereby ensuring the integrity of the target binarized image, and then using the target binarized image for image recognition, which can improve the recognition efficiency.

[0100] See also Figure 2B , Figure 2B is another network architecture diagram of the image recognition system 100 provided in the embodiment of the present application, such as Figure 2BAs shown, the network architecture includes a terminal 101, a merchant acquiring system 102, a payment system 103 and a code issuing platform 104. Among them, the terminal 101 can be a mobile terminal with wireless communication capabilities such as a mobile phone (cell phone), a tablet computer, a laptop computer, etc. A payment application (Application, App) that can scan code payment can be installed in the terminal 101. Users can generate payment QR codes through these apps, so that merchants can collect payments by scanning codes. The merchant acquiring system 102 may include a scanning device 1021 and an acquiring device 1022. In an embodiment of the present application, a fill light device may be provided on the scanning device 1021 to ensure that images with higher brightness can be captured when performing biometric payment, thereby improving recognition efficiency. However, when the user pays by presenting the QR code, since the fill-light device causes the image collected by the scanning device to include the first area image generated by the fill-light device, at this time, after acquiring the image to be identified, the scanning device 1021 first determines a first binarization threshold, and binarizes the image to be identified based on the first binarization threshold to obtain a first binarized image; determines a second binarization threshold, and binarizes the image to be identified based on the second binarization threshold to obtain a second binarized image, wherein the second binarization threshold is less than the first binarization threshold; and determines a second binarization threshold based on the first binarization threshold to obtain a second binarized image. The scanning device 1021 determines the position information of the first area image by using the binary image, and fuses the first binary image and the second binary image based on the position information to obtain the target binary image; the target binary image is recognized to obtain the recognition result, and the scanning device 1021 sends the recognition result to the acquiring device 1022, and the acquiring device 1022 generates order information based on the recognition result, and then sends the order information to the payment system 103, which can be considered as the settlement center of the merchant; the payment system sends the order information to the code issuing platform 104 to notify the code issuing platform 104 to deduct the payment. In addition to deducting the payment, the code issuing platform 104 will also provide the terminal with certificate data for generating a payment QR code, so that the terminal can generate a payment QR code based on the certificate data.

[0101] exist Figure 2B The example of QR code payment is used for illustrative explanation. The image recognition method provided in the embodiment of the present application can be applied to any scenario that requires QR code recognition, such as access control systems and public transportation systems that recognize QR codes.

[0102] In some embodiments, the server 400 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal 200 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a car-mounted smart terminal, etc., but is not limited thereto. The terminal and the server may be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiments of the present application.

[0103] See also Figure 3 , Figure 3 A schematic diagram of the structure of the server 400 provided in the embodiment of the present application, Figure 3 The server 400 shown includes: at least one processor 410, at least one network interface 420, a bus system 430 and a memory 440. The various components in the server 400 are coupled together through the bus system 430. It can be understood that the bus system 430 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 430 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 430 is not used in the example of FIG. Figure 3 Various buses are labeled as bus system 430 .

[0104] Processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0105] The memory 440 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical drives, etc. The memory 440 may optionally include one or more storage devices that are physically remote from the processor 410.

[0106] The memory 440 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 440 described in the embodiments of the present application is intended to include any suitable type of memory.

[0107] In some embodiments, memory 440 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0108] Operating system 441, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0109] A network communication module 442, for reaching other computing devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 include: Bluetooth, Wireless Compatibility Authentication (WiFi), and Universal Serial Bus (USB);

[0110] In some embodiments, the device provided in the embodiments of the present application can be implemented in software. Figure 3 An image recognition device 443 stored in a memory 440 is shown, which may be software in the form of a program or a plug-in, and includes the following software modules: a first acquisition module 4431, a first binarization module 4432, a second binarization module 4433, an image fusion module 4434, and an image recognition module 4435. These modules are logical, and thus may be arbitrarily combined or further split according to the functions implemented. The functions of each module will be described below.

[0111] In other embodiments, the device provided in the embodiments of the present application can be implemented in hardware. As an example, the device provided in the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the image recognition method provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs) or other electronic components.

[0112] The image recognition method provided in the embodiment of the present application will be described in conjunction with the exemplary application and implementation of the terminal provided in the embodiment of the present application.

[0113] The present application embodiment provides an image recognition method, which is applied to an electronic device, which may be a terminal or a server. In the present application embodiment, the electronic device is taken as a server as an example for description. Figure 4 , Figure 4 This is a schematic diagram of an implementation flow of the image recognition method provided in the embodiment of the present application. Figure 4 The steps shown illustrate the image recognition method provided in the embodiment of the present application.

[0114] Step S201, obtaining an image to be recognized.

[0115] Among them, the image to be recognized includes a first area image. The first area image can be caused by the reflection of light emitted by the fill light device on the display screen of the acquisition device when the image to be recognized is collected. The shape of the first area image is related to the shape of the light source of the fill light device. In order to make the fill light uniform, the light source of the fill light device is generally circular. When the light source shape of the fill light device is circular, the shape of the first area image is circular or elliptical. The image to be recognized can be an image recognition request sent by the terminal received by the server and parsed from the image recognition request, or it can be obtained by the server from its own storage space. In an embodiment of the present application, the image to be recognized can be a color image or a grayscale image.

[0116] Step S202: determine a first binarization threshold, and perform binarization processing on the image to be identified based on the first binarization threshold to obtain a first binarized image.

[0117] In this step, the first binarization threshold value determined is the grayscale threshold value that can distinguish the first area image and the second area image in the image to be identified. When determining the first binarization threshold value, the image to be identified can be firstly grayscale processed to obtain a grayscale image, and then the number of pixels corresponding to each grayscale value is counted, and then multiple grayscale intervals are divided from the highest grayscale value to the lowest grayscale value in each grayscale value, and multiple target grayscale intervals are selected from them, and the first binarization threshold value is determined by the difference in the total number of pixels in adjacent target grayscale intervals. When it is determined that the number of pixels has increased sharply based on the difference in the total number of pixels in adjacent target grayscale intervals, it means that the first binarization threshold value has been reached.

[0118] When the image to be identified is binarized based on the first binarization threshold, the grayscale values ​​of the pixels whose grayscale values ​​are greater than the first binarization threshold in the grayscale image corresponding to the image to be identified are set to the first preset value, and the grayscale values ​​of the pixels whose grayscale values ​​are less than or equal to the first binarization threshold are set to the second preset value. The first preset value and the second preset value are different, and the difference between the first preset value and the second preset value is greater than a certain grayscale threshold, for example, the first preset value may be 255, the second preset value may be 0, or the first preset value may be 250, and the second preset value may be 10.

[0119] Step S203, determining a second binarization threshold, and performing binarization processing on the image to be identified based on the second binarization threshold to obtain a second binarized image, wherein the second binarization threshold is smaller than the first binarization threshold.

[0120] In the implementation of this step, each grayscale value between the minimum grayscale value and the maximum grayscale value in the grayscale image corresponding to the image to be identified can be used as a candidate threshold, and then the inter-class variance of the foreground and background is determined, and the candidate threshold corresponding to the maximum inter-class variance is determined as the second binarization threshold.

[0121] When the image to be identified is binarized based on the second binarization threshold, the process is similar to that of step S202 in which the binarization is performed based on the first binarization threshold. The grayscale values ​​of pixels whose grayscale values ​​are greater than the second binarization threshold in the grayscale image corresponding to the image to be identified are set to the first preset value, and the grayscale values ​​of pixels whose grayscale values ​​are less than or equal to the second binarization threshold are set to the second preset value.

[0122] Step S204: determining the position information of the first region image based on the first binarized image, and fusing the first binarized image and the second binarized image based on the position information to obtain a target binarized image.

[0123] When implementing this step, firstly, the coordinate information of the white point set is determined based on the second binary image, and then the white point set is used for graphic fitting to obtain the regional parameters that can characterize the shape of the first regional image. The regional parameters and the white point set can be used to generate a binary reference image, wherein the pixel points in the first regional image in the binary reference image are white points, and the pixel points outside the first regional image are black points. Then, using the binary reference image as a reference, when it is determined that a certain pixel point is in the first regional image, the pixel value of the pixel point in the second binary image is replaced by the pixel value of the pixel point in the first binary image, and after traversing each pixel point in the second binary image, the target binary image is obtained.

[0124] Step S205, identifying the target binary image to obtain a recognition result.

[0125] When the image to be identified includes an identification code to be identified, when implementing this step, the identification code can be first located based on the target binary image to determine the position information of the identification code, and then the identification code image can be determined based on the position information of the identification code. Finally, the identification code image can be decoded using a preset decoding algorithm to obtain a recognition result, which can include user identification information corresponding to the identification code.

[0126] When the image to be identified is another image that does not include the identification code to be identified, when implementing this step, contour information of the object to be identified included in the image to be identified can be determined based on the target binary image, and then identification is performed based on the contour information to obtain a recognition result.

[0127] In the image recognition method provided in the embodiment of the present application, after obtaining the image to be recognized including the first area image, the first binarization threshold is first determined, and the image to be recognized is binarized based on the first binarization threshold to obtain the first binarized image, and the first area image and the second area image are distinguished by the first binarization threshold, and then the second binarization threshold is determined, and the image to be recognized is binarized based on the second binarization threshold to obtain the second binarized image, and the second binarization threshold is smaller than the first binarization threshold. Since the image to be recognized includes the first area image with higher brightness, if only the lower second binarization threshold is used to binarize the image to be recognized, the first area image will be binarized into white points, resulting in the thin film in the first area image being blurred. Section is missing, therefore, in the embodiment of the present application, a higher first binarization threshold is preferentially used to perform binarization processing on the image to be identified, so as to distinguish the first area image in the image to be identified, and also retain the details in the first area image, and then a lower second binarization threshold is used to perform binarization processing on the image to be identified, so as to perform binarization processing on other areas except the first area image (at this time, the first area image will be binarized into white points), and finally, the first binarization image and the second binarization image are fused based on the position information of the first area image to obtain a target binarization image, so that the target binarization image includes the detail information of the first area image and the second area image, thereby ensuring the integrity of the target binarization image, and then using the target binarization image for image recognition, which can improve the recognition efficiency.

[0128] In some embodiments, the above step S202 of "determining a first binarization threshold" can be implemented through the following steps S2021 to S2025, and each step is described below.

[0129] Step S2021: When the image to be identified is a color image, grayscale processing is performed on the image to be identified to obtain a grayscale image.

[0130] When the image to be identified is a color image, the color of each pixel is determined by three color components. When the color image is an RGB image, the color of each pixel is determined by three color components: R, G, and B. Grayscale processing of the image to be identified is to represent each pixel in the image to be identified with a color component. The color component is also the grayscale value, which ranges from 0 to 255.

[0131] When implementing grayscale processing of the image to be recognized, the maximum value of the three color components of the pixel can be determined as the grayscale value of the pixel, the average value of the three color components of the pixel can be determined as the grayscale value of the pixel, or the three color components of the pixel can be weighted averaged to obtain the grayscale value of the pixel.

[0132] Step S2022: Based on the grayscale image, determine the maximum grayscale value, the minimum grayscale value, and the number of pixels corresponding to each grayscale value.

[0133] After obtaining the grayscale image of the image to be recognized, traverse each pixel of the grayscale image to determine the maximum grayscale value, the minimum grayscale value and the number of pixels corresponding to each grayscale value.

[0134] In some embodiments, a grayscale histogram may be generated based on the number of pixels corresponding to each grayscale value. The grayscale histogram may be used to intuitively understand the difference in the number of pixels corresponding to different grayscale values.

[0135] Step S2023, determining a plurality of target grayscale intervals based on the maximum grayscale value and the minimum grayscale value.

[0136] In actual application, the number of grayscale intervals used to divide the grayscale intervals can be first obtained, and then multiple grayscale intervals can be divided from the maximum grayscale value to the minimum grayscale value according to the number of grayscale intervals. At this time, there are at least two ways to determine the target grayscale interval: one is to directly determine the multiple grayscale intervals obtained as the target grayscale interval; the other is to select multiple target grayscale intervals from the multiple grayscale intervals according to the preset interval intervals.

[0137] The number of grayscale intervals is an integer greater than 1, and the number of grayscale intervals can be pre-set, for example, 5, 10, etc.; it can also be determined based on the difference between the maximum grayscale value and the minimum grayscale value. In this case, the number of grayscale intervals can be a divisor of the difference. For example, if the difference between the maximum grayscale value and the minimum grayscale value is 240, the number of grayscale intervals can be 5, 8, 12, etc., to ensure that the interval between the maximum grayscale value and the minimum grayscale value can be evenly divided.

[0138] Take the maximum grayscale value as 245, the minimum grayscale value as 6, and the number of grayscale intervals as 10 as an example. At this time, the grayscale value from the maximum grayscale value to the minimum grayscale value is divided according to the number of grayscale intervals, and the grayscale values ​​are respectively divided into the first grayscale interval from 236 to 245, the second grayscale interval from 226 to 235, the third grayscale interval from 216 to 225, and so on, until the 24th grayscale interval from 6 to 15 is obtained. At this time, these 24 grayscale intervals can be determined as target grayscale intervals, or multiple target grayscale intervals can be selected from these 24 grayscale intervals according to the preset interval interval. Among them, the preset interval interval is an integer greater than or equal to 1. For example, when the preset interval interval is 1, a target grayscale interval is selected for every grayscale interval. If there are 24 grayscale intervals, that is, the first, third, fifth grayscale intervals...the twenty-third grayscale interval is selected as the target grayscale interval.

[0139] Step S2024, determining the total number of pixels corresponding to each target grayscale interval based on the number of pixels corresponding to each grayscale value.

[0140] When this step is implemented, the number of pixel points corresponding to the grayscale values ​​included in each target grayscale interval is summed up to obtain the total number of pixel points corresponding to each target grayscale interval.

[0141] Step S2025: determining a first binarization threshold based on the total number of pixels corresponding to each target grayscale interval.

[0142] In some embodiments, this step can be implemented through the following steps S251 to S253, and each step is described below.

[0143] Step S251 , sorting the target grayscale intervals from large to small according to the maximum grayscale value of the interval, to obtain sorted target grayscale intervals.

[0144] Here, each target grayscale interval corresponds to a maximum grayscale value and a minimum grayscale value. For example, for the target grayscale interval [236, 245], the maximum grayscale value is 245 and the minimum grayscale value is 236. Sort the target grayscale intervals from small to large according to the maximum grayscale value of the interval, that is, sort the target grayscale intervals from bright to dark, and obtain the sorted target grayscale intervals.

[0145] Step S252: Based on the sorted target grayscale intervals, determine the i-th difference between the total number of pixels corresponding to the i+1-th target grayscale interval and the total number of pixels corresponding to the i-th target grayscale interval.

[0146] Wherein, i is an integer between 1 and (N-1), and N is the total number of target grayscale intervals. In this step, the difference between the total number of pixels corresponding to two adjacent target grayscale intervals is calculated. When implemented, the total number of pixels corresponding to the i-th target grayscale interval is subtracted from the total number of pixels in the i+1-th target grayscale interval to obtain the i-th difference.

[0147] Step S253, determining whether the i-th difference is greater than a preset difference threshold.

[0148] Among them, since the image to be identified in the embodiment of the present application includes a first area image whose brightness is significantly higher than that of most areas of the image to be identified, when the difference in the total number of pixels corresponding to two adjacent target grayscale intervals is greater than a preset difference threshold, it means that the grayscale value for distinguishing the first area image and the second area image has been reached, and step S254 is entered at this time; when the i-th difference is less than or equal to the difference threshold, it means that the grayscale value for distinguishing the first area image and the second area image has not been reached, and step S255 is entered at this time.

[0149] Step S254: Determine the minimum grayscale value of the interval corresponding to the i-th target grayscale interval as the first binarization threshold.

[0150] Here, when it is determined that the difference in the total number of pixels between two adjacent target grayscale intervals reaches the difference threshold, it means that the distinguishing grayscale value between the first area image and the second area image is reached. Then, at the first binarization threshold, the minimum grayscale value of the interval corresponding to the i-th target grayscale interval is determined as the first binarization threshold, that is, when the minimum grayscale value corresponding to the previous target grayscale interval between two adjacent target grayscale intervals is determined as the first binarization threshold.

[0151] Step S255, set i to i+1 and go to step S252.

[0152] When determining the first binarization threshold for distinguishing the first region image from the second region image, through the above steps S2021 to S2025, the image to be identified is firstly grayscale processed to obtain a grayscale image, and then the number of pixels corresponding to each grayscale value in the grayscale image is counted, and the maximum grayscale value and the minimum grayscale value of the grayscale image are used to determine multiple target grayscale spaces, and then the first binarization threshold is determined based on the difference in the total number of pixels in adjacent target grayscale intervals. Since the average brightness of the first region image is significantly higher than the average brightness of the second region image, in the embodiment of the present application, the total number of pixels is counted from the maximum grayscale value forward, and when it is determined that the difference in the total number of pixels in adjacent target grayscale intervals is greater than the preset difference threshold, it means that the total number of pixels in the two adjacent target grayscale intervals has undergone a sudden change, and it is considered that the distinguishing grayscale value for distinguishing the first region image from the second region image has been reached, thereby ensuring that the determined first binarization threshold can accurately realize the binarization distinction between the first region image and the second region image.

[0153] In some embodiments, "determining the second binarization threshold" in the above step S203 can be implemented through the following steps S2031 to S2034, and each step is described below.

[0154] Step S2031, determining each grayscale value from the minimum grayscale value to the maximum grayscale value as a candidate threshold value in sequence.

[0155] For example, the minimum grayscale value is 6 and the maximum grayscale value is 245, then the grayscale values ​​6, 7, 8, ..., 245 are respectively determined as candidate thresholds.

[0156] Step S2032, determining a plurality of foreground pixels and a plurality of background pixels corresponding to each candidate threshold.

[0157] In an embodiment of the present application, pixels whose grayscale values ​​are less than the candidate threshold are determined as background pixels, and pixels whose grayscale values ​​are greater than or equal to the candidate threshold are determined as foreground pixels.

[0158] Step S2033: determining the inter-class variance corresponding to each candidate threshold based on the grayscale value corresponding to each foreground pixel and the grayscale value corresponding to each background pixel.

[0159] When implementing this step, first calculate the background pixel ratio and foreground pixel ratio under each candidate threshold, and calculate the background grayscale average and foreground grayscale average, and then use the variance calculation formula to calculate the inter-class variance of the foreground and background corresponding to each candidate threshold.

[0160] Step S2034: determine the candidate threshold corresponding to the maximum inter-class variance as the second binarization threshold.

[0161] Since variance is a measure of the uniformity of grayscale distribution, the larger the inter-class variance between the background and the foreground, the greater the difference between the two parts that constitute the image. Therefore, in this step, the candidate threshold corresponding to the maximum inter-class variance is determined as the second binarization threshold, which can ensure that the foreground and background parts can be distinguished to the greatest extent by using the second binarization threshold.

[0162] In some embodiments, the above step S2033 "determining the inter-class variance corresponding to each candidate threshold based on the grayscale value corresponding to each foreground pixel and the grayscale value corresponding to each background pixel" can be implemented through the following steps S331 to S335, and each step is explained below.

[0163] Step S331, determining a foreground pixel ratio and a background pixel ratio based on the total number of foreground pixel points and the total number of background pixel points.

[0164] When implementing this step, first obtain the total number of image pixels of the grayscale image, then divide the total number of foreground pixels by the total number of image pixels to obtain the foreground pixel ratio, and divide the total number of background pixels by the total number of image pixels to obtain the background pixel ratio. Therefore, the foreground pixel ratio and the background pixel ratio are both real numbers greater than 0 and less than 1, and the sum of the foreground pixel ratio and the background pixel ratio is 1.

[0165] Step S332: determining a foreground grayscale average value based on the grayscale value corresponding to each foreground pixel and the total number of foreground pixels.

[0166] Here, the grayscale values ​​corresponding to the foreground pixels are summed to obtain the foreground total grayscale value, and then the foreground total grayscale value is divided by the total number of foreground pixels to obtain the foreground grayscale average value, which is a real number between 0 and 255.

[0167] Step S333: determining the background grayscale average value based on the grayscale value corresponding to each background pixel and the total number of background pixels.

[0168] Here, the grayscale values ​​corresponding to the background pixels are summed to obtain the total background grayscale value, and then the total background grayscale value is divided by the total number of background pixels to obtain the background grayscale average value, which is also a real number between 0 and 255.

[0169] Step S334, determining the average grayscale value of the grayscale image.

[0170] When implementing this step, the grayscale values ​​of each pixel are summed to obtain the total grayscale value of the image, and then the total grayscale value of the image is divided by the total number of image pixels to obtain the average grayscale value of the image.

[0171] Step S335 , determining the inter-class variance corresponding to the candidate threshold based on the image grayscale average, the foreground pixel ratio, the background pixel ratio, the foreground grayscale average, and the background grayscale average.

[0172] Here, we can use formula (1-1) to determine the between-class variance:

[0173] g=ω0*(u0-u)*(u0-u)+ω1*(u1-u)*(u1-u) (1-1);

[0174] Among them, u is the average grayscale value of the image, ω0 is the foreground pixel ratio, ω1 is the background pixel ratio, u0 is the foreground grayscale average, and u1 is the background grayscale average.

[0175] In the embodiment of steps S331 to S335, the foreground pixel ratio and the background pixel ratio are first determined, and then the foreground grayscale average and the background grayscale average are determined, and then the variance formula is used to determine the inter-class variance corresponding to each candidate threshold, thereby providing a data basis for determining the second binarization threshold.

[0176] In some embodiments, the above step S204 "determining the position information of the first region image based on the first binarized image, and fusing the first binarized image and the second binarized image based on the position information to obtain a target binarized image" can be performed by Figure 5 Steps S2041 to S2049 shown in the figure are implemented by combining the following Figure 5 Each step is explained.

[0177] Step S2041: filter the first binarized image to obtain a filtered binarized image.

[0178] Although the first binarization threshold can distinguish the first area image and the second area image in the image to be identified, since the first binarization threshold is determined by dividing the target grayscale interval, if the number of grayscale intervals in the target grayscale interval is large, the accuracy of the first binarization threshold will be reduced, resulting in the presence of noise points in the first binarized image obtained by binarization processing with the first binarization threshold. Therefore, it is necessary to filter the first binarized image to filter out the noise points in the first binarized image. In the implementation of this step, the first binarized image can be median filtered to obtain a filtered binarized image, or the first binarized image can be mean filtered to obtain a filtered binarized image. For example, the neighborhood size can be selected as 3*3, and mean filtering or median filtering can be performed.

[0179] Step S2042, obtaining a set of white points in the filtered binary image.

[0180] Since the filtered binary image only includes two types of pixels: one is a white pixel and the other is a black pixel, this step can be implemented by traversing the filtered binary image and obtaining the grayscale value of each pixel in turn. If the grayscale value of a pixel indicates that the pixel is a white pixel, the pixel is added to the white point set; if the grayscale value of the pixel indicates that the pixel is a black pixel, the pixel is ignored.

[0181] In the actual implementation process, in order to improve the traversal efficiency, the number of rows or columns may be preset at each interval to obtain the set of white points in the filtered binary image.

[0182] Step S2043: fitting the region parameters corresponding to the first region image based on the white point set.

[0183] Since the light source of the fill light device is generally annular in actual application, the first area image formed is annular or elliptical. Based on this feature, the fitEllipse function provided by OpenCV can be used. The input of this function is a set of white points, and the output is the area parameters of the fitted ellipse. When the fitted first area image is an ellipse, the area parameters may include two focus coordinates, two major axis vertex coordinates, and two minor axis vertex coordinates. When the fitted first area image is a rectangle, the area parameters may be the coordinates of the two diagonal vertices.

[0184] Step S2044: determining the position information of the first region image based on the region parameters and the white point set.

[0185] Since the edge of the first region image is generally incomplete in the first binary image obtained by binarizing the image to be identified by the first binary threshold, after the region parameters are determined, a binary reference image with complete edges can be drawn based on the region parameters. In the embodiment of the present application, the position information of the first region image can be the coordinate information of each pixel in the first region image, and therefore, the position information of the first region image is obtained by determining the coordinate information of each white point in the binary reference image.

[0186] Step S2045, obtaining coordinate information of the j-th pixel point in the second binary image.

[0187] Wherein, j=1, 2, ..., M, where M is the total number of pixels of the second binary image. The coordinate information of the j-th pixel is a two-dimensional coordinate, for example, (x j, ,y j ), where x j, is the horizontal coordinate value, y j Is the vertical coordinate value.

[0188] Step S2046, determining whether the jth pixel is located in the first region image based on the coordinate information of the jth pixel and the binary reference image.

[0189] When implementing this step, the grayscale value of the jth pixel in the binary reference image can be obtained based on the coordinate information of the jth pixel. If the grayscale value of the jth pixel in the binary reference image represents that the jth pixel is a white point in the binary reference image, it means that the jth pixel is in the first area image, and step S2047 is entered at this time; if the grayscale value of the jth pixel in the binary reference image represents that the jth pixel is a black point in the binary reference image, it means that the jth pixel is outside the first area image, and step S2049 is entered at this time.

[0190] Step S2047: Obtain a first pixel value of the j-th pixel in the first binary image.

[0191] During implementation, the first pixel value of the jth pixel in the first binary image may be obtained based on the coordinate information of the jth pixel.

[0192] Step S2048: Replace the pixel value of the j-th pixel in the second binary image with the first pixel value.

[0193] Assuming that the pixel value of the j-th pixel in the second binary image is 255, and the pixel value of the j-th pixel in the first binary image is 0, then the pixel value of the j-th pixel in the second binary image is replaced with 0.

[0194] Step S2049, keeping the pixel value of the j-th pixel point unchanged.

[0195] Step S20410: determine the processed second binarized image as the target binarized image.

[0196] In the embodiment of the present application, since the second binarized image is obtained by binarizing the image to be identified based on the second binarization threshold, the first area image with higher brightness will be binarized as white dots in the second binarized image, which will cause loss of details in the first area image. The first binarized image is obtained by binarizing the image to be identified based on the first binarization threshold higher than the second binarization threshold. Therefore, the details of the first area image are retained in the first binarized image, but most of the second area image will be binarized as black dots. Therefore, when the first binarized image and the second binarized image are fused, the pixel values ​​of the pixels in the second binarized image at the corresponding positions of the first area image are replaced with the corresponding pixels in the first binarized image. In this way, the target binarized image can include both the detail information in the highlighted first area image and the detail information in the non-highlighted area.

[0197] In some embodiments, the above step S2044 "determining the position information of the first area image based on the area parameters and the white point set" can be implemented through the following steps S441 to S444, and each step is described below.

[0198] Step S441, obtaining width information corresponding to the first region image.

[0199] In the embodiment of the present application, the width information corresponding to the first region image may be preset, or may be determined according to the length and / or width of the first region image. For example, the width information corresponding to the first region image may be obtained by multiplying the length of the first region image by a preset coefficient.

[0200] Step S442: Generate a binary reference image based on the region parameters, the white point set and the width information.

[0201] When implementing this step, if the first region image is an elliptical ring, the drawellipse function in openCV can be used, and the region parameters, the white point set and the width information are used as the output of the function, and the output is a binary reference image. In the binary reference image, the pixels in the first region image are white points, and the pixels outside the first region image are black points.

[0202] Step S443: determining the coordinate information of each pixel point in the first region image based on the binary reference image.

[0203] Here, the white pixel points in the binary reference image can be obtained by traversing each pixel point in the binary reference image. Since the white pixel points in the binary reference image are the pixel points in the first area image, the coordinate information of the white pixel points in the binary reference image is determined as the coordinate information of each pixel point in the first area image.

[0204] Step S444: determine the coordinate information of each pixel point as the position information of the first area image.

[0205] Through the above steps S441 to S444, a binary reference image including a complete first area image corresponding to the area parameters can be generated based on the set of white points in the first binary image, the area parameters and width information of the first area image, and then the position information of the first area image can be determined based on the binary reference image, thereby providing reference data for the subsequent fusion processing of the first binary image and the second binary image.

[0206] Based on the above embodiments, the present application further provides an image recognition method, which is applied to Figure 2B The network architecture shown, Figure 6 Another implementation flow diagram of the image recognition method provided in the embodiment of the present application is shown below in combination with Figure 6 The image recognition method provided in the embodiment of the present application is described.

[0207] Step S301: The terminal generates and outputs an identification code based on the received operation instruction for scanning by a scanning device.

[0208] Here, when step S301 is implemented, the terminal responds to the received operation instruction, obtains the identification code from the code issuing platform, and outputs the identification code on its own display device. The identification code can be a payment identification code, a door access identification code, a ride identification code, etc. In the embodiment of the present application, the identification code is taken as an example for explanation.

[0209] Step S302: The scanning device obtains the image to be recognized.

[0210] The scanning device is provided with a fill light device, and the fill light device is in a circular ring or an elliptical ring shape. The fill light device is always on after being powered on on the scanning device, and can also be started to fill light when an object is detected approaching. Therefore, the image to be recognized acquired by the scanning device includes a first area image and a second area image, and the brightness of the first area image is higher than that of the second area image. The first area image is a highlight area caused by abnormal exposure due to fill light performed by the fill light device.

[0211] Step S303: The scanning device determines a first binarization threshold, and performs binarization processing on the image to be identified based on the first binarization threshold to obtain a first binarized image.

[0212] Step S304: The scanning device determines a second binarization threshold, and performs binarization processing on the image to be identified based on the second binarization threshold to obtain a second binarized image.

[0213] The second binarization threshold is smaller than the first binarization threshold.

[0214] Step S305 : The scanning device determines the position information of the first region image based on the first binary image, and performs fusion processing on the first binary image and the second binary image based on the position information to obtain a target binary image.

[0215] It should be noted that the implementation process of the above steps S302 to S305 is the same as the implementation process of steps S201 to S204. In actual application, the implementation process of steps S201 to S204 may be referred to.

[0216] Step S306: The scanning device performs identification code positioning processing on the target binary image to obtain location information of the identification code.

[0217] The identification code is positioned, that is, the positioning corner points of the three corners of the identification code are determined. In implementation, the contour information in the target binary image can be determined based on the target binary image, and the three closest in area are determined based on the contour information, which are the positioning corner points of the identification code. Then, the largest angle of the triangle surrounded by the three corner points is determined as the point of the upper left corner of the identification code. Then, the lower left and upper right positions of the other two corner points are determined based on the angle difference between the two sides of this angle, so that the position of the lower right corner of the identification code can be determined. In the embodiment of the present application, the position information of the identification code can be represented by the coordinates of the two diagonal points of the upper left and lower right.

[0218] Step S307: The scanning device obtains the identification code image from the target binary image based on the position information of the identification code.

[0219] Since the identification code is generally a square or a rectangle, once the coordinate information of the two diagonal vertices of the identification code is determined, the identification code image can be determined from the target binary image based on the coordinate information of the two diagonal vertices.

[0220] Step S308: The scanning device recognizes the identification code image using a preset decoding algorithm to obtain a recognition result.

[0221] The identification result at least includes the user identification corresponding to the identification code, and may also include certificate data, message digest data and the generation time of the identification code.

[0222] Step S309: The scanning device sends the recognition result to the acquiring device.

[0223] Step S310: The acquiring device generates order information based on the recognition result and sends the order information to the payment system.

[0224] Here, after obtaining the recognition result, the acquiring device verifies the recognition result, for example, first determines whether the user identification exists. If so, it verifies the certificate data, message summary data and generation time of the identification code. If the verification passes, the acquiring device can generate order information based on the recognition result and payment amount, and send the order information to the payment system.

[0225] Step S311, the payment system sends the order information to the code issuing platform.

[0226] In an embodiment of the present application, the order information may include an order number, a payment amount, and a product identifier. After receiving the order information, the payment system first verifies the order information. Among them, the order information may be verified to determine whether the order number is repeated, and whether the product identifier and the payment amount correspond. If the order number is not repeated, and the product identifier and the payment amount correspond, then the order information verification is considered to have passed, and the payment system will send the order information to the code issuing platform to notify the code issuing platform to deduct the payment.

[0227] Step S312: the code issuing platform deducts payment based on the order information.

[0228] Here, the order information includes not only the order number, payment amount and product identification, but also the payment identification code information. The payment identification code information includes at least the transaction authorization cryptogram (TAC). When the code issuing platform deducts money based on the order information, it needs to verify the order number and the transaction authentication code. During implementation, the code issuing platform can determine whether there is an order number that is the same as the order number; if there is no order number that is the same as the order number, the code issuing platform will further verify the transaction authentication code.

[0229] The TAC key is generated by the code issuing platform. When generating the TAC password, it can be based on but not limited to the certificate validity time, certificate expiration time, QR code generation time and user identification information. TAC is a dynamic key unknown to the merchant, which can prevent the merchant from forging a QR code based on the user's certificate data to initiate a deduction. If the order number and TAC information are verified, the code issuing platform will deduct the user account corresponding to the order information based on the payment amount.

[0230] Step S313: the code issuing platform sends the deduction information to the terminal and the payment system.

[0231] In some embodiments, after receiving the deduction information, the terminal will output the deduction information, and after receiving the deduction information, the payment system will perform reconciliation and fund settlement.

[0232] In the image recognition method provided in the embodiment of the present application, when the terminal needs to scan the code, it presents the generated payment identification code, and the scanning device obtains the image to be recognized. Since a fill light device is provided in the scanning device, the image to be recognized obtained by the scanning device includes a first area image whose brightness is significantly higher than that of other areas. Therefore, when the scanning device recognizes the payment code, it is necessary to first use a higher first binarization threshold to binarize the image to be recognized to obtain a first binarized image. The first binarized image retains the detail information in the highlighted first area image, while most of the pixels in the non-highlighted second area image are binarized into black points, and then a first binarization threshold is used to binarize the image to be recognized. The second binarization threshold of the threshold is used to binarize the image to be identified to obtain a second binarized image. In the second binarized image, most of the pixels in the first area image are binarized as white points, but the detail information in the non-highlighted second area image is retained. Therefore, the first binarized image and the second binarized image are fused to obtain a target binarized image, including the detail information of the first area image and the second area image, to ensure the integrity of the target binarized image, and then use the target binarized image to perform image recognition, which can improve the recognition accuracy. After that, the subsequent payment process is performed based on the recognition result of the target binarized image, which can improve the payment efficiency.

[0233] The following is an explanation of an exemplary application of an embodiment of the present application in a practical application scenario.

[0234] An embodiment of the present application provides an image recognition method, which can be applied to identification code recognition, and a scene where a fill light is provided on the identification code scanning device. In order to make the fill light uniform, a circular fill light is generally used. Since the reflection of the circular fill light on the display screen of the smart device is elliptical and the light diffusion is controllable, the circular area can be filtered out through preprocessing, and the ellipse fitting method of OpenCV is used to find the ellipse area. Then, different threshold algorithms are used for binarization of the ellipse area and the non-ellipse area respectively, and then the contents of the two parts are spliced ​​into a complete binary image to realize identification code recognition.

[0235] Figure 7 A schematic diagram of another implementation process of the image recognition method provided in the embodiment of the present application is shown in the figure, and the process includes:

[0236] Step S701 : performing a first binarization process on the color image to determine a bright circular ring region in the color image (corresponding to the first region image in other embodiments).

[0237] Based on prior knowledge: the QR code area in the initially collected color image has a large bright area with a white background, while the ring (white light) will be brighter. First, the color image is gray-scaled to remove useless information in the 3-channel color image and convert it into a single-channel image that preserves brightness information (grayscale value), that is, a grayscale image. Fig. 8A The grayscale image is obtained by graying the color image. Then, a binarization threshold for the first binarization process is determined (corresponding to the first binarization threshold in other embodiments), and the color image is binarized according to the binarization threshold to determine the bright circular area in the color image.

[0238] In this step, it is necessary to determine the binarization threshold that can extract the ring, that is, to find a value between the gray value of the white background in the middle of the screen and the gray value of the ring as the threshold for this binarization process. When implementing, you can first determine the gray value histogram of the color image, such as Figure 8B As shown. Figure 8B From the grayscale value histogram shown, it can be seen that in the high grayscale value (brightness) area, the grayscale value area of ​​the ring will have the first peak (near 220), and in the white background area there will be a second peak (140-180). In actual implementation, the grayscale value of 255 can be accumulated forward in a certain step length, the number of pixels in each step length can be added, and then the change in the number of pixels in adjacent steps can be compared. When the change in the number of pixels in adjacent steps increases sharply and the increase in the increase reaches the preset threshold, it is determined that the white background threshold is reached. At this time, the minimum grayscale value in the previous step length in the adjacent step length is taken as the binarization threshold for the first binarization process.

[0239] For example, the accumulation is performed with a step size of 5, first determining the total number of pixels in the step size of 251 to 255, then determining the total number of pixels in the step size of 246 to 250, and then determining the difference between the total number of pixels in these two step sizes. If the difference does not reach the difference threshold, continue to determine the total number of pixels in the step size of 241 to 245, and then determine the difference between the total number of pixels in the step size of 241 to 245 and the total number of pixels in the step size of 246 to 250. If the difference does not reach the difference threshold, repeat the above process. If the difference reaches the difference threshold, 241 is determined as the binarization threshold for the first binarization process.

[0240] pass Figure 8B The gray value histogram shown in FIG. 1 determines that the first binarization threshold is 195. Based on the binarization threshold, the Fig. 8AThe grayscale image shown is binarized to obtain Figure 8C The binarized image shown.

[0241] In some embodiments, the pixel points corresponding to the grayscale value of a certain step length may be taken at every certain grayscale value interval, for example, the total number of pixel points corresponding to the grayscale interval of 5 step length may be taken at every ten grayscale values ​​interval.

[0242] Step S702: fitting an ellipse using the bright circular area.

[0243] pass Figure 8C In the binarized image shown, it can be seen that in addition to the circular area, there are also many small noise points due to inaccurate binarization thresholds. In order to minimize the influence of the white points outside the circular area on the fitted ellipse, in the embodiment of the present application, Figure 8C The binarized image shown in the figure is processed by a median step to remove most of the noise. For example, if the filter size is 3, we get Fig.8D The median filtering result is shown.

[0244] In the embodiment of the present application, a set of white points in the median filtering result is obtained, and then the fitEllipse function provided by c++opencv (the function accepts a vector <point>The obtained white point set is used as the input of the fitEllipse function to obtain the fitted ellipse parameters. When traversing the median filter result image to obtain the white point set, the traversal can be performed across rows and columns because the number of image pixels is large. Because there are many points on the ring, this will not greatly affect the final fitted ellipse parameters, and can also improve the traversal speed and speed up the ellipse fitting speed.

[0245] Step S703, performing a second binarization process on the color image.

[0246] In this step, the original image can be directly processed by the OTSU binarization algorithm of opencv to perform a second binarization on the color image. Because the white brightness of the ring part is high, in this binarization process, the ring part will be binarized to white and the QR code part will be black. At this time, the following is obtained: Fig. 9 As shown, the black QR code except for the circle blocking it.

[0247] Step S704: using the results of the two binarization processes to determine the final binarization result of the color image.

[0248] like Figure 8C As shown, the first binarization result includes noise points, and the ring is missing (the missing point mainly affects the black area where the ring and the QR code overlap). In this step, the ring needs to be filled first.

[0249] In step S703, the fitted ellipse coefficients have been obtained. Through the ellipse method of OpenCV, the corresponding ellipse can be drawn on the black image with a certain line width based on the fitted ellipse coefficients to complete the filling of the ring. The line width of the ellipse can be adjusted according to the width and height of the ellipse, and finally the filled ellipse can cover the halo area in the image. At this time, Fig.10 Schematic diagram of the completed ellipse shown.

[0250] When determining the final binarization result of the color image based on the two binarization results, each pixel point on the second binarization result can be traversed, and then based on Fig.10 The complete ellipse shown in the figure determines whether the pixel point is within the circular ring area. If the pixel point is within the circular ring area, the pixel value of the pixel point in the second binarization result is replaced with the pixel value of the pixel point in the first binarization result. After traversing the second binarization result image, the final binarization result is obtained. The QR code area in the final binarization result is compared with Fig. 9 The QR code shown is more complete.

[0251] Step S705: decode the final binarization result of the color image using a decoding algorithm to obtain a recognition result.

[0252] In the image recognition method provided in the embodiment of the present application, the first binarization processing uses a higher binarization threshold, which is equivalent to binarizing the highlighted circular ring area, and other non-highlighted areas will be binarized to black, and the second binarization processing uses a lower binarization threshold, which is equivalent to binarizing other areas except the circular ring area, and the highlighted circular ring area will be binarized to white. Finally, the circular ring range is determined by opencv, and the binary images obtained by the two binarization processing are fused, so that the fused binary image includes detailed information of the highlighted area and the non-highlighted area, which can avoid the inability to decode after global binarization due to the high brightness of the circular ring. This solution can effectively solve the problem of insufficient code scanning recognition performance in the product circular ring fill light scenario.

[0253] It is understandable that in the embodiments of the present application, the content involving user information, such as two-dimensional identification codes and other related data, when the embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0254] The following is a description of an exemplary structure of the image recognition device 443 provided in the embodiment of the present application implemented as a software module. In some embodiments, Figure 3 As shown, the software modules stored in the image recognition device 443 of the memory 440 may include:

[0255] A first acquisition module 4431 is used to acquire an image to be recognized, wherein the image to be recognized includes a first region image, and the brightness of the first region image is higher than the brightness of the second region image;

[0256] A first binarization module 4432 is used to determine a first binarization threshold, and perform binarization processing on the image to be identified based on the first binarization threshold to obtain a first binarized image;

[0257] A second binarization module 4433 is used to determine a second binarization threshold, and perform binarization processing on the image to be identified based on the second binarization threshold to obtain a second binarized image, wherein the second binarization threshold is less than the first binarization threshold;

[0258] An image fusion module 4434 is used to determine the position information of the first region image based on the first binarized image, and fuse the first binarized image with the second binarized image based on the position information to obtain a target binarized image;

[0259] The image recognition module 4435 is used to recognize the target binary image and obtain a recognition result.

[0260] In some embodiments, the first binarization module 4432 is further used to:

[0261] When the image to be identified is a color image, grayscale processing is performed on the image to be identified to obtain a grayscale image;

[0262] Based on the grayscale image, determining the maximum grayscale value, the minimum grayscale value and the number of pixels corresponding to each grayscale value;

[0263] Determine a plurality of target grayscale intervals based on the maximum grayscale value and the minimum grayscale value;

[0264] Determine the total number of pixels corresponding to each target grayscale interval based on the number of pixels corresponding to each grayscale value;

[0265] A first binarization threshold is determined based on the total number of pixels corresponding to each target grayscale interval.

[0266] In some embodiments, the first binarization module 4432 is further used to:

[0267] Get the number of grayscale intervals used to divide the grayscale interval;

[0268] Dividing a plurality of grayscale intervals from the maximum grayscale value to the minimum grayscale value according to the number of grayscale intervals;

[0269] determining the grayscale interval as the target grayscale interval; or,

[0270] A plurality of target grayscale intervals are selected from the plurality of grayscale intervals according to preset interval intervals.

[0271] In some embodiments, the first binarization module 4432 is further used to:

[0272] Sorting the target grayscale intervals from large to small according to the maximum grayscale value of the interval to obtain sorted target grayscale intervals;

[0273] Based on the sorted target grayscale intervals, determining an i-th difference between the total number of pixels corresponding to the i+1-th target grayscale interval and the total number of pixels corresponding to the i-th target grayscale interval;

[0274] When the i-th difference is greater than a preset difference threshold, the minimum grayscale value corresponding to the i-th target grayscale interval is determined as a first binarization threshold, where i is an integer between 1 and (N-1), and N is the total number of target grayscale intervals.

[0275] In some embodiments, the second binarization module 4433 is further used to:

[0276] Determining each grayscale value from the minimum grayscale value to the maximum grayscale value as a candidate threshold value in sequence;

[0277] Determine a plurality of foreground pixels and a plurality of background pixels corresponding to each candidate threshold;

[0278] Determine the inter-class variance corresponding to each candidate threshold based on the grayscale value corresponding to each foreground pixel and the grayscale value corresponding to each background pixel;

[0279] The candidate threshold corresponding to the maximum inter-class variance is determined as the second binarization threshold.

[0280] In some embodiments, the second binarization module 4433 is further used to:

[0281] Determine the foreground pixel ratio and the background pixel ratio based on the total number of foreground pixels and the total number of background pixels.

[0282] Determine a foreground grayscale average value based on the grayscale value corresponding to each foreground pixel and the total number of foreground pixels;

[0283] Determine the background grayscale average value based on the grayscale value corresponding to each background pixel and the total number of background pixels;

[0284] Determining an image grayscale average value of the grayscale image;

[0285] The inter-class variance corresponding to the candidate threshold is determined based on the image average, the foreground pixel ratio, the background pixel ratio, the foreground grayscale average, and the background grayscale average.

[0286] In some embodiments, the image fusion module 4434 is further used to:

[0287] Performing filtering on the first binary image to obtain a filtered binary image;

[0288] Obtaining a set of white points in the filtered binary image;

[0289] Fitting the region parameters corresponding to the first region image based on the white point set;

[0290] Position information of the first area image is determined based on the area parameter and the white point set.

[0291] In some embodiments, the image fusion module 4434 is further used to:

[0292] Obtaining width information corresponding to the first area image;

[0293] generating a binary reference image based on the region parameter, the white point set and the width information, wherein the pixel points in the first region image in the binary reference image are white points, and the pixel points outside the first region image are black points;

[0294] Determine the coordinate information of each pixel point in the first area image based on the binary reference image;

[0295] The coordinate information of each pixel point is determined as the position information of the first area image.

[0296] In some embodiments, the image fusion module 4434 is further used to:

[0297] Obtaining coordinate information of the j-th pixel point in the second binary image, where j=1, 2, ..., M, and M is the total number of pixels in the second binary image;

[0298] When it is determined that the jth pixel is located in the first regional image based on the coordinate information of the jth pixel and the binary reference image, obtaining a first pixel value of the jth pixel in the first binary image;

[0299] Replacing the pixel value of the j-th pixel in the second binary image with the first pixel value;

[0300] When it is determined that the j-th pixel point is outside the first regional image, the pixel value of the j-th pixel point is kept unchanged,

[0301] The processed second binarized image is determined as a target binarized image.

[0302] In some embodiments, when the image to be identified includes an identification code to be identified, the image identification module 4435 is further used to:

[0303] Performing identification code positioning processing on the target binary image to obtain position information of the identification code;

[0304] Acquire an identification code image from the target binary image based on the position information of the identification code;

[0305] The identification code image is identified using a preset decoding algorithm to obtain an identification result.

[0306] It should be noted that the description of the image recognition device in the embodiment of the present application is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the embodiment of the device, please refer to the description of the method embodiment of the present application for understanding.

[0307] The embodiment of the present application provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device performs the image recognition method described in the embodiment of the present application.

[0308] The present application embodiment provides a computer-readable storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will execute the image recognition method provided by the present application embodiment, for example, Figure 4 , Figure 5 and Figure 6 An image recognition method is shown.

[0309] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface storage, optical disk, or CD-ROM; or it may be various devices including one or any combination of the above memories.

[0310] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0311] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0312] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0313] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.< / point>

Claims

1. An image recognition method, characterized in that: The method comprises: Acquire an image to be identified, wherein the image to be identified includes a first region image and a second region image, and the brightness of the first region image is higher than the brightness of the second region image; determining a first binarization threshold, where the first binarization threshold is used to distinguish between the first region image and the second region image; Performing binarization processing on the image to be identified based on the first binarization threshold to obtain a first binarized image; Determine a second binarization threshold, where the second binarization threshold is used to distinguish between a foreground area and a background area in the image to be identified, and the second binarization threshold is less than the first binarization threshold; Performing binarization processing on the image to be identified based on the second binarization threshold to obtain a second binarized image; Determining the position information of the first region image based on the first binarized image, and fusing the first binarized image with the second binarized image based on the position information to obtain a target binarized image; The target binary image is recognized to obtain a recognition result.

2. The method according to claim 1, characterized in that Determining the first binarization threshold comprises: When the image to be identified is a color image, grayscale processing is performed on the image to be identified to obtain a grayscale image; Based on the grayscale image, determining the maximum grayscale value, the minimum grayscale value and the number of pixels corresponding to each grayscale value; Determine a plurality of target grayscale intervals based on the maximum grayscale value and the minimum grayscale value; Determine the total number of pixels corresponding to each target grayscale interval based on the number of pixels corresponding to each grayscale value; A first binarization threshold is determined based on the total number of pixels corresponding to each target grayscale interval.

3. The method according to claim 2, characterized in that The determining of a plurality of target grayscale intervals based on the maximum grayscale value and the minimum grayscale value comprises: Get the number of grayscale intervals used to divide the grayscale interval; Dividing a plurality of grayscale intervals from the maximum grayscale value to the minimum grayscale value according to the number of grayscale intervals; determining the grayscale interval as the target grayscale interval; or, A plurality of target grayscale intervals are selected from the plurality of grayscale intervals according to preset interval intervals.

4. The method according to claim 2, characterized in that The determining of the first binarization threshold based on the total number of pixels corresponding to each target grayscale interval includes: Sorting the target grayscale intervals from large to small according to the maximum grayscale value of the interval to obtain sorted target grayscale intervals; Based on the sorted target grayscale intervals, determining an i-th difference between the total number of pixels corresponding to the i+1-th target grayscale interval and the total number of pixels corresponding to the i-th target grayscale interval; When the i-th difference is greater than a preset difference threshold, the minimum grayscale value corresponding to the i-th target grayscale interval is determined as a first binarization threshold, where i is an integer between 1 and (N-1), and N is the total number of target grayscale intervals.

5. The method according to claim 2, characterized in that Determining the second binarization threshold comprises: Determining each grayscale value from the minimum grayscale value to the maximum grayscale value as a candidate threshold value in sequence; Determine a plurality of foreground pixels and a plurality of background pixels corresponding to each candidate threshold; Determine the inter-class variance corresponding to each candidate threshold based on the grayscale value corresponding to each foreground pixel and the grayscale value corresponding to each background pixel; The candidate threshold corresponding to the maximum inter-class variance is determined as the second binarization threshold.

6. The method according to claim 5, characterized in that The determining of the inter-class variance corresponding to each candidate threshold based on the grayscale value corresponding to each foreground pixel and the grayscale value corresponding to each background pixel includes: Determine the foreground pixel ratio and the background pixel ratio based on the total number of foreground pixels and the total number of background pixels. Determine a foreground grayscale average value based on the grayscale value corresponding to each foreground pixel and the total number of foreground pixels; Determine the background grayscale average value based on the grayscale value corresponding to each background pixel and the total number of background pixels; Determining an image grayscale average value of the grayscale image; The inter-class variance corresponding to the candidate threshold is determined based on the image grayscale average, the foreground pixel ratio, the background pixel ratio, the foreground grayscale average and the background grayscale average.

7. The method according to claim 1, characterized in that The determining the position information of the first region image based on the first binarized image includes: Performing filtering on the first binary image to obtain a filtered binary image; Obtaining a set of white points in the filtered binary image; Fitting the region parameters corresponding to the first region image based on the white point set; Position information of the first area image is determined based on the area parameter and the white point set.

8. The method according to claim 7, characterized in that The determining the position information of the first area image based on the area parameter and the white point set includes: Obtaining width information corresponding to the first area image; generating a binary reference image based on the region parameter, the white point set and the width information, wherein the pixel points in the first region image in the binary reference image are white points, and the pixel points outside the first region image are black points; Determine the coordinate information of each pixel point in the first area image based on the binary reference image; The coordinate information of each pixel point is determined as the position information of the first area image.

9. The method according to claim 8, characterized in that The fusing the first binary image and the second binary image based on the position information to obtain a target binary image comprises: Obtaining coordinate information of the j-th pixel in the second binary image, where j=1, 2, ..., M, and M is the total number of pixels in the second binary image; When it is determined that the jth pixel is located in the first regional image based on the coordinate information of the jth pixel and the binary reference image, obtaining a first pixel value of the jth pixel in the first binary image; Replacing the pixel value of the j-th pixel in the second binary image with the first pixel value; When it is determined that the j-th pixel point is outside the first regional image, the pixel value of the j-th pixel point is kept unchanged, The processed second binarized image is determined as a target binarized image.

10. The method according to any one of claims 1 to 9, characterized in that: When the image to be recognized includes an identification code to be recognized, recognizing the target binary image to obtain a recognition result includes: Performing identification code positioning processing on the target binary image to obtain position information of the identification code; Acquire an identification code image from the target binary image based on the position information of the identification code; The identification code image is identified using a preset decoding algorithm to obtain an identification result.

11. An image recognition device, characterized in that: The device comprises: A first acquisition module is used to acquire an image to be identified, wherein the image to be identified includes a first region image and a second region image, and the brightness of the first region image is higher than the brightness of the second region image; A first binarization module is used to determine a first binarization threshold, where the first binarization threshold is used to distinguish the first region image from the second region image; and binarize the image to be identified based on the first binarization threshold to obtain a first binarized image; A second binarization module is used to determine a second binarization threshold, where the second binarization threshold is used to distinguish between a foreground area and a background area in the image to be identified, and the second binarization threshold is less than the first binarization threshold; binarization is performed on the image to be identified based on the second binarization threshold to obtain a second binarized image; An image fusion module, used for determining the position information of the first region image based on the first binarized image, and fusing the first binarized image with the second binarized image based on the position information to obtain a target binarized image; The image recognition module is used to recognize the target binary image and obtain a recognition result.

12. An electronic device, characterized in that: The electronic device comprises: A memory for storing executable instructions; A processor, configured to implement the image recognition method according to any one of claims 1 to 10 when executing the executable instructions stored in the memory.

13. A computer-readable storage medium storing executable instructions, characterized in that: When the executable instructions are executed by a processor, the image recognition method according to any one of claims 1 to 10 is implemented.

14. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the image recognition method according to any one of claims 1 to 10 is implemented.

Images