A method for image recognition, related device, equipment and storage medium

Abstract

The present application discloses a method for image recognition, and the application scenarios include at least various types of terminal devices, such as: code scanning terminals, mobile phones, vehicle-mounted terminals, etc. The present application includes: obtaining the average object distance for the object to be identified; if the average object distance is less than or equal to the object distance threshold, then obtaining the target infrared image, the target infrared image includes the image corresponding to the object to be identified, and each pixel in the target infrared image corresponds to a brightness value; according to the brightness values ​​of multiple pixels in the target infrared image, the average image brightness of the target infrared image is determined; according to the size relationship between the image average brightness and the image brightness threshold, the category recognition result of the object to be identified is determined, and the image brightness threshold is determined based on the average object distance. The present application also provides an apparatus, a device and a storage medium. The present application solves the problem of insufficient robustness of traditional detection algorithms under complex light, thereby improving the accuracy of image recognition.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image recognition method, related apparatus, device and storage medium. Background Art

[0002] Under the in-depth influence of mobile Internet, the development trend of code scanning terminals is becoming more and more intelligent, and code scanning terminals integrating multiple functions are becoming more popular. For code scanning terminals integrating multiple functions (for example, palm payment and code scanning payment), it is necessary to determine the type of object (for example, palm or QR code) after the object approaches, so as to perform different mode processing.

[0003] In order to ensure that the barcode scanning terminal can quickly respond to different barcode scanning capabilities, the color sensor can collect color images in real time. At present, the object category is mainly identified through the collected color images, that is, the collected color images are algorithmically identified. For example, if the object in the image is judged to be a palm, the palm swiping mode is entered; if the object in the image is judged to be a mobile phone, the barcode scanning mode is entered.

[0004] However, the inventors have found that there are at least the following problems in the existing solutions: since the collected color images are easily affected by object movement, overly bright environment, overly dark environment or backlighting, the algorithm that relies on color images for object recognition has poor robustness and low image recognition accuracy. Summary of the invention

[0005] The embodiments of the present application provide a method, a related device, a device and a storage medium for image recognition. The present application solves the problem that the traditional detection algorithm is not robust enough under complex light conditions, thereby improving the accuracy of image recognition.

[0006] In view of this, the present application provides, on one hand, a method for image recognition, comprising:

[0007] Obtaining an average object distance for an object to be identified;

[0008] If the object average distance is less than or equal to the object distance threshold, a target infrared image is acquired, wherein the target infrared image includes an image corresponding to the object to be identified, and each pixel in the target infrared image corresponds to a brightness value;

[0009] According to the brightness values ​​of multiple pixels in the target infrared image, the average brightness of the image corresponding to the target infrared image is determined;

[0010] The category recognition result of the object to be recognized is determined according to the magnitude relationship between the average brightness of the image and the image brightness threshold, wherein the image brightness threshold is determined based on the average distance of the object.

[0011] Another aspect of the present application provides an image recognition device, comprising:

[0012] An acquisition module, used for acquiring an average object distance for an object to be identified;

[0013] The acquisition module is further used to acquire a target infrared image if the average object distance is less than or equal to the object distance threshold, wherein the target infrared image includes an image corresponding to the object to be identified, and each pixel in the target infrared image corresponds to a brightness value;

[0014] A determination module, used to determine the average brightness of the image corresponding to the target infrared image according to the brightness values ​​of multiple pixels in the target infrared image;

[0015] The determination module is further used to determine the category recognition result of the object to be recognized according to the size relationship between the average brightness of the image and the image brightness threshold, wherein the image brightness threshold is determined based on the average distance of the object.

[0016] In one possible design, in another implementation of another aspect of the embodiment of the present application,

[0017] An acquisition module, specifically used to collect distance data through a distance sensor;

[0018] If the distance data is within the object detection range, the distance data is taken as the average distance of the object.

[0019] In one possible design, in another implementation of another aspect of the embodiment of the present application,

[0020] An acquisition module, specifically configured to collect Q distance data through Q distance sensors, wherein Q is an integer greater than 1;

[0021] If the Q distance data are all within the object detection range, the Q distance data are averaged to obtain the average distance of the object;

[0022] If at least one of the Q distance data is not within the object detection range, the remaining distance data of the Q distance data are averaged to obtain an average distance of the object.

[0023] In one possible design, in another implementation of another aspect of the embodiment of the present application,

[0024] A determination module is specifically used to sum the brightness values ​​of each pixel in the target infrared image to obtain a first brightness sum;

[0025] According to the total number of pixels of the target infrared image, the first brightness sum is averaged to obtain the average brightness of the image, wherein the target infrared image includes W×H pixels, W represents the number of width pixels of the target infrared image, H represents the number of height pixels of the target infrared image, and H and W are both integers greater than 1.

[0026] In one possible design, in another implementation of another aspect of the embodiment of the present application,

[0027] A determination module, specifically used to obtain an infrared sub-image from a target infrared image, wherein the target infrared image includes W×H pixels, W represents the number of pixels of the width of the target infrared image, H represents the number of pixels of the height of the target infrared image, and H and W are both integers greater than 1;

[0028] The brightness value of each pixel in the infrared sub-image is summed to obtain a second brightness sum;

[0029] According to the total number of pixels of the infrared sub-image, the second brightness sum is averaged to obtain the average brightness of the image, wherein the infrared sub-image includes w×h pixels, w represents the number of width pixels of the infrared sub-image, h represents the number of height pixels of the infrared sub-image, h is an integer greater than 1 and less than H, and w is an integer greater than 1 and less than W.

[0030] In one possible design, in another implementation of another aspect of the embodiment of the present application,

[0031] The acquisition module is also used to calculate the image brightness threshold by a high-order polynomial function based on the average distance of the object before determining the category recognition result of the object to be identified according to the size relationship between the average brightness of the image and the image brightness threshold, wherein the high-order polynomial function includes (M+1) coefficients, and M is an integer greater than or equal to 1.

[0032] or,

[0033] The acquisition module is also used to obtain the image brightness threshold through a brightness prediction model based on the average distance of the object.

[0034] In one possible design, in another implementation of another aspect of the embodiment of the present application, the image recognition device further includes a construction module and a generation module;

[0035] A construction module, configured to construct a first brightness curve for a first object, wherein the first brightness curve represents a relationship between each average distance and a first average brightness;

[0036] The construction module is further used to construct a second brightness curve for the second object, wherein the second brightness curve represents the relationship between each average distance and the second average brightness;

[0037] The construction module is further used to construct a brightness separation curve according to the first brightness curve and the second brightness curve, wherein the brightness separation curve represents the relationship between each average distance and the brightness calibration threshold;

[0038] A generation module is used to construct a high-order polynomial function according to the brightness separation curve, or to train a brightness prediction model according to the brightness separation curve.

[0039] In one possible design, in another implementation of another aspect of the embodiment of the present application,

[0040] A construction module is specifically used to collect a first sample image set corresponding to K average distances through an infrared camera, wherein the first sample image set includes a first sample image corresponding to each average distance, the first sample image includes an image corresponding to the first object, and each pixel in the first sample image corresponds to a brightness value, and K is an integer greater than 1;

[0041] For each first sample image in the first sample image set, determining a first average brightness corresponding to the first sample image according to brightness values ​​of a plurality of pixels in the first sample image;

[0042] A first brightness curve for the first object is constructed according to the first average brightness and the average distance corresponding to each first sample image.

[0043] In one possible design, in another implementation of another aspect of the embodiment of the present application,

[0044] A construction module is specifically used to collect a second sample image set corresponding to T average distances through an infrared camera, wherein the second sample image set includes a second sample image corresponding to each average distance, the second sample image includes an image corresponding to the second object, and each pixel in the second sample image corresponds to a brightness value, and T is an integer greater than 1;

[0045] For each second sample image in the second sample image set, determining a second average brightness corresponding to the second sample image according to brightness values ​​of a plurality of pixels in the second sample image;

[0046] A second brightness curve for the second object is constructed according to the second average brightness and the average distance corresponding to each second sample image.

[0047] In one possible design, in another implementation of another aspect of the embodiment of the present application,

[0048] A construction module is specifically used to sample X average distances, where X is an integer greater than 1;

[0049] For each average distance among the X average distances, determine a first average brightness corresponding to the average distance according to the first brightness curve, and determine a second average brightness corresponding to the average distance according to the second brightness curve;

[0050] For each average distance of the X average distances, the first average brightness and the second average brightness are averaged to obtain a brightness calibration threshold;

[0051] According to the brightness calibration threshold corresponding to each average distance, a brightness separation curve is constructed.

[0052] In one possible design, in another implementation of another aspect of the embodiment of the present application,

[0053] A generation module, specifically used to obtain N groups of sample parameters according to the brightness separation curve sampling, wherein each group of sample parameters includes an average distance and a brightness calibration threshold, and N is an integer greater than or equal to (M+1);

[0054] Constructing N correlation functions according to N groups of sample parameters, wherein each correlation function is constructed based on a group of sample parameters;

[0055] (M+1) coefficients are calculated based on the N correlation functions, and a high-order polynomial function is generated.

[0056] In one possible design, in another implementation of another aspect of the embodiment of the present application,

[0057] A generation module, specifically used to obtain R groups of sample parameters according to the brightness separation curve, wherein each group of sample parameters includes an average distance and a brightness calibration threshold, and R is an integer greater than 1;

[0058] For each group of sample parameters in the R groups of sample parameters, based on the average distance included in the sample parameters, a brightness prediction threshold is obtained through a brightness prediction model;

[0059] Determine the target loss according to R brightness prediction thresholds and R brightness prediction thresholds, wherein the R brightness prediction thresholds are derived from R groups of sample parameters, and the R brightness prediction thresholds are obtained based on the average distance prediction included in each group of sample parameters in the R groups of sample parameters;

[0060] The target loss is used to update the model parameters of the brightness prediction model.

[0061] In one possible design, in another implementation of another aspect of the embodiment of the present application,

[0062] A determination module, specifically configured to determine that the category recognition result of the object to be recognized is a palm if the average brightness of the image is greater than the image brightness threshold;

[0063] If the average brightness of the image is less than or equal to the image brightness threshold, the category recognition result of the object to be recognized is determined to be a terminal device.

[0064] In one possible design, in another implementation of another aspect of the embodiment of the present application, the image recognition device further includes a verification module;

[0065] The acquisition module is further used to start the fill light if the category recognition result is a palm, and to collect a first image through a color camera, wherein the first image includes an image corresponding to the palm print;

[0066] A verification module, used for verifying the palm print included in the first image;

[0067] or,

[0068] The acquisition module is further configured to turn off the fill light and capture a second image through a color camera if the category recognition result is a terminal device, wherein the second image includes an image corresponding to the payment code;

[0069] The verification module is further used to verify the payment code included in the second image.

[0070] Another aspect of the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned methods when executing the computer program.

[0071] Another aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the above-mentioned methods are implemented.

[0072] Another aspect of the present application provides a computer program product, including a computer program, which implements the above-mentioned methods when executed by a processor.

[0073] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0074] In an embodiment of the present application, a method for image recognition is provided, which can obtain the average object distance for the object to be recognized. If the average object distance is less than or equal to the object distance threshold, a target infrared image is obtained, and the target infrared image includes the image corresponding to the object to be recognized. Based on this, on the one hand, the image brightness threshold is determined according to the average object distance. On the other hand, the image average brightness corresponding to the target infrared image is determined according to the brightness values ​​of multiple pixels in the target infrared image. Therefore, the category recognition result of the object to be recognized can be determined according to the size relationship between the image average brightness and the image brightness threshold. In the above manner, since different objects will present different reflective performances and imaging effects under the same distance of infrared fill light, therefore, after calculating the average image brightness of the target infrared image, the size relationship between the image brightness threshold and the image average brightness can be used to quickly distinguish the objects to be recognized included in the target infrared image, solving the problem of insufficient robustness of traditional detection algorithms under complex light, thereby improving the accuracy of image recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 A schematic diagram of an application environment of the image recognition system in an embodiment of the present application;

[0076] Figure 2 A schematic diagram of mirror reflection in an embodiment of the present application;

[0077] Figure 3 A schematic diagram of diffuse reflection in an embodiment of the present application;

[0078] Figure 4 A schematic diagram of a flow chart of an image recognition method in an embodiment of the present application;

[0079] Figure 5 A schematic diagram of a single distance sensor provided in an embodiment of the present application;

[0080] Figure 6 A schematic diagram of a plurality of distance sensors provided in an embodiment of the present application;

[0081] Figure 7 A schematic diagram of an infrared image of a target in an embodiment of the present application;

[0082] Figure 8 A schematic diagram of extracting an infrared sub-image from a target infrared image in an embodiment of the present application;

[0083] Fig. 9 This is a schematic diagram of an infrared imaging effect of a mobile phone under a fill light in an embodiment of the present application;

[0084] Fig.10 This is a schematic diagram of an infrared imaging effect of a palm under a fill light in an embodiment of the present application;

[0085] Fig.11 A schematic diagram showing a comparison between a mobile phone infrared image and a palm infrared image in an embodiment of the present application;

[0086] Fig.12 A schematic diagram of a first brightness curve and a second brightness curve in an embodiment of the present application;

[0087] Fig.13 A schematic diagram of a brightness separation curve in an embodiment of the present application;

[0088] Fig.14 A schematic diagram of a flow chart for determining the corresponding category recognition result of an object in an embodiment of the present application;

[0089] Fig.15 This is a schematic diagram of a QR code scanning payment scenario in an embodiment of the present application;

[0090] Fig.16 A schematic diagram of an image recognition device in an embodiment of the present application;

[0091] Fig.17 A schematic diagram of the structure of a terminal device in an embodiment of the present application;

[0092] Fig.18 A structural diagram of a server in an embodiment of the present application. DETAILED DESCRIPTION

[0093] The embodiments of the present application provide a method, a related device, a device and a storage medium for image recognition. The present application solves the problem that the traditional detection algorithm is not robust enough under complex light conditions, thereby improving the accuracy of image recognition.

[0094] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "corresponding to" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0095] Under the in-depth influence of mobile Internet, the development trend of code scanning terminals is becoming more and more intelligent, and code scanning terminals integrating multiple functions are becoming more popular. For example, in the fusion scenario of palm swiping payment and code scanning payment, when the object approaches, the code scanning terminal needs to quickly determine whether it is a palm or a mobile phone, so as to perform different mode processing. For example, in the fusion scenario of face swiping customs clearance and code scanning customs clearance, when the object approaches, the code scanning terminal needs to quickly determine whether it is a face or a mobile phone, so as to perform different mode processing. For example, in the fusion scenario of swiping paper QR code points and scanning electronic QR code points, when the object approaches, the code scanning terminal needs to quickly determine whether it is paper or a mobile phone, so as to perform different mode processing.

[0096] In order to detect the category of objects more efficiently and accurately in the above scenario, this application proposes an image recognition method, which is applied to Figure 1 The image recognition system shown, as shown in the figure, includes a terminal device 110 and a server 120, and the client is deployed on the terminal device, wherein the client can be run on the terminal device in the form of a browser, or can be run on the terminal device in the form of an independent application (application, APP), etc., and the specific presentation form of the client is not limited here. The server involved in this application can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms. The terminal device can be a scanning terminal, a mobile phone, a tablet computer, a laptop, a PDA, a personal computer, a smart TV, a smart watch, a vehicle-mounted device, a wearable device, etc., but is not limited thereto. The terminal device and the server can be directly or indirectly connected by wired or wireless communication, and this application is not limited here. The number of servers and terminal devices is also not limited. The solution provided in this application can be completed independently by a terminal device, can be completed independently by a server, or can be completed by a terminal device and a server in cooperation. This application does not make any specific limitations on this.

[0097] Exemplarily, the solution provided by the present application can be completed independently by a terminal device. The terminal device analyzes the distance data in real time to determine whether an object is approaching. If the terminal device determines that an object is approaching, the infrared fill light can be triggered to work and the target infrared image can be collected. Based on this, the terminal device calculates the average image brightness of the target infrared image and determines the image brightness threshold based on the current distance data. Finally, the terminal device determines the category recognition result of the object based on the size relationship between the average image brightness and the image brightness threshold.

[0098] Exemplarily, the solution provided by the present application can be completed by the cooperation of a terminal device and a server. The terminal device analyzes the distance data in real time to determine whether an object is approaching. If the terminal device determines that an object is approaching, it can trigger the infrared fill light to work and collect the target infrared image. Based on this, the terminal device can feed back the target infrared image and the currently measured distance data to the server, and the server calculates the average image brightness of the target infrared image and determines the image brightness threshold based on the current distance data. Finally, the server determines the category recognition result of the object based on the size relationship between the average image brightness and the image brightness threshold, and then returns the category recognition result to the terminal device.

[0099] It is understandable that, taking the object category recognition result of "palm" or "mobile phone" as an example, since the surface of the mobile phone is a mirror reflection, the light directionality is good, and the concentration of reflected light is high. The surface of the palm is a diffuse reflection, the light directionality is poor, the light reflection concentration is weak, and there are fewer overexposed areas. Therefore, the surface of the mobile phone and the surface of the palm appear different under the infrared fill light, that is, at the same distance, the infrared image including the mobile phone and the infrared image including the palm have a more obvious difference in average brightness.

[0100] Since this application involves some terms related to professional fields, they will be explained below for ease of understanding.

[0101] (1) Color image: A color image captured by a color sensor (i.e., a color camera) using natural light. It is usually used for face or palm optimization and comparative recognition in face-swiping or palm payment.

[0102] (2) Infrared image: An infrared image captured by an infrared sensor (i.e., an infrared camera) that generates infrared light images. It is usually used for liveness detection in face-swiping or palm-swiping payment.

[0103] (3) Optimization: Select a group of color images and infrared images that meet the preconditions of the liveness detection and contrast recognition algorithm. For example, the color images are optimized based on the palm angle, palm size, palm centering and color image clarity. For another example, the infrared images are optimized based on the brightness of the infrared images.

[0104] (4) Proximity sensor (psensor): A sensor that detects the presence of a nearby object without any physical contact. A proximity sensor typically emits an electromagnetic field or a beam of electromagnetic radiation (e.g., infrared) and looks for changes in the field or return signal. The object being sensed is often called the target of the proximity sensor.

[0105] (5) Specular reflection: For ease of understanding, please refer to Figure 2 , Figure 2 This is a schematic diagram of mirror reflection in an embodiment of the present application. As shown in the figure, if the reflective surface is relatively smooth, when parallel incident light hits the reflective surface, it will still be reflected in parallel in one direction.

[0106] (6) Diffuse reflection: For ease of understanding, please refer to Figure 3 , Figure 3 This is a schematic diagram of diffuse reflection in an embodiment of the present application. As shown in the figure, when a beam of parallel incident light hits a rough surface, the surface will reflect the light in all directions. Therefore, although the incident rays are parallel to each other, the normal directions of each point are inconsistent, causing the reflected light to be reflected irregularly in different directions.

[0107] In combination with the above introduction, the image recognition method in this application will be introduced below. Figure 4 In the embodiment of the present application, the image recognition method can be executed by a computer device, which can be a terminal device (the present application takes the terminal device as a code scanning terminal as an example, but this should not be understood as a limitation of the present application) or a server. The image recognition method provided by the present application includes:

[0108] 210. Obtain an average object distance for the object to be identified;

[0109] In one or more embodiments, in the initialization state, the code scanning terminal does not determine what object the user will use to scan the code. Therefore, in order to ensure that the code scanning terminal can quickly respond to the code scanning capability, the color camera, infrared camera and distance sensor in the code scanning terminal are all in working state, so that the distance data, color image and infrared image of the object to be identified located directly above the device can be collected in real time. Based on this, the average object distance for the object to be identified can be determined according to the distance data measured in real time.

[0110] 220. If the object average distance is less than or equal to the object distance threshold, acquiring a target infrared image, wherein the target infrared image includes an image corresponding to the object to be identified, and each pixel in the target infrared image corresponds to a brightness value;

[0111] In one or more embodiments, if the average object distance is less than or equal to the object distance threshold, it means that the object to be identified is close enough, based on which a target infrared image can be acquired. The target infrared image is acquired by an infrared camera, and the target infrared image includes an image corresponding to the object to be identified, and each pixel in the target infrared image corresponds to a brightness value.

[0112] It should be noted that the value range of the brightness value can be set to 0 to 255, wherein the larger the brightness value of the pixel point, the closer the pixel point is to white, and conversely, the smaller the brightness value of the pixel point, the closer the pixel point is to black.

[0113] Specifically, it is assumed that several distance data are captured in the first time frame, and the average distance of the object is calculated based on these distance data. When it is determined that the average distance of the object is less than or equal to the object distance threshold, in one case, the infrared camera can be started to capture the infrared image corresponding to the second time frame in real time, and the infrared image is used as the target infrared image. Among them, the second time frame is usually the next time frame adjacent to the first time frame, which can make the average distance of the object and the target infrared image "aligned" in the time dimension to a certain extent, which is conducive to subsequent detection. At the same time, there is no need to save redundant infrared images, which can save the storage resources of the code scanning terminal. In another case, considering that the infrared camera can continuously shoot multiple frames of continuous infrared images, based on this, the infrared image corresponding to the first time frame can be extracted from the continuous image as the target infrared image, which can ensure that the average distance of the object and the target infrared image are "aligned" in the time dimension, which is conducive to improving the accuracy of subsequent detection.

[0114] 230. Determine an average brightness of an image corresponding to the target infrared image according to brightness values ​​of a plurality of pixels in the target infrared image;

[0115] In one or more embodiments, the brightness values ​​of multiple pixels in the target infrared image are averaged, thereby obtaining the average brightness of the image corresponding to the target infrared image. It can be seen that the value range of the average brightness of the image is consistent with the value range of the brightness value, and can also be set to 0 to 255. Since the target infrared image is a single-channel grayscale image, the target infrared image only has a brightness value (i.e., grayscale value), so the algorithm statistically calculates the brightness values ​​of all pixels or some pixels in the target infrared image and averages them to obtain the average brightness of the image.

[0116] 240. Determine a category recognition result of the object to be recognized based on a magnitude relationship between an average brightness of the image and an image brightness threshold, wherein the image brightness threshold is determined based on an average distance of the object.

[0117] In one or more embodiments, after determining the average distance of the object, the image brightness threshold corresponding to the average distance of the object can be calculated based on the established relationship between the distance and the brightness threshold. Thus, the category recognition result of the object to be recognized can be determined based on the size relationship between the average brightness of the image and the image brightness threshold.

[0118] In the embodiment of the present application, a method for image recognition is provided. In the above manner, since different objects will present different reflective performances and imaging effects under the same distance of infrared fill light, after calculating the average brightness of the target infrared image, the size relationship between the image brightness threshold and the image average brightness can be used to quickly distinguish the objects to be recognized included in the target infrared image, solving the problem of insufficient robustness of traditional detection algorithms under complex light, thereby improving the accuracy of image recognition.

[0119] Optionally, in the above Figure 4 On the basis of the corresponding embodiments, in another optional embodiment provided by the embodiment of the present application, obtaining the average object distance for the object to be identified may specifically include:

[0120] Collect distance data through distance sensors;

[0121] If the distance data is within the object detection range, the distance data is taken as the average distance of the object.

[0122] In one or more embodiments, a method for calculating the average distance of an object based on a single distance sensor is introduced. As can be seen from the aforementioned embodiments, a distance sensor can also be built into the code scanning terminal. When an object approaches the code scanning terminal, the distance sensor will detect the distance data from the object to the code scanning terminal. If the distance data is within the object detection range, the distance data is used as the average distance of the object.

[0123] Specifically, for ease of understanding, see Figure 5 , Figure 5 This is a schematic diagram of a single distance sensor in an embodiment of the present application. As shown in the figure, A1 is used to indicate the distance sensor, wherein the distance sensor can be set in the middle area of ​​the scanning screen. Generally, the distance sensor has a certain object detection range. Taking the object detection range of 0 to 50 centimeters (centimeter, cm) as an example, if the distance data detected by the distance sensor is within the range of 0 to 50 cm, the distance data can be directly used as the average distance of the object.

[0124] It should be noted that Figure 4 The arrangement position of the distance sensor shown is only a schematic diagram and should not be construed as a limitation to the present application.

[0125] Secondly, in the embodiment of the present application, a method for calculating the average distance of an object based on a single distance sensor is provided. Through the above method, the average distance of an object can be detected using a single distance sensor, thereby saving the production cost of the code scanning terminal to a certain extent.

[0126] Optionally, in the above Figure 4 On the basis of the corresponding embodiments, in another optional embodiment provided by the embodiment of the present application, obtaining the average object distance for the object to be identified may specifically include:

[0127] Collect Q distance data through Q distance sensors, where Q is an integer greater than 1;

[0128] If the Q distance data are all within the object detection range, the Q distance data are averaged to obtain the average distance of the object;

[0129] If at least one of the Q distance data is not within the object detection range, the remaining distance data of the Q distance data are averaged to obtain an average distance of the object.

[0130] In one or more embodiments, a method for calculating the average distance of an object based on multiple distance sensors is introduced. As can be seen from the aforementioned embodiments, Q distance sensors can also be built into the code scanning terminal, where Q is an integer greater than 1. When an object approaches the code scanning terminal, these distance sensors will respectively detect the distance data from the object to the code scanning terminal. Based on this, it is necessary to determine whether each distance data is within the object detection range, so as to calculate the average distance of the object.

[0131] Specifically, for ease of understanding, see Figure 6 , Figure 6 This is a schematic diagram of a plurality of distance sensors provided in an embodiment of the present application. As shown in the figure, B1 is used to indicate a first distance sensor, B2 is used to indicate a second distance sensor, B3 is used to indicate a third distance sensor, and B4 is used to indicate a fourth distance sensor. Among them, the four distance sensors can be respectively set at the positions of the four vertex areas of the scanning screen. Usually, each distance sensor has a certain object detection range. Taking the object detection range of each distance sensor as 0 to 50 cm as an example, if the distance data detected by the distance sensor is within the range of 0 to 50 cm, the distance data can be used to calculate the average distance of the object.

[0132] It should be noted that Figure 6 The location and number of the distance sensors shown are merely for reference only and should not be construed as limiting the present application.

[0133] Taking Q=4 as an example, if the four distance data are all within the object detection range, the four distance data are averaged to obtain the average distance of the object. For example, the distance data detected by the first distance sensor is "12", the distance data detected by the second distance sensor is "12", the distance data detected by the third distance sensor is "14", and the distance data detected by the fourth distance sensor is "14. It can be seen that the four distance data are all within the object detection range. Therefore, the four distance data are averaged to obtain an average distance of 13cm.

[0134] Taking Q=4 as an example, if at least one distance data is not within the object detection range, the remaining distance data are averaged to obtain the average distance of the object. For example, the distance data detected by the first distance sensor is "10", the distance data detected by the second distance sensor is "6", the distance data detected by the third distance sensor is "10000", and the distance data detected by the fourth distance sensor is "10000. It can be seen that 2 distance data are not within the object detection range, so the remaining two distance data are averaged to obtain an average distance of 8cm.

[0135] Secondly, in the embodiment of the present application, a method for calculating the average distance of an object based on multiple distance sensors is provided. In the above manner, multiple distance sensors are used to detect the distance data of the object respectively, and then the average distance of the object is calculated by combining these distance data. In this way, a more accurate and objective average distance of the object can be obtained, which is conducive to improving the reliability and accuracy of subsequent detection.

[0136] Optionally, in the above Figure 4 On the basis of the corresponding embodiments, in another optional embodiment provided by the embodiment of the present application, determining the average brightness of the image corresponding to the target infrared image according to the brightness values ​​of multiple pixels in the target infrared image may specifically include:

[0137] The brightness value of each pixel in the target infrared image is summed to obtain a first brightness sum;

[0138] According to the total number of pixels of the target infrared image, the first brightness sum is averaged to obtain the average brightness of the image, wherein the target infrared image includes W×H pixels, W represents the number of width pixels of the target infrared image, H represents the number of height pixels of the target infrared image, and H and W are both integers greater than 1.

[0139] In one or more embodiments, a method for calculating the average brightness of an image based on all pixels is introduced. As can be seen from the above embodiments, the target infrared image includes W×H pixels, each pixel has a brightness value, where W represents the number of pixels of the width of the target infrared image, and H represents the number of pixels of the height of the target infrared image.

[0140] Specifically, for ease of understanding, see Figure 7 , Figure 7 This is a schematic diagram of a target infrared image in an embodiment of the present application. As shown in the figure, the number of pixels of the width of the target infrared image is W, and the number of pixels of the height of the target infrared image is H. Based on this, the average brightness of the image corresponding to the target infrared image can be calculated in the following way:

[0141]

[0142] Where avglight represents the average brightness of the image. grey represents the brightness value of a pixel. W represents the number of pixels in the width of the target infrared image. H represents the number of pixels in the height of the target infrared image. x represents the xth pixel in the target infrared image. y represents the yth pixel in the target infrared image.

[0143] It should be noted that the image average brightness corresponding to the target infrared image may refer to the image average brightness of the target infrared image, or may refer to the image average brightness of a partial area (i.e., infrared sub-image) in the target infrared image. In this embodiment, the image average brightness corresponding to the target infrared image is the image average brightness of the target infrared image.

[0144] Secondly, in the embodiment of the present application, a method for calculating the average brightness of an image based on all pixels is provided. Through the above method, after the target infrared image is acquired, the average brightness of the image can be directly calculated based on the brightness value of each pixel in the target infrared image, without the need to perform image cropping and other processing on the target infrared image, thereby improving processing efficiency.

[0145] Optionally, in the above Figure 4 On the basis of the corresponding embodiments, in another optional embodiment provided by the embodiment of the present application, determining the average brightness of the image corresponding to the target infrared image according to the brightness values ​​of multiple pixels in the target infrared image may specifically include:

[0146] Acquire an infrared sub-image from the target infrared image, wherein the target infrared image includes W×H pixels, W represents the number of pixels of the width of the target infrared image, H represents the number of pixels of the height of the target infrared image, and H and W are both integers greater than 1;

[0147] The brightness value of each pixel in the infrared sub-image is summed to obtain a second brightness sum;

[0148] According to the total number of pixels of the infrared sub-image, the second brightness sum is averaged to obtain the average brightness of the image, wherein the infrared sub-image includes w×h pixels, w represents the number of width pixels of the infrared sub-image, h represents the number of height pixels of the infrared sub-image, h is an integer greater than 1 and less than H, and w is an integer greater than 1 and less than W.

[0149] In one or more embodiments, a method for calculating the average brightness of an image based on a portion of pixels is introduced. As can be seen from the above embodiments, the target infrared image includes W×H pixels, each pixel has a brightness value, where W represents the number of pixels of the width of the target infrared image, and H represents the number of pixels of the height of the target infrared image.

[0150] Specifically, for ease of understanding, see Figure 8 , Figure 8 This is a schematic diagram of extracting an infrared sub-image from a target infrared image in an embodiment of the present application. As shown in the figure, the number of pixels in the width of the target infrared image is W, and the number of pixels in the height of the target infrared image is H. Thus, the middle area can be extracted from the target infrared image as an infrared sub-image, where the number of pixels in the width of the infrared sub-image is w, and the number of pixels in the height of the infrared sub-image is h. Based on this, the average brightness of the image corresponding to the target infrared image can be calculated in the following way:

[0151]

[0152] Where avglight represents the average brightness of the image. grey represents the brightness value of a pixel. w represents the number of pixels in the width of the infrared sub-image. h represents the number of pixels in the height of the infrared sub-image. x represents the xth pixel in the infrared sub-image. y represents the yth pixel in the infrared sub-image.

[0153] For example, Figure 8 For example, the middle area of ​​the target infrared image is used as the infrared sub-image, that is, the height of the infrared sub-image is 1 / 4H to 3 / 4H, and the width of the infrared sub-image is the width W of the target infrared image. Based on this, the image average brightness of the infrared sub-image can be calculated as follows:

[0154]

[0155] It should be noted that the image average brightness corresponding to the target infrared image may refer to the image average brightness of the target infrared image, or may refer to the image average brightness of a partial area (i.e., infrared sub-image) in the target infrared image. In this embodiment, the image average brightness corresponding to the target infrared image is the image average brightness of the infrared sub-image.

[0156] Secondly, in the embodiment of the present application, a method for calculating the average brightness of an image based on some pixels is provided. In the above method, after the target infrared image is acquired, it is considered that the object to be identified is often in the middle area of ​​the image. Therefore, in order to avoid possible interference, in the actual calculation process, the middle area of ​​the image can also be selected as the infrared sub-image, so that the average brightness of the image calculated in this way has better reliability and accuracy.

[0157] Optionally, in the above Figure 4 On the basis of the corresponding embodiments, in another optional embodiment provided by the embodiment of the present application, before determining the category recognition result of the object to be recognized according to the magnitude relationship between the average brightness of the image and the image brightness threshold, the following may also be included:

[0158] Based on the average distance of the objects, the image brightness threshold is calculated by a high-order polynomial function, where the high-order polynomial function includes (M+1) coefficients, and M is an integer greater than or equal to 1.

[0159] or,

[0160] Based on the average distance of the object, the image brightness threshold is obtained through the brightness prediction model.

[0161] In one or more embodiments, two methods for determining the image brightness threshold are introduced. It can be seen from the above embodiments that light follows different reflection rules on planes on different media. For example, a mobile phone follows mirror reflection due to the glass plane, while a palm follows partial diffuse reflection due to the skin. The following will take "mobile phone" and "palm" as examples to illustrate the method of constructing the relationship between the average distance of an object and the image brightness threshold.

[0162] For ease of understanding, please refer to Fig. 9 , Fig. 9 This is a schematic diagram of an infrared imaging effect of a mobile phone under a fill light in an embodiment of the present application. Fig. 9 The middle (A) figure shows an infrared image including the "mobile phone screen" in the picture. Fig. 9 The middle (B) figure shows a color image of the "mobile phone screen" in the picture. It can be seen that due to the good directionality of the mirror reflection light, the infrared light in the collected image is prone to appear mostly dark, while some areas are concentrated and bright or overexposed. Therefore, the infrared imaging presents a light spot effect with a relatively concentrated brightness.

[0163] For ease of understanding, please refer to Fig.10 , Fig.10 This is a schematic diagram of an infrared imaging effect of a palm under a fill light in an embodiment of the present application. Fig.10 The middle figure (A) shows an infrared image including the "palm" in the picture. Fig.10 The middle (B) figure shows a color image including the "palm" in the picture. It can be seen that due to the directional dispersion of the mirror-reflected light, the infrared imaging presents a large range of light with relatively uniform brightness.

[0164] Specifically, for ease of understanding, see Fig.11 , Fig.11 This is a schematic diagram comparing the infrared image of a mobile phone and the infrared image of a palm in the embodiment of the present application. Infrared images of the "mobile phone screen" and the "palm" are collected at the same average object distance, wherein: Fig.11 The middle (A) figure shows an infrared image including the "mobile phone screen" in the picture. Fig.11 The middle (B) figure shows an infrared image including a "palm" in the picture. It can be seen that, under the same average object distance, if the corresponding relationship between the average object distance and the image brightness threshold is determined, the category of the object to be identified can be determined by determining the size between the image average brightness of the infrared image and the image brightness threshold.

[0165] Secondly, in the embodiment of the present application, two methods for determining the image brightness threshold are provided. Through the above method, one method is to substitute the average distance of the object into a high-order polynomial function, thereby calculating the image brightness threshold. This method has high computational efficiency and can quickly determine the category of the object to be identified. Another method is to input the average distance of the object into a brightness prediction model, and the image brightness threshold is predicted by the model. This method can utilize the learning ability of the neural network to predict a more reliable image brightness threshold. It can be seen that the above two methods can determine the image brightness threshold, thereby increasing the feasibility and flexibility of the solution.

[0166] Optionally, in the above Figure 4 On the basis of the corresponding embodiments, another optional embodiment provided by the embodiment of the present application may further include:

[0167] Constructing a first brightness curve for the first object, wherein the first brightness curve represents a relationship between each average distance and a first average brightness;

[0168] constructing a second brightness curve for the second object, wherein the second brightness curve represents a relationship between each average distance and a second average brightness;

[0169] Constructing a brightness separation curve according to the first brightness curve and the second brightness curve, wherein the brightness separation curve represents the relationship between each average distance and the brightness calibration threshold;

[0170] A high-order polynomial function is constructed based on the brightness separation curve, or a brightness prediction model is trained based on the brightness separation curve.

[0171] In one or more embodiments, a method for constructing a function or model based on a brightness separation curve is introduced. It can be seen from the aforementioned embodiments that the brightness curves of the corresponding objects are constructed respectively by utilizing the differences in the reflection effects of different objects under infrared fill light, thereby generating a brightness separation curve based on the brightness curves of different objects. Among them, the first brightness curve represents the relationship between each average distance and the first average brightness, and the second brightness curve represents the relationship between each average distance and the second average brightness. Based on this, the brightness separation curve can not only construct a high-order polynomial function, but also train a brightness prediction model.

[0172] Specifically, when the function convergence condition is met, a high-order polynomial function is determined, wherein the function convergence condition is usually that the loss is less than or equal to a preset threshold. When the model convergence condition is met, a brightness prediction model is trained, wherein the model convergence condition is usually that the loss is less than or equal to a preset threshold, or a preset number of model iterations is reached.

[0173] Again, in an embodiment of the present application, a method for constructing a function or model based on a brightness separation curve is provided. In the above manner, a high-order polynomial function can be constructed or a brightness prediction model can be trained by utilizing the difference in reflection effects of different objects under infrared fill light. In practical applications, a high-order polynomial function or a brightness prediction model can be selected according to needs to determine the image brightness threshold, thereby increasing the feasibility and operability of the solution.

[0174] Optionally, in the above Figure 4 On the basis of the corresponding embodiments, in another optional embodiment provided by the embodiment of the present application, constructing a first brightness curve for a first object may specifically include:

[0175] Collecting a first sample image set corresponding to K average distances by an infrared camera, wherein the first sample image set includes a first sample image corresponding to each average distance, the first sample image includes an image corresponding to the first object, and each pixel in the first sample image corresponds to a brightness value, and K is an integer greater than 1;

[0176] For each first sample image in the first sample image set, determining a first average brightness corresponding to the first sample image according to brightness values ​​of a plurality of pixels in the first sample image;

[0177] A first brightness curve for the first object is constructed according to the first average brightness and the average distance corresponding to each first sample image.

[0178] In one or more embodiments, a method for constructing a first brightness curve is introduced. As can be seen from the above embodiments, the first brightness curve represents the relationship between each average distance and the first average brightness. Therefore, it is necessary to collect corresponding infrared images (i.e., first sample images) at different average distances through an infrared camera, and these infrared images can be used as a first sample image set. It can be understood that the calculation method of the average distance is similar to the calculation method of the average distance of the object introduced in the above embodiments, so it will not be repeated here.

[0179] Specifically, take K as 15 as an example, and collect a first sample image every 1 cm of the average distance, thereby collecting 15 first sample images. Among them, each first sample image includes an image corresponding to the first object, for example, the first object can be a "palm". Therefore, the first average brightness corresponding to each first sample image is calculated separately, wherein the first average brightness can be calculated based on the brightness values ​​of all pixels in the first sample image, or based on the brightness values ​​of some pixels in the first sample image (for example, pixels in the middle area of ​​the image). Next, according to the average distance of each first sample image and its corresponding first average brightness, a curve is fitted as the first brightness curve of the first object. It can be seen that when constructing the first brightness curve, it is only necessary to place the first object directly above the scanning screen of the code scanning terminal at different distances, and perform infrared image acquisition and average brightness calibration.

[0180] For easier understanding, see Fig.12 , Fig.12 This is a schematic diagram of the first brightness curve and the second brightness curve in the embodiment of the present application. As shown in the figure, A1 is used to indicate the first brightness curve for the first object (e.g., a palm), and B1 is used to indicate the second brightness curve for the second object (e.g., a mobile phone). It can be seen that at different average distances, the average brightness of the first object and the second object has a good degree of distinction.

[0181] It should be noted that since the same barcode scanning terminal uses the same hardware devices and optical parameters, there is little difference in the final imaging effect. Based on this, a unified calibration set of parameters can be applied to a class of barcode scanning terminals.

[0182] Furthermore, in an embodiment of the present application, a method for constructing a first brightness curve is provided. In the above method, a series of infrared images of the first object are collected by an infrared camera to calculate the first average brightness corresponding to each infrared image. At the same time, the distance data of each infrared image is collected by a distance sensor to calculate the average distance corresponding to each infrared image. Thus, according to the corresponding relationship between the first average brightness and the average distance, a corresponding first brightness curve is constructed, and the first brightness curve conforms to the real change relationship between brightness and distance, which is conducive to improving the reliability of subsequent calculations.

[0183] Optionally, in the above Figure 4 On the basis of the corresponding embodiments, in another optional embodiment provided by the embodiment of the present application, constructing a second brightness curve for a second object may specifically include:

[0184] Collecting a second sample image set corresponding to T average distances by an infrared camera, wherein the second sample image set includes a second sample image corresponding to each average distance, the second sample image includes an image corresponding to the second object, and each pixel in the second sample image corresponds to a brightness value, and T is an integer greater than 1;

[0185] For each second sample image in the second sample image set, determining a second average brightness corresponding to the second sample image according to brightness values ​​of a plurality of pixels in the second sample image;

[0186] A second brightness curve for the second object is constructed according to the second average brightness and the average distance corresponding to each second sample image.

[0187] In one or more embodiments, a method for constructing a second brightness curve is introduced. As can be seen from the above embodiments, the second brightness curve represents the relationship between each average distance and the second average brightness. Therefore, it is necessary to collect corresponding infrared images (i.e., second sample images) at different average distances through an infrared camera, and these infrared images can be used as a second sample image set. It can be understood that the calculation method of the average distance is similar to the calculation method of the average distance of the object introduced in the above embodiments, so it will not be repeated here.

[0188] Specifically, take T as 15 as an example, and collect a second sample image every 1 cm of the average distance, thereby collecting 15 second sample images. Among them, each second sample image includes an image corresponding to the second object, for example, the second object can be a "mobile phone". Therefore, the second average brightness corresponding to each second sample image is calculated separately, wherein the second average brightness can be calculated based on the brightness values ​​of all pixels in the second sample image, or based on the brightness values ​​of some pixels in the second sample image (for example, pixels in the middle area of ​​the image). Next, according to the average distance of each second sample image and its corresponding second average brightness, a curve is fitted as the second brightness curve of the second object. It can be seen that when constructing the second brightness curve, it is only necessary to place the second object directly above the scanning screen of the code scanning terminal at different distances, and perform infrared image acquisition and average brightness calibration.

[0189] For ease of understanding, please refer again to Fig.12 , Fig.12 This is a schematic diagram of the first brightness curve and the second brightness curve in the embodiment of the present application. As shown in the figure, A1 is used to indicate the first brightness curve for the first object (e.g., a palm), and B1 is used to indicate the second brightness curve for the second object (e.g., a mobile phone). It can be seen that at different average distances, the average brightness of the first object and the second object has a good degree of distinction.

[0190] It should be noted that since the same barcode scanning terminal uses the same hardware devices and optical parameters, there is little difference in the final imaging effect. Based on this, a unified calibration set of parameters can be applied to a class of barcode scanning terminals.

[0191] Furthermore, in an embodiment of the present application, a method for constructing a second brightness curve is provided. In the above method, a series of infrared images of the second object are collected by the infrared camera to calculate the second average brightness corresponding to each infrared image. At the same time, the distance data of each infrared image is collected by the distance sensor to calculate the average distance corresponding to each infrared image. Thus, according to the corresponding relationship between the second average brightness and the average distance, a corresponding second brightness curve is constructed, and the second brightness curve conforms to the real change relationship between brightness and distance, which is conducive to improving the reliability of subsequent calculations.

[0192] Optionally, in the above Figure 4 On the basis of the corresponding embodiments, in another optional embodiment provided by the embodiment of the present application, constructing a brightness separation curve according to the first brightness curve and the second brightness curve may specifically include:

[0193] Sample X average distances, where X is an integer greater than 1;

[0194] For each average distance among the X average distances, determine a first average brightness corresponding to the average distance according to the first brightness curve, and determine a second average brightness corresponding to the average distance according to the second brightness curve;

[0195] For each average distance of the X average distances, the first average brightness and the second average brightness are averaged to obtain a brightness calibration threshold;

[0196] According to the brightness calibration threshold corresponding to each average distance, a brightness separation curve is constructed.

[0197] In one or more embodiments, a method for constructing a brightness separation curve is introduced. As can be seen from the above embodiments, after calibrating the first brightness curve and the second brightness curve, a brightness separation curve can be manually designed. The following will introduce how to construct the brightness separation curve in conjunction with the diagram.

[0198] Specifically, for ease of understanding, see Fig.13 , Fig.13 This is a schematic diagram of the brightness separation curve in the embodiment of the present application. As shown in the figure, A1 is used to indicate the first brightness curve for the first object (e.g., palm), and B1 is used to indicate the second brightness curve for the second object (e.g., mobile phone). After the brightness curves of different objects (i.e., the first object and the second object) at different average distances have been collected, X average distances can be sampled. It can be understood that the X average distances should in principle be less than or equal to the reliable distance (e.g., 13 cm), where the reliable distance is usually related to product design requirements. In this application, the object distance threshold and the reliable distance may be equal.

[0199] Take X as 12 as an example, and the interval between each sampled average distance is 1 cm. Based on this, for each of the X average distances, the average of the average distances corresponding to the first average brightness of the first object (e.g., palm) and the second average brightness of the second object (e.g., mobile phone) can be taken at the same average distance to obtain the segmentation point (i.e., brightness calibration threshold) corresponding to the average distance. Thus, according to the corresponding relationship between the X brightness calibration thresholds and the X average distances, the corresponding brightness separation curve is constructed. Please refer to Fig.13 It can be seen that C1 is used to indicate the brightness separation curve, and D1 is used to indicate a segmentation point on the brightness separation curve (ie, the brightness calibration threshold).

[0200] Furthermore, in an embodiment of the present application, a method for constructing a brightness separation curve is provided. Through the above method, after calibrating the first brightness curve and the second brightness curve, a brightness separation curve can be artificially designed. Thus, in practical applications, the category recognition result of the object to be recognized can be quickly determined by judging whether the average brightness of the image is above or below the brightness separation curve.

[0201] Optionally, in the above Figure 4 On the basis of the corresponding embodiments, in another optional embodiment provided by the embodiment of the present application, constructing a high-order polynomial function according to the brightness separation curve may specifically include:

[0202] According to the brightness separation curve, N groups of sample parameters are obtained by sampling, wherein each group of sample parameters includes an average distance and a brightness calibration threshold, and N is an integer greater than or equal to (M+1);

[0203] Constructing N correlation functions according to N groups of sample parameters, wherein each correlation function is constructed based on a group of sample parameters;

[0204] (M+1) coefficients are calculated based on the N correlation functions, and a high-order polynomial function is generated.

[0205] In one or more embodiments, a method of constructing a high-order polynomial function based on a brightness separation curve is introduced. It can be seen from the aforementioned embodiments that the image brightness threshold can be calculated by a high-order polynomial function, wherein the high-order polynomial function can specifically be a polynomial function of the second order or above, that is, M is an integer greater than or equal to 1, and the high-order polynomial function includes at least (M+1) coefficients. It should be noted that solving (M+1) coefficients requires at least (M+1) sets of equations, that is, N is an integer greater than or equal to (M+1),

[0206] Specifically, firstly, N average distances are sampled in the reliable distances. Then, based on the brightness separation curve, the brightness calibration threshold corresponding to each of the N average distances is determined. Thus, several groups of sample parameters are constructed, and each group of sample parameters includes the average distance and its corresponding brightness calibration threshold. Then, through the polynomial fitting principle, N correlation functions can be constructed in the following way:

[0207]

[0208] Where f(x) represents the brightness calibration threshold. x represents the average distance. α j represents the jth coefficient. U represents the total number of coefficients, that is, U is equal to (M+1).

[0209] Based on this, for the sake of convenience, the following will take the high-order polynomial function as a 4th-order polynomial function as an example for explanation. At this time, M is equal to 4, so combined with formula (4), the following formula can be obtained:

[0210] f(x)=α 0 +α 1 x+α 2 x 2 +α 3 x 3 +α 4 x 4 ; Formula (5)

[0211] Among them, α 0 , α 1 , α 2 , α 3 , α 4 are all coefficients, that is, there are 5 coefficients that need to be solved. Assume that 5 sets of sample parameters are obtained by sampling the brightness separation curve, which are (x 1 ,y 1 ), (x 2 ,y 2 ), (x 3 ,y 3 ), (x 4 ,y 4 ) and (x 5 ,y 5 ), where x represents the average distance and y represents the brightness calibration threshold. Then, combined with formula (5), the following formula can be obtained:

[0212]

[0213] From this, the coefficient α can be solved 0 , α 1 , α 2 , α 3 , α 4 , after substituting these coefficients into formula (5), we can get a fourth-order polynomial function.

[0214] Furthermore, in an embodiment of the present application, a method for constructing a high-order polynomial function based on a brightness separation curve is provided. Through the above method, after the brightness separation curve is constructed, multiple groups of sample parameters are sampled based on the brightness separation curve, and then the coefficients involved in the high-order polynomial function are derived. It can be seen that only a few groups of sample parameters are needed to construct a high-order polynomial function, and the derivation process is relatively simple and efficient.

[0215] Optionally, in the above Figure 4On the basis of the corresponding embodiments, in another optional embodiment provided by the embodiment of the present application, training a brightness prediction model according to a brightness separation curve may specifically include:

[0216] According to the brightness separation curve, R groups of sample parameters are obtained, wherein each group of sample parameters includes an average distance and a brightness calibration threshold, and R is an integer greater than 1;

[0217] For each group of sample parameters in the R groups of sample parameters, based on the average distance included in the sample parameters, a brightness prediction threshold is obtained through a brightness prediction model;

[0218] Determine the target loss according to R brightness prediction thresholds and R brightness prediction thresholds, wherein the R brightness prediction thresholds are derived from R groups of sample parameters, and the R brightness prediction thresholds are obtained based on the average distance prediction included in each group of sample parameters in the R groups of sample parameters;

[0219] The target loss is used to update the model parameters of the brightness prediction model.

[0220] In one or more embodiments, a method for training a brightness prediction model is introduced. As can be seen from the above embodiments, the brightness threshold of an image can be predicted by the brightness prediction model. The brightness prediction model can use a multilayer perceptron (MLP). The MLP includes an input layer, a hidden layer, and an output layer, and different layers of the MLP are fully connected.

[0221] Specifically, firstly, R average distances are sampled in the reliable distances. Then, based on the brightness separation curve, the brightness calibration threshold corresponding to each of the R average distances is determined. Thus, several groups of sample parameters are constructed, and each group of sample parameters includes the average distance and its corresponding brightness calibration threshold. Then, the average distance is used as the input of the brightness prediction model, and the brightness prediction threshold is output through the brightness prediction model, wherein the brightness prediction threshold is the "prediction value" and the brightness calibration threshold is the "true value".

[0222] Based on this, in a feasible way, the mean absolute error (MAE) function can be used to calculate the target loss, that is:

[0223]

[0224] Among them, L MAE represents the target loss. R represents the total number of sample parameters. i represents the i-th sample parameter. y i Represents the brightness calibration threshold in the i-th group of sample parameters. Represents the brightness prediction threshold obtained based on the average distance prediction in the i-th group of sample parameters.

[0225] Based on this, in another feasible way, the mean absolute error loss (MAE) function can be used to calculate the target loss, that is:

[0226]

[0227] Among them, L MSE represents the target loss.

[0228] After calculating the target loss, the stochastic gradient descent (SGD) method can be used to perform gradient backward calculation on the target loss calculated in this round, and obtain the updated values ​​of all model parameters, thereby updating the model parameters.

[0229] It should be noted that, in addition to the MAE function and MSE function mentioned in the above example, other types of loss functions can be used to calculate the target loss in practical applications. This is only for illustration and should not be construed as a limitation on this application.

[0230] Furthermore, in an embodiment of the present application, a method for training a brightness prediction model is provided. Through the above method, after constructing a brightness separation curve, multiple groups of sample parameters are sampled based on the brightness separation curve, and then the model parameters of the brightness prediction model are trained. Since the brightness prediction model belongs to a neural network and has a certain learning ability, it is conducive to predicting a more accurate image brightness threshold.

[0231] Optionally, in the above Figure 4 On the basis of the corresponding embodiments, in another optional embodiment provided by the embodiment of the present application, determining the category recognition result of the object to be recognized according to the magnitude relationship between the average brightness of the image and the image brightness threshold may specifically include:

[0232] If the average brightness of the image is greater than the image brightness threshold, the category recognition result of the object to be recognized is determined to be a palm;

[0233] If the average brightness of the image is less than or equal to the image brightness threshold, the category recognition result of the object to be recognized is determined to be a terminal device.

[0234] In one or more embodiments, a method for identifying a palm and a terminal device is introduced. It can be seen from the aforementioned embodiments that the present application can detect whether the object to be identified is a palm or a terminal device (for example, a mobile phone). That is, after determining the image brightness threshold according to the average distance of the object, it can be determined whether the average image brightness is greater than the image brightness threshold. If the average image brightness is greater than the image brightness threshold, the object to be identified is determined to be a "palm". On the contrary, if the average image brightness is less than or equal to the image brightness threshold, the object to be identified is determined to be a "terminal device (for example, a mobile phone)".

[0235] For easier understanding, see Fig.14 , Fig.14 This is a flow chart of determining the corresponding category recognition result of an object in an embodiment of the present application, as shown in the figure, specifically:

[0236] In step S1, distance data is collected by one or more distance sensors, and then an average distance of the object is calculated based on at least one collected distance data.

[0237] In step S2, by judging the size relationship between the average object distance and the object distance threshold, it is detected whether the object to be identified is close enough. If the average object distance is less than or equal to the object distance threshold (for example, 12 cm), it is determined that the object to be detected is close enough, so step S3 is continued.

[0238] In step S3, the infrared fill light is turned on.

[0239] In step S4, the next frame of infrared image is collected as the target infrared image. Then, based on formula (1), the brightness values ​​corresponding to all pixels in the target infrared image are taken to calculate the average brightness of the image. Alternatively, based on formula (2), the brightness values ​​corresponding to the pixels in the middle area of ​​the target infrared image are taken to calculate the average brightness avglight of the image.

[0240] In step S5, the average distance of the object is substituted into formula (4) to obtain the image brightness threshold. If the average image brightness is greater than the image brightness threshold, the object to be identified is determined to be a palm, and then step S6 is executed. If the average image brightness is less than or equal to the image brightness threshold, the object to be identified is determined to be a terminal device (for example, a mobile phone), and then step S7 is executed. Specifically, the following formula can be used for determination:

[0241]

[0242] Wherein, t represents the judgment value, t=0 represents the object to be identified as a palm, and t=1 represents the object to be identified as a mobile phone. f′(x) represents the image brightness threshold. avglight represents the average brightness of the image.

[0243] In step S6, switch to palm brushing mode.

[0244] In step S7, switch to code scanning mode.

[0245] Secondly, in the embodiment of the present application, a method for identifying a palm and a terminal device is provided. Through the above method, combined with the actual scene requirements, the size relationship between the average brightness of the image and the image brightness threshold can be used to determine whether the object to be identified is a palm or a terminal device. Thus, the feasibility and operability of the solution are improved.

[0246] Optionally, in the above Figure 4 On the basis of the corresponding embodiments, another optional embodiment provided by the embodiment of the present application may further include:

[0247] If the category recognition result is a palm, the fill light is activated, and a first image is captured by a color camera, wherein the first image includes an image corresponding to the palm print;

[0248] Verifying the palm print included in the first image;

[0249] Optionally, include:

[0250] If the category recognition result is a terminal device, the fill light is turned off, and a second image is captured by a color camera, wherein the second image includes an image corresponding to the payment code;

[0251] The payment code included in the second image is verified.

[0252] In one or more embodiments, a method for executing a payment process based on recognition results is introduced. It can be seen from the aforementioned embodiments that by using the different reflection performances and imaging effects of the terminal device (e.g., mobile phone) and the palm at the same distance under the irradiation of infrared fill light, it is possible to quickly distinguish the image based on the pre-calculated average brightness and image brightness threshold, thereby accurately determining the user's willingness to scan the code (e.g., using palm print payment or QR code payment).

[0253] Specifically, different processing is usually performed based on different scanning intentions. For example, in the palm swiping mode, the fill light needs to be turned on for fill light. However, in the QR code scanning mode, the fill light needs to be turned off to avoid the reflection of the terminal device (for example, a mobile phone) causing the QR code detection to fail. In addition, the exposure algorithm of the color image can also be adjusted to avoid overexposure caused by the terminal device (for example, a mobile phone) being too close. Fig.15 The specific scenario provided is explained.

[0254] For ease of understanding, please refer to Fig.15 , Fig.15This is a schematic diagram of a scan code payment scenario in an embodiment of the present application. Fig.15 As shown in Figure (A), the user places his palm in the middle area of ​​the scanning screen of the code scanning terminal. When the palm is close enough, the code scanning terminal can collect a target infrared image through the infrared camera. At this time, the code scanning terminal uses the method introduced in the above embodiment to determine whether the object to be identified captured in the target infrared image is a palm. If it is a palm, the fill light is started, and the first image is collected through the color camera, where the first image includes the image corresponding to the palm print. Based on this, the palm print in the first image can be verified. After the verification is passed, the payment or release process can be carried out.

[0255] For ease of understanding, please refer to Fig.15 , Fig.15 This is a schematic diagram of a scan code payment scenario in an embodiment of the present application. Fig.15 As shown in Figure (B), the user places the mobile phone screen in the middle area of ​​the scanning screen of the code scanning terminal. When the mobile phone screen is close enough, the code scanning terminal can collect a target infrared image through the infrared camera. At this time, the code scanning terminal uses the method introduced in the above embodiment to determine whether the object to be identified photographed in the target infrared image is a mobile phone. If it is a mobile phone, turn off the fill light and collect the second image through the color camera, where the second image includes the image corresponding to the payment code. Based on this, the payment code in the second image can be verified. After the verification is passed, the payment or release process can be carried out.

[0256] It is understandable that the process of using a scanning terminal to realize payment does not need to be adjusted, and the original interaction process can be maintained. For example, the user opens the payment QR code and approaches the camera of the scanning terminal to make a QR code payment, or the user extends his palm and approaches the camera of the scanning terminal to make a palm payment.

[0257] Again, in the embodiment of the present application, a method for executing the payment process based on the recognition result is provided. Through the above method, when the object to be identified is close to the code scanning terminal, the infrared fill light is turned on. According to the difference in the reflection effect of the palm and the terminal device under the infrared fill light, it is possible to quickly distinguish whether it is a palm or a terminal device, and switch to the corresponding payment mode. Since the brightness of the infrared fill light is fixed, the reflection effect is related to the material, and there is an object approaching interception mechanism. Therefore, the infrared imaging effect is fixed and is not affected by complex light, thereby improving the code scanning effect and optimizing the payment process.

[0258] The image recognition device in this application is described in detail below. Fig.16 , Fig.16 This is a schematic diagram of an embodiment of an image recognition device in an embodiment of the present application. The image recognition device 30 includes:

[0259] An acquisition module 310 is used to acquire an average object distance for an object to be identified;

[0260] The acquisition module 310 is further configured to acquire a target infrared image if the average object distance is less than or equal to the object distance threshold, wherein the target infrared image includes an image corresponding to the object to be identified, and each pixel in the target infrared image corresponds to a brightness value;

[0261] A determination module 320, configured to determine an average image brightness corresponding to the target infrared image according to brightness values ​​of a plurality of pixels in the target infrared image;

[0262] The determination module 320 is further used to determine the category recognition result of the object to be recognized according to the size relationship between the average brightness of the image and the image brightness threshold, wherein the image brightness threshold is determined based on the average distance of the object.

[0263] In the embodiment of the present application, an image recognition device is provided. With the above device, since different objects will present different reflection performances and imaging effects under the same distance of infrared fill light, after calculating the average brightness of the target infrared image, the size relationship between the image brightness threshold and the image average brightness can be used to quickly distinguish the objects to be recognized included in the target infrared image, solving the problem of insufficient robustness of traditional detection algorithms under complex light, thereby improving the accuracy of image recognition.

[0264] Optionally, in the above Fig.16 On the basis of the corresponding embodiment, in another embodiment of the image recognition device 30 provided in the embodiment of the present application,

[0265] The acquisition module 310 is specifically used to collect distance data through a distance sensor;

[0266] If the distance data is within the object detection range, the distance data is taken as the average distance of the object.

[0267] In an embodiment of the present application, an image recognition device is provided. By adopting the above device, the average distance of an object can be detected by using one distance sensor, thereby saving the production cost of the terminal device to a certain extent.

[0268] Optionally, in the above Fig.16 On the basis of the corresponding embodiment, in another embodiment of the image recognition device 30 provided in the embodiment of the present application,

[0269] The acquisition module 310 is specifically configured to collect Q distance data through Q distance sensors, where Q is an integer greater than 1;

[0270] If the Q distance data are all within the object detection range, the Q distance data are averaged to obtain the average distance of the object;

[0271] If at least one of the Q distance data is not within the object detection range, the remaining distance data of the Q distance data are averaged to obtain an average distance of the object.

[0272] In an embodiment of the present application, an image recognition device is provided. Using the above device, multiple distance sensors are used to detect the distance data of the object respectively, and then the average distance of the object is calculated by combining these distance data. In this way, a more accurate and objective average distance of the object can be obtained, which is conducive to improving the reliability and accuracy of subsequent detection.

[0273] Optionally, in the above Fig.16 On the basis of the corresponding embodiment, in another embodiment of the image recognition device 30 provided in the embodiment of the present application,

[0274] The determination module 320 is specifically used to sum the brightness values ​​of each pixel in the target infrared image to obtain a first brightness sum;

[0275] According to the total number of pixels of the target infrared image, the first brightness sum is averaged to obtain the average brightness of the image, wherein the target infrared image includes W×H pixels, W represents the number of width pixels of the target infrared image, H represents the number of height pixels of the target infrared image, and H and W are both integers greater than 1.

[0276] In an embodiment of the present application, an image recognition device is provided. With the above device, after acquiring a target infrared image, the average brightness of the image can be directly calculated based on the brightness value of each pixel in the target infrared image, without the need to perform image cropping or other processing on the target infrared image, thereby improving processing efficiency.

[0277] Optionally, in the above Fig.16 On the basis of the corresponding embodiment, in another embodiment of the image recognition device 30 provided in the embodiment of the present application,

[0278] The determination module 320 is specifically configured to obtain an infrared sub-image from a target infrared image, wherein the target infrared image includes W×H pixels, W represents the number of pixels of the width of the target infrared image, H represents the number of pixels of the height of the target infrared image, and H and W are both integers greater than 1;

[0279] The brightness value of each pixel in the infrared sub-image is summed to obtain a second brightness sum;

[0280] According to the total number of pixels of the infrared sub-image, the second brightness sum is averaged to obtain the average brightness of the image, wherein the infrared sub-image includes w×h pixels, w represents the number of width pixels of the infrared sub-image, h represents the number of height pixels of the infrared sub-image, h is an integer greater than 1 and less than H, and w is an integer greater than 1 and less than W.

[0281] In an embodiment of the present application, an image recognition device is provided. With the above device, after acquiring the target infrared image, considering that the object to be recognized is often in the middle area of ​​the image screen. Therefore, in order to avoid possible interference, in the actual calculation process, the middle area of ​​the image screen can also be selected as the infrared sub-image, so that the average brightness of the image calculated in this way has better reliability and accuracy.

[0282] Optionally, in the above Fig.16 On the basis of the corresponding embodiment, in another embodiment of the image recognition device 30 provided in the embodiment of the present application,

[0283] The acquisition module 310 is also used to calculate the image brightness threshold by a high-order polynomial function based on the average distance of the object before determining the category recognition result of the object to be identified according to the size relationship between the average brightness of the image and the image brightness threshold, wherein the high-order polynomial function includes (M+1) coefficients, and M is an integer greater than or equal to 1.

[0284] or,

[0285] The acquisition module 310 is further configured to acquire an image brightness threshold value through a brightness prediction model based on an average object distance.

[0286] In an embodiment of the present application, an image recognition device is provided. Using the above device, one method is to substitute the average distance of the object into a high-order polynomial function to calculate the image brightness threshold. This method has high computational efficiency and can quickly determine the category of the object to be identified. Another method is to input the average distance of the object into a brightness prediction model, and the image brightness threshold is predicted by the model. This method can utilize the learning ability of the neural network to predict a more reliable image brightness threshold. It can be seen that the above two methods can determine the image brightness threshold, thereby increasing the feasibility and flexibility of the solution.

[0287] Optionally, in the above Fig.16 On the basis of the corresponding embodiment, in another embodiment of the image recognition device 30 provided in the embodiment of the present application, the image recognition device 30 further includes a construction module 330 and a generation module 340;

[0288] A construction module 330 is used to construct a first brightness curve for the first object, wherein the first brightness curve represents a relationship between each average distance and a first average brightness;

[0289] The construction module 330 is further used to construct a second brightness curve for the second object, wherein the second brightness curve represents the relationship between each average distance and the second average brightness;

[0290] The construction module 330 is further used to construct a brightness separation curve according to the first brightness curve and the second brightness curve, wherein the brightness separation curve represents the relationship between each average distance and the brightness calibration threshold;

[0291] The generating module 340 is used to construct a high-order polynomial function according to the brightness separation curve, or to train a brightness prediction model according to the brightness separation curve.

[0292] In an embodiment of the present application, an image recognition device is provided. By using the above device, a high-order polynomial function or a brightness prediction model can be constructed by utilizing the difference in reflection effects of different objects under infrared fill light. In practical applications, a high-order polynomial function or a brightness prediction model can be selected according to needs to determine the image brightness threshold, thereby increasing the feasibility and operability of the solution.

[0293] Optionally, in the above Fig.16 On the basis of the corresponding embodiment, in another embodiment of the image recognition device 30 provided in the embodiment of the present application,

[0294] A construction module 330 is specifically configured to collect a first sample image set corresponding to K average distances through an infrared camera, wherein the first sample image set includes a first sample image corresponding to each average distance, the first sample image includes an image corresponding to the first object, and each pixel in the first sample image corresponds to a brightness value, and K is an integer greater than 1;

[0295] For each first sample image in the first sample image set, determining a first average brightness corresponding to the first sample image according to brightness values ​​of a plurality of pixels in the first sample image;

[0296] A first brightness curve for the first object is constructed according to the first average brightness and the average distance corresponding to each first sample image.

[0297] In an embodiment of the present application, an image recognition device is provided. Using the above device, an infrared camera collects a series of infrared images of a first object, thereby calculating the first average brightness corresponding to each infrared image. At the same time, a distance sensor collects distance data of each infrared image, thereby calculating the average distance corresponding to each infrared image. Thus, according to the corresponding relationship between the first average brightness and the average distance, a corresponding first brightness curve is constructed, and the first brightness curve conforms to the real change relationship between brightness and distance, thereby facilitating the improvement of the reliability of subsequent calculations.

[0298] Optionally, in the above Fig.16 On the basis of the corresponding embodiment, in another embodiment of the image recognition device 30 provided in the embodiment of the present application,

[0299] A construction module 330 is specifically configured to collect a second sample image set corresponding to T average distances through an infrared camera, wherein the second sample image set includes a second sample image corresponding to each average distance, the second sample image includes an image corresponding to the second object, and each pixel in the second sample image corresponds to a brightness value, and T is an integer greater than 1;

[0300] For each second sample image in the second sample image set, determining a second average brightness corresponding to the second sample image according to brightness values ​​of a plurality of pixels in the second sample image;

[0301] A second brightness curve for the second object is constructed according to the second average brightness and the average distance corresponding to each second sample image.

[0302] In an embodiment of the present application, an image recognition device is provided. Using the above device, an infrared camera collects a series of infrared images of a second object to calculate the second average brightness corresponding to each infrared image. At the same time, a distance sensor collects distance data of each infrared image to calculate the average distance corresponding to each infrared image. Thus, according to the corresponding relationship between the second average brightness and the average distance, a corresponding second brightness curve is constructed, and the second brightness curve conforms to the real change relationship between brightness and distance, which is conducive to improving the reliability of subsequent calculations.

[0303] Optionally, in the above Fig.16 On the basis of the corresponding embodiment, in another embodiment of the image recognition device 30 provided in the embodiment of the present application,

[0304] A construction module 330 is specifically configured to sample X average distances, where X is an integer greater than 1;

[0305] For each average distance among the X average distances, determine a first average brightness corresponding to the average distance according to the first brightness curve, and determine a second average brightness corresponding to the average distance according to the second brightness curve;

[0306] For each average distance of the X average distances, the first average brightness and the second average brightness are averaged to obtain a brightness calibration threshold;

[0307] According to the brightness calibration threshold corresponding to each average distance, a brightness separation curve is constructed.

[0308] In an embodiment of the present application, an image recognition device is provided. With the above device, after calibrating the first brightness curve and the second brightness curve, a brightness separation curve can be artificially designed. Therefore, in practical applications, the category recognition result of the object to be recognized can be quickly determined by judging whether the average brightness of the image is above or below the brightness separation curve.

[0309] Optionally, in the above Fig.16 On the basis of the corresponding embodiment, in another embodiment of the image recognition device 30 provided in the embodiment of the present application,

[0310] A generating module 340 is specifically used to obtain N groups of sample parameters according to the brightness separation curve sampling, wherein each group of sample parameters includes an average distance and a brightness calibration threshold, and N is an integer greater than or equal to (M+1);

[0311] Constructing N correlation functions according to N groups of sample parameters, wherein each correlation function is constructed based on a group of sample parameters;

[0312] (M+1) coefficients are calculated based on the N correlation functions, and a high-order polynomial function is generated.

[0313] In an embodiment of the present application, an image recognition device is provided. Using the above device, after constructing a brightness separation curve, multiple groups of sample parameters are sampled based on the brightness separation curve, and then the coefficients involved in the high-order polynomial function are derived. It can be seen that only a few groups of sample parameters are needed to construct a high-order polynomial function, and the derivation process is relatively simple and efficient.

[0314] Optionally, in the above Fig.16 On the basis of the corresponding embodiment, in another embodiment of the image recognition device 30 provided in the embodiment of the present application,

[0315] A generating module 340 is specifically used to obtain R groups of sample parameters according to the brightness separation curve sampling, wherein each group of sample parameters includes an average distance and a brightness calibration threshold, and R is an integer greater than 1;

[0316] For each group of sample parameters in the R groups of sample parameters, based on the average distance included in the sample parameters, a brightness prediction threshold is obtained through a brightness prediction model;

[0317] Determine the target loss according to R brightness prediction thresholds and R brightness prediction thresholds, wherein the R brightness prediction thresholds are derived from R groups of sample parameters, and the R brightness prediction thresholds are obtained based on the average distance prediction included in each group of sample parameters in the R groups of sample parameters;

[0318] The target loss is used to update the model parameters of the brightness prediction model.

[0319] In an embodiment of the present application, an image recognition device is provided. Using the above device, after constructing a brightness separation curve, multiple groups of sample parameters are sampled based on the brightness separation curve, and then the model parameters of the brightness prediction model are trained. Since the brightness prediction model belongs to a neural network and has a certain learning ability, it is conducive to predicting a more accurate image brightness threshold.

[0320] Optionally, in the above Fig.16 On the basis of the corresponding embodiment, in another embodiment of the image recognition device 30 provided in the embodiment of the present application,

[0321] A determination module 320 is specifically configured to determine that the category recognition result of the object to be recognized is a palm if the average brightness of the image is greater than the image brightness threshold;

[0322] If the average brightness of the image is less than or equal to the image brightness threshold, the category recognition result of the object to be recognized is determined to be a terminal device.

[0323] In the embodiment of the present application, an image recognition device is provided. By using the above device, combined with the actual scene requirements, the size relationship between the average brightness of the image and the image brightness threshold can be used to determine whether the object to be recognized is a palm or a terminal device. Thus, the feasibility and operability of the solution are improved.

[0324] Optionally, in the above Fig.16 On the basis of the corresponding embodiment, in another embodiment of the image recognition device 30 provided in the embodiment of the present application, the image recognition device 30 further includes a verification module 350;

[0325] The acquisition module 310 is further configured to start the fill light if the category recognition result is a palm, and to collect a first image through a color camera, wherein the first image includes an image corresponding to the palm print;

[0326] A verification module 350, configured to verify the palm print included in the first image;

[0327] or,

[0328] The acquisition module 310 is further configured to turn off the fill light and capture a second image through a color camera if the category recognition result is a terminal device, wherein the second image includes an image corresponding to the payment code;

[0329] The verification module 350 is further used to verify the payment code included in the second image.

[0330] In the embodiment of the present application, an image recognition device is provided. With the above device, when the object to be identified is close to the code scanning terminal, the infrared fill light is turned on. According to the difference in the reflection effect of the palm and the terminal device under the infrared fill light, it is possible to quickly distinguish whether it is a palm or a terminal device, and switch to the corresponding payment mode. Since the brightness of the infrared fill light is fixed, the reflection effect is related to the material, and there is an object approaching interception mechanism. Therefore, the infrared imaging effect is fixed and is not affected by complex light, thereby improving the code scanning effect and optimizing the payment process.

[0331] The present application also provides an image recognition device that can be deployed in a terminal device, such as Fig.17 For the sake of convenience, only the parts related to the embodiments of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiments of the present application. In the embodiments of the present application, the terminal device is a mobile phone as an example for explanation:

[0332] Fig.17 FIG. 1 is a block diagram showing a partial structure of a mobile phone related to a terminal device provided in an embodiment of the present application. Fig.17 The mobile phone includes: a radio frequency (RF) circuit 410, a memory 420, an input unit 430, a display unit 440, a sensor 450, an audio circuit 460, a wireless fidelity (WiFi) module 470, a processor 480, and a power supply 490. Those skilled in the art will understand that Fig.17 The mobile phone structure shown in the figure does not constitute a limitation on the mobile phone, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0333] Combine the following Fig.17 A detailed introduction to the various components of the mobile phone:

[0334] The RF circuit 410 can be used for receiving and sending signals during the process of sending and receiving information or making calls. In particular, after receiving the downlink information of the base station, it is sent to the processor 480 for processing; in addition, the designed uplink data is sent to the base station. Generally, the RF circuit 410 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, etc. In addition, the RF circuit 410 can also communicate with the network and other devices through wireless communication. The above-mentioned wireless communication can use any communication standard or protocol, including but not limited to the global system of mobile communication (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), long term evolution (LTE), email, short messaging service (SMS), etc.

[0335] The memory 420 can be used to store software programs and modules. The processor 480 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 420. The memory 420 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 420 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0336] The input unit 430 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile phone. Specifically, the input unit 430 may include a touch panel 431 and other input devices 432. The touch panel 431, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 431 or near the touch panel 431 using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program. Optionally, the touch panel 431 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the processor 480, and can receive and execute commands sent by the processor 480. In addition, the touch panel 431 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 431, the input unit 430 may further include other input devices 432. Specifically, the other input devices 432 may include but are not limited to one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a mouse, a joystick, etc.

[0337] The display unit 440 may be used to display information input by the user or information provided to the user and various menus of the mobile phone. The display unit 440 may include a display panel 441. Optionally, the display panel 441 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, the touch panel 431 may cover the display panel 441. When the touch panel 431 detects a touch operation on or near it, it is transmitted to the processor 480 to determine the type of touch event. Subsequently, the processor 480 provides a corresponding visual output on the display panel 441 according to the type of touch event. Although in Fig.17 In the embodiment, the touch panel 431 and the display panel 441 are used as two independent components to realize the input and output functions of the mobile phone, but in some embodiments, the touch panel 431 and the display panel 441 can be integrated to realize the input and output functions of the mobile phone.

[0338] The mobile phone may also include at least one sensor 450, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 441 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 441 and / or the backlight when the mobile phone is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.

[0339] The audio circuit 460, the speaker 461, and the microphone 462 can provide an audio interface between the user and the mobile phone. The audio circuit 460 can transmit the received audio data to the speaker 461 after converting the received audio data into an electrical signal, which is converted into a sound signal for output; on the other hand, the microphone 462 converts the collected sound signal into an electrical signal, which is received by the audio circuit 460 and converted into audio data, and then the audio data is output to the processor 480 for processing, and then sent to another mobile phone through the RF circuit 410, or the audio data is output to the memory 420 for further processing.

[0340] WiFi is a short-range wireless transmission technology. The mobile phone can help users send and receive emails, browse web pages and access streaming media through the WiFi module 470. It provides users with wireless broadband Internet access. Fig.17 A WiFi module 470 is shown, but it is understandable that it is not an essential component of the mobile phone and can be omitted as needed without changing the essence of the invention.

[0341] The processor 480 is the control center of the mobile phone. It uses various interfaces and lines to connect various parts of the entire mobile phone. It executes various functions of the mobile phone and processes data by running or executing software programs and / or modules stored in the memory 420, and calling data stored in the memory 420. Optionally, the processor 480 may include one or more processing units; optionally, the processor 480 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 480.

[0342] The mobile phone also includes a power source 490 (such as a battery) for supplying power to various components. Optionally, the power source can be logically connected to the processor 480 through a power management system, thereby implementing functions such as charging, discharging, and power consumption management through the power management system.

[0343] Although not shown, the mobile phone may also include a camera, a Bluetooth module, etc., which will not be described in detail here.

[0344] The steps performed by the terminal device in the above embodiment can be based on the Fig.17 The terminal device structure shown.

[0345] The present application also provides an image recognition device that can be deployed on a server. Fig.18 : is a schematic diagram of a server structure provided in an embodiment of the present application. The server 500 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 522 (for example, one or more processors) and a memory 532, and one or more storage media 530 (for example, one or more mass storage devices) storing application programs 542 or data 544. Among them, the memory 532 and the storage medium 530 can be short-term storage or permanent storage. The program stored in the storage medium 530 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the central processing unit 522 can be configured to communicate with the storage medium 530 to execute a series of instruction operations in the storage medium 530 on the server 500.

[0346] The server 500 may also include one or more power supplies 526, one or more wired or wireless network interfaces 550, one or more input and output interfaces 558, and / or one or more operating systems 541, such as Windows Server 2003, ... TM , Mac OS X TM , Unix TM ,Linux TM , FreeBSD TM etc.

[0347] The steps performed by the server in the above embodiment can be based on the Fig.18 The server structure shown.

[0348] A computer device is also provided in an embodiment of the present application, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the methods described in the above embodiments are implemented.

[0349] A computer-readable storage medium is also provided in an embodiment of the present application, on which a computer program is stored. When the computer program is executed by a processor, the steps of the methods described in the above embodiments are implemented.

[0350] A computer program product is also provided in an embodiment of the present application, including a computer program, which, when executed by a processor, implements the steps of the methods described in the above embodiments.

[0351] It can be understood that in the specific implementation of this application, user information, palm print information, payment code and other related data are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0352] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0353] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0354] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0355] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0356] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a server or terminal device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store computer programs.

[0357] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for image recognition, characterized in that: include: Obtaining an average object distance for an object to be identified; If the object average distance is less than or equal to the object distance threshold, a target infrared image is acquired, wherein the target infrared image includes an image corresponding to the object to be identified, and each pixel in the target infrared image corresponds to a brightness value; Determining an average brightness of an image corresponding to the target infrared image according to brightness values ​​of a plurality of pixels in the target infrared image; Based on the average distance of the objects, the image brightness threshold is calculated by a high-order polynomial function, wherein the high-order polynomial function includes M+1 coefficients, and M is an integer greater than or equal to 1; or based on the average distance of the objects, the image brightness threshold is obtained by a brightness prediction model; Determining the category recognition result of the object to be recognized according to the magnitude relationship between the average brightness of the image and the image brightness threshold; The method further comprises: Constructing a first brightness curve for the first object, wherein the first brightness curve represents a relationship between each average distance and a first average brightness; constructing a second brightness curve for the second object, wherein the second brightness curve represents a relationship between each average distance and a second average brightness; Constructing a brightness separation curve according to the first brightness curve and the second brightness curve, wherein the brightness separation curve represents the relationship between each average distance and a brightness calibration threshold; The high-order polynomial function is constructed according to the brightness separation curve, or the brightness prediction model is trained according to the brightness separation curve.

2. The method according to claim 1, characterized in that: The obtaining of the average distance of the object to be identified includes: Collect distance data through distance sensors; If the distance data is within the object detection range, the distance data is used as the average distance of the object.

3. The method according to claim 1, characterized in that The obtaining of the average distance of the object to be identified includes: Collect Q distance data through Q distance sensors, where Q is an integer greater than 1; If the Q distance data are all within the object detection range, the Q distance data are averaged to obtain the average distance of the object; If at least one of the Q distance data is not within the object detection range, the remaining distance data of the Q distance data are averaged to obtain the average distance of the object.

4. The method according to claim 1, characterized in that: Determining the average brightness of the image corresponding to the target infrared image according to the brightness values ​​of a plurality of pixels in the target infrared image includes: Summing the brightness values ​​of each pixel in the target infrared image to obtain a first brightness sum; According to the total number of pixels of the target infrared image, the first brightness sum is averaged to obtain the average brightness of the image, wherein the target infrared image includes W×H pixels, W represents the number of pixels of the width of the target infrared image, and H represents the number of pixels of the height of the target infrared image, and both H and W are integers greater than 1.

5. The method according to claim 1, characterized in that Determining the average brightness of the image corresponding to the target infrared image according to the brightness values ​​of a plurality of pixels in the target infrared image includes: Acquire an infrared sub-image from the target infrared image, wherein the target infrared image includes W×H pixels, W represents the number of pixels of the width of the target infrared image, H represents the number of pixels of the height of the target infrared image, and both H and W are integers greater than 1; Summing the brightness values ​​of each pixel in the infrared sub-image to obtain a second brightness sum; According to the total number of pixels of the infrared sub-image, the second brightness sum is averaged to obtain the average brightness of the image, wherein the infrared sub-image includes w×h pixels, w represents the number of pixels in the width of the infrared sub-image, h represents the number of pixels in the height of the infrared sub-image, h is an integer greater than 1 and less than H, and w is an integer greater than 1 and less than W.

6. The method according to claim 1, characterized in that The step of constructing a first brightness curve for a first object includes: Collecting a first sample image set corresponding to K average distances by an infrared camera, wherein the first sample image set includes a first sample image corresponding to each average distance, the first sample image includes an image corresponding to the first object, and each pixel in the first sample image corresponds to a brightness value, and K is an integer greater than 1; For each first sample image in the first sample image set, determining a first average brightness corresponding to the first sample image according to brightness values ​​of a plurality of pixels in the first sample image; The first brightness curve for the first object is constructed according to the first average brightness and the average distance corresponding to each first sample image.

7. The method according to claim 1, characterized in that The constructing a second brightness curve for the second object includes: Collecting a second sample image set corresponding to T average distances by an infrared camera, wherein the second sample image set includes a second sample image corresponding to each average distance, the second sample image includes an image corresponding to the second object, and each pixel in the second sample image corresponds to a brightness value, and T is an integer greater than 1; For each second sample image in the second sample image set, determining a second average brightness corresponding to the second sample image according to brightness values ​​of a plurality of pixels in the second sample image; The second brightness curve for the second object is constructed according to the second average brightness and the average distance corresponding to each second sample image.

8. The method according to claim 1, characterized in that The step of constructing a brightness separation curve according to the first brightness curve and the second brightness curve includes: Sampling X average distances, where X is an integer greater than 1; For each average distance of the X average distances, determining a first average brightness corresponding to the average distance according to the first brightness curve, and determining a second average brightness corresponding to the average distance according to the second brightness curve; For each average distance of the X average distances, averaging the first average brightness and the second average brightness to obtain a brightness calibration threshold; The brightness separation curve is constructed according to the brightness calibration threshold corresponding to each average distance.

9. The method according to claim 1, characterized in that: The constructing the high-order polynomial function according to the brightness separation curve comprises: Sampling according to the brightness separation curve to obtain N groups of sample parameters, wherein each group of sample parameters includes an average distance and a brightness calibration threshold, and N is an integer greater than or equal to M+1; Constructing N correlation functions according to the N groups of sample parameters, wherein each correlation function is constructed based on a group of sample parameters; The M+1 coefficients are calculated according to the N correlation functions, and the high-order polynomial function is generated.

10. The method according to claim 1, characterized in that The step of training the brightness prediction model according to the brightness separation curve comprises: According to the brightness separation curve, R groups of sample parameters are obtained by sampling, wherein each group of sample parameters includes an average distance and a brightness calibration threshold, and R is an integer greater than 1; For each group of sample parameters in the R groups of sample parameters, based on an average distance included in the sample parameters, obtaining a brightness prediction threshold through the brightness prediction model; Determining the target loss according to R brightness prediction thresholds and R brightness prediction thresholds, wherein the R brightness prediction thresholds are derived from the R groups of sample parameters, and the R brightness prediction thresholds are obtained based on the average distance prediction included in each group of sample parameters in the R groups of sample parameters; The target loss is used to update the model parameters of the brightness prediction model.

11. The method according to any one of claims 1 to 10, characterized in that Determining the category recognition result of the object to be recognized according to the magnitude relationship between the average brightness of the image and the image brightness threshold includes: If the average brightness of the image is greater than the image brightness threshold, determining that the category recognition result of the object to be recognized is a palm; If the average brightness of the image is less than or equal to the image brightness threshold, it is determined that the category recognition result of the object to be identified is a terminal device.

12. The method according to claim 11, characterized in that The method further comprises: If the category recognition result is a palm, start the fill light and capture a first image through a color camera, wherein the first image includes an image corresponding to the palm print; Verifying the palm print included in the first image; Alternatively, the method further comprises: If the category recognition result is a terminal device, turn off the fill light and capture a second image through a color camera, wherein the second image includes an image corresponding to the payment code; The payment code included in the second image is verified.

13. An image recognition device, characterized in that: include: An acquisition module, used for acquiring an average object distance for an object to be identified; The acquisition module is further configured to acquire a target infrared image if the object average distance is less than or equal to an object distance threshold, wherein the target infrared image includes an image corresponding to the object to be identified, and each pixel in the target infrared image corresponds to a brightness value; A determination module, used to determine the average brightness of the image corresponding to the target infrared image according to the brightness values ​​of multiple pixels in the target infrared image; The acquisition module is further configured to calculate an image brightness threshold value through a high-order polynomial function based on the average distance of the object, wherein the high-order polynomial function includes M+1 coefficients, and M is an integer greater than or equal to 1; or, based on the average distance of the object, obtain the image brightness threshold value through a brightness prediction model; The determination module is further used to determine the category recognition result of the object to be recognized according to the magnitude relationship between the average brightness of the image and the image brightness threshold, wherein the image brightness threshold is determined based on the average distance of the object; A construction module, configured to construct a first brightness curve for a first object, wherein the first brightness curve represents a relationship between each average distance and a first average brightness; The construction module is further used to construct a second brightness curve for the second object, wherein the second brightness curve represents the relationship between each average distance and the second average brightness; The construction module is further used to construct a brightness separation curve according to the first brightness curve and the second brightness curve, wherein the brightness separation curve represents the relationship between each average distance and the brightness calibration threshold; A generating module is used to construct the high-order polynomial function according to the brightness separation curve, or to train the brightness prediction model according to the brightness separation curve.

14. The device according to claim 13, characterized in that The determination module is specifically used for: If the average brightness of the image is greater than the image brightness threshold, determining that the category recognition result of the object to be recognized is a palm; If the average brightness of the image is less than or equal to the image brightness threshold, it is determined that the category recognition result of the object to be identified is a terminal device.

15. The device according to claim 14, characterized in that The device also includes: a verification module; The acquisition module is further configured to start a fill light if the category recognition result is a palm, and to capture a first image through a color camera, wherein the first image includes an image corresponding to a palm print; The verification module is used to verify the palm print included in the first image; or, The acquisition module is further configured to, if the category recognition result is a terminal device, turn off the fill light and capture a second image through a color camera, wherein the second image includes an image corresponding to the payment code; The verification module is used to verify the payment code included in the second image.

16. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 12 are implemented.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

18. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

Images