Image processing method and device

Abstract

The embodiments of the present specification provide a method and device for image processing, wherein the image processing method includes: inputting at least one acquired image to be detected into an image detection model, obtaining a human body frame and / or a face frame of at least one initial object in the at least one image to be detected, and an initial object identifier corresponding to the human body frame and / or the face frame of at least one initial object; determining an association relationship between initial object identifiers corresponding to the human body frame and / or the face frame of at least one initial object in the at least one image to be detected, and determining at least one target object in the at least one image to be detected based on the association relationship; processing the human body frame and / or the face frame of at least one target object in the at least one image to be detected, and obtaining a human body timing diagram and / or a face timing diagram of at least one target object; inputting the human body timing diagram and / or the face timing diagram of at least one target object into the image prediction model, and determining the target intention of at least one target object.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of computer technology, and in particular to a method for processing an image. Background Art

[0002] Face payment is a new payment method based on facial recognition. The process is incredibly simple. Users don't need to carry a wallet, bank card, or phone. Simply face the screen of the facial recognition device, and the system automatically deducts funds based on the user's facial recognition and personal account information. The entire transaction process is incredibly convenient. However, current facial payment methods require multiple clicks to complete payment, compatibility with advertising and payment functions, resulting in a poor payment experience. Summary of the Invention

[0003] In view of this, the present invention provides an image processing method. One or more embodiments of the present invention also relate to an image processing apparatus, a computing device, and a computer-readable storage medium to address technical deficiencies in the prior art.

[0004] According to a first aspect of an embodiment of this specification, a method for processing an image is provided, including:

[0005] Inputting the acquired at least one image to be detected into an image detection model, obtaining a body frame and / or a face frame of at least one initial object in the at least one image to be detected, and an initial object identifier corresponding to the body frame and / or the face frame of the at least one initial object;

[0006] determining an association relationship between initial object identifiers corresponding to a human body frame and / or a human face frame of at least one initial object in the at least one image to be detected, and determining at least one target object in the at least one image to be detected based on the association relationship;

[0007] Processing a human body frame and / or a human face frame of at least one target object in the at least one image to be detected to obtain a human body timing diagram and / or a human face timing diagram of the at least one target object;

[0008] The human body timing diagram and / or face timing diagram of the at least one target object is input into the image prediction model to determine the target intention of the at least one target object.

[0009] According to a second aspect of the embodiments of this specification, there is provided an image processing apparatus, including:

[0010] a human body detection module configured to input the acquired at least one image to be detected into an image detection model, obtain a human body frame and / or a human face frame of at least one initial object in the at least one image to be detected, and an initial object identifier corresponding to the human body frame and / or human face frame of the at least one initial object;

[0011] a target object determination module configured to determine an association relationship between initial object identifiers corresponding to a human body frame and / or a face frame of at least one initial object in the at least one image to be detected, and determine at least one target object in the at least one image to be detected based on the association relationship;

[0012] a timing diagram generating module, configured to process a human body frame and / or a human face frame of at least one target object in the at least one image to be detected to obtain a human body timing diagram and / or a human face timing diagram of the at least one target object;

[0013] The picture prediction module is configured to input the human body timing diagram and / or face timing diagram of the at least one target object into the picture prediction model to determine the target intention of the at least one target object.

[0014] According to a third aspect of an embodiment of this specification, a computing device is provided, including:

[0015] memory and processor;

[0016] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, wherein the processor implements the steps of the image processing method when executing the computer-executable instructions.

[0017] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of any one of the image processing methods are implemented.

[0018] One embodiment of the present specification inputs at least one acquired picture to be detected into a picture detection model to obtain a human body frame and / or a face frame of at least one initial object in the at least one picture to be detected, and an initial object identifier corresponding to the human body frame and / or the face frame of the at least one initial object; determines an association relationship between the initial object identifiers corresponding to the human body frame and / or the face frame of the at least one initial object in the at least one picture to be detected, and determines at least one target object in the at least one picture to be detected based on the association relationship; processes the human body frame and / or the face frame of the at least one target object in the at least one picture to be detected to obtain a human body timing diagram and / or a face timing diagram of the at least one target object; and inputs the human body timing diagram and / or the face timing diagram of the at least one target object into a picture prediction model to determine the target intention of the at least one target object.

[0019] Specifically, by inputting the collected user body frame timing diagram and / or face frame timing diagram into the image prediction model, it is predicted whether the user has the willingness to pay by face recognition. This allows the user to avoid clicking on the face recognition machine screen. The server can predict the target user who is willing to pay by face recognition based on the real-time video of the user in front of the face recognition machine, thereby reducing the actual user interaction process and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a system architecture diagram of an image processing method provided by an embodiment of this specification applied to a system;

[0021] Figure 2 This is a flowchart of a picture processing method provided by one embodiment of this specification;

[0022] Figure 3 This is a flowchart of a processing process of an image processing method provided by an embodiment of this specification;

[0023] Figure 4 This is a structural diagram of an image processing device provided by an embodiment of this specification;

[0024] Figure 5 This is a structural block diagram of a computing device provided by one embodiment of this specification. DETAILED DESCRIPTION

[0025] The following description sets forth many specific details to facilitate a thorough understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0026] The terms used in one or more embodiments of this specification are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms "a," "the," and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0027] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0028] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0029] First, the terms involved in one or more embodiments of this specification are explained.

[0030] Face recognition device: a device that supports face recognition payment.

[0031] Face recognition: Based on deep learning technology, user identity is confirmed through face recognition.

[0032] Behavior recognition: Based on deep learning technology, user behavior recognition is performed.

[0033] In the current facial recognition payment model, users need to click up to three times to complete payment in order to accommodate both advertising and payment functions, resulting in a poor payment experience. This specification proposes a facial recognition payment solution and device that supports photo payment via a welcome screen. The face recognition device defaults to an advertisement page. When a user approaches the face recognition device and gazes at the screen, the welcome screen predicts the user's willingness to pay and automatically switches to the face recognition mirror page, eliminating multiple clicks and improving the face recognition experience.

[0034] It should be noted that the image processing method provided in the embodiments of this specification can not only be applied to the field of face-swiping payment, but can also be applied to various technical fields that require effective collection of user faces, such as face recognition unlocking and face recognition security inspection. In the embodiments of this specification, the image processing method is explained in detail by taking the scenario of face-swiping payment by the user as an example.

[0035] In this specification, a picture processing method is provided. This specification also relates to a picture processing apparatus, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0036] Figure 1 A system architecture diagram showing an image processing method provided according to an embodiment of this specification applied to a system.

[0037] It should be noted that in order to reduce the number of user interactions during the face-scanning payment process, the image processing method provided in the embodiment of this specification is a welcome prediction algorithm, which predicts the user's willingness to pay based on algorithms such as human body detection, human body tracking, and payment prediction, and automatically completes the page switching function of the face-scanning device.

[0038] Figure 1 The guest welcome prediction system may include the following modules: human body detection, human body tracking and payment prediction, wherein the payment prediction also includes human body sequence, behavior recognition and payment prediction.

[0039] Human detection uses deep learning technology to train a human detection model and locate human and face frames. Human tracking uses deep learning technology to associate IDs between human and face frames in a time series. Payment prediction includes three modules: human sequence, behavior recognition, and payment prediction. Human sequence uses the tracking association results to extract target frames and generate time series graphs. Behavior recognition uses deep learning technology to classify time series graphs. Payment prediction uses the results of behavior recognition to predict the user's willingness to pay. For example, if a user walks towards a facial recognition device from a distance and looks at it, the model will recognize the user's willingness to scan and automatically redirect to the photo payment page.

[0040] The image processing method provided in the embodiments of this specification predicts whether the user is willing to pay by face recognition based on the user video collected in real time. When the user with the willingness to pay is determined, the advertising page of the face recognition machine is automatically jumped to the photo payment page, so as to reduce the number of times the user interacts with the face recognition machine and realize the face recognition payment process.

[0041] Figure 2 A flowchart of an image processing method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0042] It should be noted that in order to reduce the number of interactions between the user and the face recognition machine, the user can jump from the advertising page on the face recognition machine display interface to the shooting mode without clicking on the screen of the face recognition machine when approaching the face recognition machine, and start the face recognition process. Then, the server can obtain multiple frames of pictures of multiple users in real time through the acquisition device of the face recognition machine, and process the multiple frames of pictures to predict the target users who are willing to pay by face recognition in the multiple frames of pictures, which will make the face recognition machine jump directly from the advertising page to the shooting interface; the specific method of how to predict which user has the willingness to pay by face recognition through multiple frames of pictures can refer to the image processing method provided in this embodiment, and its specific implementation method is not limited to the following one method. The following implementation method is used as an example for detailed explanation.

[0043] Step 202: Input the acquired at least one image to be detected into an image detection model to obtain a body frame and / or face frame of at least one initial object in the at least one image to be detected, and an initial object identifier corresponding to the body frame and / or face frame of the at least one initial object.

[0044] The pictures to be detected may be understood as a plurality of pictures collected by a picture acquisition device, or may be understood as each frame of a multi-frame video collected by the picture acquisition device.

[0045] The initial object can be understood as a user appearing in at least one image to be detected. For example, if four users appear in the first frame and three users appear in the second frame, then the different users in each frame can be called the initial object.

[0046] The human body frame can be understood as the human body contour frame corresponding to the initial object in each frame of the picture, and the face frame can be understood as the face contour frame corresponding to the initial object in each frame of the picture.

[0047] During specific implementation, the server can capture multiple frames of video in real time through an image acquisition device. The multiple frames of video can be composed of multiple images, and each frame of image is the image to be detected; the server can input the image to be detected into the image detection model, and then obtain the human body frame and / or face frame of at least one initial object in each frame of the image to be detected, as well as the initial object identifier corresponding to the human body frame and / or face frame corresponding to each initial object.

[0048] In actual applications, the image detection model is a human body detection model pre-trained based on deep learning technology to complete the positioning of the human body frame and / or face frame. In this embodiment, the specific training process of the image detection model will not be described in detail. For example, there are three pictures to be detected, namely, picture 1 to be detected, picture 2 to be detected, and picture 3 to be detected. Then, picture 1 to be detected, picture 2 to be detected, and picture 3 to be detected are respectively input into the picture detection model, and a human body frame and / or face frame of each initial object in at least one picture to be detected, as well as the corresponding initial object identification can be obtained. For example, if there are two initial objects in picture 1 to be detected, the picture detection model outputs two different human body frames and / or face frames, and the initial object identification corresponding to the human body frame and / or face frame of each initial object, that is, two initial object identifications; if there are three initial objects in picture 2 to be detected, the picture detection model outputs three different human body frames and / or face frames, and the initial object identification corresponding to the human body frame and / or face frame of each initial object, that is, three initial object identifications; if there are three initial objects in picture 3 to be detected, the picture detection model outputs three different human body frames and / or face frames, and the initial object identification corresponding to the human body frame and / or face frame of each initial object, that is, three initial object identifications.

[0049] It should be noted that, among the multiple frames of images captured by the image acquisition device, there may be an initial object that is in a state of passing through the image acquisition device. Therefore, in the multiple frames of images captured subsequently, there may be no initial object that has passed through the image acquisition device. That is, in the above example, there will be two initial objects in the image to be detected 1, and three initial objects in the images to be detected 2 and 3.

[0050] Step 204: Determine an association relationship between initial object identifiers corresponding to a human body frame and / or a face frame of the at least one initial object in the at least one image to be detected, and determine at least one target object in the at least one image to be detected based on the association relationship.

[0051] In actual applications, after determining the initial object identifier corresponding to the human body frame and / or face frame of at least one initial object in at least one picture to be detected, the server needs to further determine the association relationship between each initial object in at least one picture to be detected, which can be understood as the association relationship between the two initial object identifiers in picture 1 to be detected and the three initial object identifiers in picture 2 to be detected. In actual scenarios, the two initial objects in picture 1 to be detected stand in the acquisition area of ​​the picture acquisition device, and the three initial objects in picture 2 to be detected also stand in the acquisition area of ​​the picture acquisition device, but the two initial objects in picture 1 to be detected and two of the three initial objects in picture 2 to be detected are the same person, that is, initial object 1 appears in picture 1 to be detected and in picture 2 to be detected at the same time, initial object 2 appears in picture 1 to be detected and in picture 2 to be detected at the same time, and another initial object in picture 2 to be detected appears in picture 2 to be detected but does not appear in picture 1 to be detected. Therefore, the above-mentioned situation of two initial objects in picture 1 to be detected and three initial objects in picture 2 to be detected occurs.

[0052] In addition, the server also needs to determine the association between the two initial object identifiers in the image to be detected 1 and the three initial object identifiers in the image to be detected 3, as well as the association between the three initial object identifiers in the image to be detected 2 and the three initial object identifiers in the image to be detected 3.

[0053] After the server determines the association relationship between the initial object identifiers corresponding to each initial object body frame and / or face frame in at least one picture to be detected, it can determine that all initial objects appearing in at least one picture to be detected are target objects based on the association relationship. For example, there are three target objects in the three pictures to be detected mentioned above, namely, the initial objects with initial object identifiers 1, 2, and 3. Therefore, in subsequent processing, users with initial object identifiers 1, 2, and 3 are used as target objects.

[0054] It should be noted that the target object can be understood as an object determined to be willing to pay by face recognition among multiple initial objects. Therefore, at least one target object can be understood as two or more people among the multiple initial objects who may be predicted to have the willingness to pay by face recognition. In actual applications, after the server determines the association between each initial object, it can determine that at least one of the images to be detected contains the same person and that the person's body frame and / or face frame will be detected in each image. This allows the prediction of whether the person has the actual willingness to pay by face recognition based on the person's body frame and / or face frame.

[0055] Furthermore, the server may determine the association relationship between the initial object identifiers corresponding to the human frame and / or face frame of each initial object using different determination methods. In this embodiment, the association relationship between each initial object identifier is determined by determining similarity as an example, but the method for determining the association relationship is not limited in any way. Specifically, determining the association relationship between the initial object identifiers corresponding to the human frame and / or face frame of at least one initial object in the at least one image to be detected includes:

[0056] Determine the similarity between the human body frame and / or the face frame of the at least one initial object in the at least one to-be-detected picture, and when the similarity satisfies a preset threshold, determine that there is an association relationship between the initial object identifiers corresponding to the human body frame and / or the face frame of the at least one initial object.

[0057] In actual applications, the server can calculate the similarity between each human frame and / or face frame in the process of determining the association relationship between the human body frame and / or face frame of at least one initial object in at least one picture to be detected. Continuing with the above example, the two initial object identifiers in the picture to be detected 1 are initial object identifier 1 and initial object identifier 2; the three initial object identifiers in the picture to be detected 2 are initial object identifier 3, initial object identifier 4, and initial object identifier 5; the three initial object identifiers in the picture to be detected 3 are initial object identifier 6, initial object identifier 7, and initial object identifier 8.

[0058] The similarity between the human body frame and / or face frame of the initial object identifier 1 in the image to be detected 1 and the human body frame and / or face frame of the initial object identifier 3 in the image to be detected 2 can be calculated. When it is determined that the similarity is 90%, that is, greater than the preset threshold value of 70%, it can be determined that the initial object identifier 1 and the initial object identifier 3 have an association relationship, that is, it can be understood that the person corresponding to the initial object identifier 1 and the initial object identifier 3 is the same person. Then, the initial object identifier 3 in the image to be detected 2 can be modified to the initial object identifier 1, that is, the three initial object identifiers in the image to be detected 2 are initial object identifier 1, initial object identifier 4 and initial object identifier 5 respectively.

[0059] By using the above-mentioned method of calculating similarity, the association relationship between each initial object between at least one picture to be detected can be calculated separately to determine whether there is the same initial object in at least one picture to be detected. The specific calculation process of the association relationship with the initial object identifier 1 in the picture to be detected 3 will not be elaborated here. Similarly, the method of determining the association relationship between other initial object identifiers will not be elaborated here, and the above examples can be used as a reference.

[0060] Finally, based on the above method, it can be determined that the two initial object identifiers in the picture to be detected 1 are initial object identifier 1 and initial object identifier 2, the three initial object identifiers in the picture to be detected 2 are initial object identifier 1, initial object identifier 2 and initial object identifier 3, and the three initial object identifiers in the picture to be detected 3 are initial object identifier 1, initial object identifier 2 and initial object identifier 3; that is, in the actual application scenario, there are two identical initial objects in the picture to be detected 1, the picture to be detected 2 and the picture to be detected 3, and there are three identical initial objects in the picture to be detected 2 and the picture to be detected 3.

[0061] The image processing method provided in the embodiments of this specification determines the similarity between the human body frame and / or face frame of each initial object in at least one image to be detected, and determines the association relationship between each initial object identifier. This method can not only quickly determine the initial objects with an associated relationship in each image to be detected, but also facilitates subsequent processing of the human body frame and / or face frame of each initial object.

[0062] Step 206: Processing the human body frame and / or human face frame of at least one target object in the at least one image to be detected to obtain a human body timing diagram and / or human face timing diagram of the at least one target object.

[0063] Among them, the human body timing diagram can be understood as a timing diagram composed of multiple human body frames, and the face frame timing diagram can be understood as a timing diagram composed of multiple face frames.

[0064] In actual applications, after the server obtains at least one target object determined in at least one picture to be detected, each target object in the at least one picture to be detected can be cut out. Continuing with the above example, the object identifiers of the target objects in the picture to be detected 1, the picture to be detected 2, and the picture to be detected 3 are initial object identifier 1, initial object identifier 2, and initial object identifier 3, respectively; further, the target object in the at least one picture to be detected is cut out, and the human body timing diagram and / or face timing diagram of each target object is determined based on the processed results.

[0065] Furthermore, the processing of the human body frame and / or human face frame of at least one target object in the at least one image to be detected to obtain the human body timing diagram and / or human face timing diagram of the at least one target object includes:

[0066] performing a cutout process on a human body frame and / or a human face frame of at least one target object in at least one picture to be detected, to obtain a frame image of the human body frame and / or the human face frame of at least one target object in the at least one picture to be detected;

[0067] The framework images of the human body frame and / or the face frame of the at least one target object that have an associated relationship between the at least one picture to be detected are spliced ​​to obtain the human body timing diagram and / or the face timing diagram of the at least one target object.

[0068] In actual applications, the server can perform cutout processing on the human body frame and / or face frame of the target object determined in at least one picture to be detected, and then obtain a framework diagram of the human body frame and / or face frame of each target object in at least one picture to be detected, and then based on the association relationship of the target objects in at least one picture to be detected, the framework diagrams of the human body frames and / or face frames of the target objects being the same person are spliced ​​to obtain a human body timing diagram and / or face timing diagram of each target object.

[0069] Continuing with the above example, the server finally obtains the human body timing diagram and / or face timing diagram of target object 1 whose initial object identification is 1, the human body timing diagram and / or face timing diagram of target object 2 whose initial object identification is 2, and the human body timing diagram and / or face timing diagram of target object 3 whose initial object identification is 3.

[0070] The image processing method provided in the embodiments of this specification can obtain a human body timing diagram and / or human face timing diagram for each target object by performing cutout processing on the human body frame and / or human face frame of each target object and then splicing the processed frame diagrams, thereby facilitating subsequent guest welcome prediction based on the human body timing diagram and / or human face timing diagram of each target object.

[0071] Step 208: Inputting the human body timing diagram and / or face timing diagram of the at least one target object into the image prediction model to determine the target intention of the at least one target object.

[0072] Among them, in the scenario of face-swiping payment, the target intention can be understood as the target object's face-swiping intention; in the scenario of face-swiping unlocking, the target intention can be understood as the target object's unlocking intention. The meaning of the target intention will be different in different application scenarios. In this embodiment, there are no excessive restrictions on the target intention.

[0073] In actual applications, the server inputs the human body timing diagram and / or face timing diagram of each target object into the image prediction model, classifies the human body timing diagram and / or face timing diagram based on deep learning technology, and predicts whether the user is willing to pay based on the recognition results. For example, if the user walks towards and looks at the face recognition device from a distance, the model will recognize him as a user who is willing to scan his face and automatically jump to the photo payment page.

[0074] It should be noted that the image prediction model is a model that can predict whether a target object has the willingness to pay by face recognition based on the target object's body time sequence diagram and / or face time sequence diagram. Specifically, the image prediction model is pre-trained as follows:

[0075] Using the human body time series training image and / or the human face time series training image of the at least one target object as a training sample, inputting the training sample into a classification model, and outputting sample labels of the human body time series training image and / or the human face time series training image;

[0076] An image prediction model is trained based on the training samples and the sample labels.

[0077] In actual applications, the process of training the image prediction model is to use the human body timing training graph and / or face timing training graph of each target object as a training sample and input it into the classification model, so that the sample labels corresponding to the human body timing training graph and / or face timing training graph can be output. For example, the human body timing training graph and / or face timing training graph of target object 1 is input into the classification model. If the human body timing graph and / or face timing graph of the target object is facing the face recognition device from the front, the sample labels of the human body timing graph and / or face timing graph of such target object can be marked as positive samples. If the human body timing graph and / or face timing graph of the target object is facing the face recognition device from the side, the sample labels of the human body timing graph and / or face timing graph of such target object can be marked as negative samples. Based on this, the image prediction model learns the training samples and the corresponding sample labels, and then trains an image prediction model that can predict the target intention of the human body timing graph and / or face timing graph of the target object.

[0078] It should be noted that if the human body timing diagram or face timing diagram of the target object is input into the image prediction model, it is a single-input and single-output model feature. If the human body timing diagram and face timing diagram of the target object are input into the image prediction model, it is a dual-input and single-output model feature.

[0079] The image processing method provided in the embodiments of this specification trains an image prediction model that can predict the target intention of the target object of the human body timing diagram and / or the human face timing diagram, so that in the subsequent application process, the target intention of the target object can be directly outputted after only the human body timing diagram and / or the human face timing diagram of the target object is input into the image prediction model.

[0080] When it is determined that there are two or more target objects with a target intent, and it is impossible to specifically determine which target object is to perform face-scanning payment for the current transaction order, then the server will further comprehensively consider other location factors to ultimately determine the target object that needs to perform face-scanning payment in the current transaction; specifically, inputting the human body timing diagram and / or face timing diagram of the at least one target object into the image prediction model, and after determining the target intent of the at least one target object, further comprising:

[0081] Determine the location information of each target object within the area captured by the image acquisition device;

[0082] A target processing object is screened out from the at least one target object based on the location information, and a target processing intention of the target processing object is determined.

[0083] Among them, the position information of the target object in the area captured by the image acquisition device can be understood as the target object being within the range captured by the lens of the face recognition machine. The target object is the occupied position information, and the position information can be expressed by coordinates. This embodiment does not make specific limitations on this.

[0084] In actual applications, the server can filter out a final target processing object from two or more target objects based on the location information of the determined target object in the collection area, where the target processing object can understand the object that requires face payment in the current transaction; finally, determine the target processing intention of the target processing object.

[0085] For example, the server can determine two target objects, namely target object A and target object B. In the actual application scenario, it means that target object A and target object B are both facing the face recognition machine, which will cause the face recognition machine to mistakenly recognize that both target objects have the intention to scan their faces. Therefore, the server will also consider the position information occupied by target object A and target object B. If the position occupied by target object A is in the middle of the collected area, and if the position occupied by target object B is in the lower position of the collected area, then in this case, the position information of the two target objects will be comprehensively considered, and target object A in the middle position will be selected from the two target objects as the target processing object, and then the target processing intention of the target processing object will be determined.

[0086] It should be noted that the other factors considered by the server in this embodiment are location factors, but it is not limited to considering only location factors. Other influencing factors can also be considered to determine the final target processing object. This embodiment does not impose too many restrictions on this.

[0087] The image processing method provided in the embodiments of this specification comprehensively considers the location information of each target object to solve the target processing object with real face-swiping intention among two or more target objects, and can accurately predict the target object's face-swiping payment intention to enhance the user experience.

[0088] After the server determines the target processing intent of the target processing object, the advertising page of the face recognition machine can automatically switch to the shooting mode, so as to reduce the user's interaction process with the face recognition machine; specifically, after determining the target processing intent of the target processing object, it also includes:

[0089] Based on the target processing intention of the target processing object, switching the display interface of the image acquisition device to a video recording mode;

[0090] A target face of the target processing object is obtained based on the video recording mode of the display interface.

[0091] In actual applications, the server can switch the facial recognition device's display interface from an advertising page to a recording mode based on the target processing object's target processing intent. The server can then use the facial recognition device's recording mode to capture the target object's face in real time. This stage can be understood as the photo-and-pay phase of facial recognition payment. The facial recognition device interface switches to recording mode, allowing users to see their face in real time and adjust its position based on the interactive text, allowing them to use the facial recognition device to make payments.

[0092] The image processing method provided in the embodiments of this specification can automatically switch to the recording mode and execute the subsequent face-scanning payment process without the user clicking on the face-scanning machine when the server determines the target processing intention of the target processing object.

[0093] After the user adjusts the position, he can click the face recognition instruction in the display interface; specifically, after obtaining the target face of the target processing object based on the recording mode of the display interface, it also includes:

[0094] receiving a face recognition activation instruction sent by the target processing object, and responding to the face recognition activation instruction;

[0095] The target face picture currently being processed by the target processing object is collected by the picture collection device, the target face picture is input into the face recognition model, and when the recognition result meets the resource processing condition, the resource processing confirmation information is returned to the target processing object.

[0096] In actual applications, after receiving the face recognition start instruction sent by the target processing object by clicking the display interface, the server can respond to the face recognition start instruction, that is, the face recognition machine can collect the current target face image of the target processing object, input the target face image into the face recognition model, and start the face recognition process. The identity information of the target processing object can be determined through the face recognition model, and compared with the pre-stored identity information database based on the identity information. When it is determined that there is a matching target processing object, it can be indicated that the recognition result meets the resource processing conditions, and the confirmation information of the face payment transaction can be sent to the target processing object. It should be noted that this embodiment takes the face recognition model as an example for explanation. In the process of face recognition, other methods can also be used, and this embodiment does not limit this.

[0097] The image processing method provided in the embodiments of this specification uses a face recognition model to enable recognition of the target face obtained by the target processing object, thereby facilitating the subsequent face payment process based on the face recognition results.

[0098] After the server returns the resource processing confirmation information to the face recognition device and displays it to the target processing object, it may also receive a confirmation instruction from the user; specifically, after returning the resource processing confirmation information to the target processing object, it also includes:

[0099] A resource processing instruction is received from the target processing object based on the resource processing confirmation information, and resources of the target processing object are processed based on the account information carried in the resource processing instruction.

[0100] In actual applications, after the user sees the resource processing confirmation information of the current transaction on the face recognition machine, after confirming that the information is correct, he can click the confirmation control in the face recognition machine display interface to send the resource processing instruction to the server. After receiving the resource processing instruction sent by the user, the server can process the personal account resources of the target processing object according to the account information carried in the resource processing instruction.

[0101] The image processing method provided in the embodiments of this specification can directly process the user's personal account resources through the payment confirmation instruction clicked by the user, which not only enables a convenient transaction process, but also reduces the number of interactions between the user and the face-scanning machine during the entire face-scanning payment process, thereby enhancing the user's transaction experience.

[0102] In summary, the image processing method provided in the embodiments of this specification proposes a welcome prediction algorithm, which predicts the user's willingness to pay based on algorithms such as human body detection, human body tracking, and payment prediction, and automatically completes the page switching function of the face-scanning device, reducing the number of interactions between the user and the face-scanning machine, and improving the user's face-scanning payment transaction experience.

[0103] The following combined Figure 3 , taking the application of the image processing method provided in this specification in face payment as an example, the image processing method is further explained. Figure 3 A flowchart of a processing process of an image processing method provided by an embodiment of this specification is shown, which specifically includes the following steps.

[0104] It should be noted that the welcome prediction algorithm proposed in this embodiment can predict whether the user in front of the face recognition machine has the intention to pay by face recognition. If there is a user who is willing to pay by face recognition, the advertising page of the face recognition machine can be automatically switched to shooting mode to execute the subsequent face recognition payment process.

[0105] Step 302: Advertisement standby.

[0106] Specifically, the default screen of the face-scanning device is an advertising page, which is convenient for business operations.

[0107] Step 304: guest welcome prediction.

[0108] Specifically, when a user approaches a facial recognition device to scan their face, the welcome mode automatically switches from the advertising page to the photo-and-pay page. The welcome mode includes three modules: human detection, human tracking, and payment prediction.

[0109] Human detection uses deep learning technology to train a human detection model to locate human and face frames. Human tracking uses deep learning technology to associate IDs between human and face frames over time.

[0110] Payment prediction includes three modules: human body sequence, behavior recognition, and payment prediction. Human body sequence generates target bounding boxes and time sequence graphs based on tracking association results. Behavior recognition classifies time sequence graphs using deep learning technology. Payment prediction uses behavior recognition results to predict whether a user is willing to pay. For example, if a user walks towards a facial recognition device from a distance and looks at it, the model will identify the user's willingness to scan and automatically redirect to the photo payment page.

[0111] Step 306: Take a photo and pay.

[0112] Specifically, the face recognition device page switches to recording mode, and users can see their faces in real time and adjust their positions according to the interactive text to cooperate with face recognition.

[0113] Step 308: Click Face Scan.

[0114] Specifically, after the user adjusts the position, he clicks on face scan to start the face recognition process.

[0115] Step 310: Face recognition.

[0116] Specifically, face recognition and payment are performed based on the face at the time of taking the photo.

[0117] Step 312: Confirm payment.

[0118] Specifically, the user's avatar and Alipay account number will pop up on the page, asking the user to click again to confirm payment and complete the payment.

[0119] The image processing method provided in this embodiment proposes a welcome prediction algorithm, which predicts the user's willingness to pay based on algorithms such as human body detection, human body tracking, and payment prediction, and automatically completes the page switching function of the face-scanning device, reducing the number of interactions between the user and the face-scanning machine and improving the user's face-scanning payment transaction experience.

[0120] Corresponding to the above method embodiment, this specification also provides an image processing device embodiment, Figure 4 FIG. 1 shows a schematic diagram of the structure of an image processing device provided by an embodiment of this specification. Figure 4 As shown, the device includes:

[0121] The human body detection module 402 is configured to input the acquired at least one image to be detected into an image detection model, obtain a human body frame and / or a human face frame of at least one initial object in the at least one image to be detected, and obtain an initial object identifier corresponding to the human body frame and / or human face frame of the at least one initial object;

[0122] a target object determination module 404 configured to determine an association relationship between initial object identifiers corresponding to a human body frame and / or a human face frame of at least one initial object in the at least one image to be detected, and determine at least one target object in the at least one image to be detected based on the association relationship;

[0123] A timing diagram generating module 406 is configured to process a human body frame and / or a human face frame of at least one target object in the at least one image to be detected to obtain a human body timing diagram and / or a human face timing diagram of the at least one target object;

[0124] The image prediction module 408 is configured to input the human body timing diagram and / or face timing diagram of the at least one target object into the image prediction model to determine the target intention of the at least one target object.

[0125] Optionally, the target object determination module 404 is further configured to:

[0126] Determine the similarity between the human body frame and / or the face frame of the at least one initial object in the at least one to-be-detected picture, and when the similarity satisfies a preset threshold, determine that there is an association relationship between the initial object identifiers corresponding to the human body frame and / or the face frame of the at least one initial object.

[0127] Optionally, the timing diagram generating module 406 is further configured to:

[0128] performing a cutout process on a human body frame and / or a human face frame of at least one target object in at least one picture to be detected, to obtain a frame image of the human body frame and / or the human face frame of at least one target object in the at least one picture to be detected;

[0129] The framework images of the human body frame and / or the face frame of the at least one target object that have an associated relationship between the at least one picture to be detected are spliced ​​to obtain the human body timing diagram and / or the face timing diagram of the at least one target object.

[0130] Optionally, the device further includes:

[0131] a position determination module configured to determine position information of each target object within the area captured by the image capture device;

[0132] The intention determination module is configured to filter out a target processing object from the at least one target object based on the location information and determine the target processing intention of the target processing object.

[0133] Optionally, the device further includes:

[0134] a switching module configured to switch the display interface of the image acquisition device to a video recording mode based on the target processing intention of the target processing object;

[0135] The face acquisition module is configured to acquire the target face of the target processing object based on the video recording mode of the display interface.

[0136] Optionally, the device further includes:

[0137] an instruction receiving module, configured to receive a face recognition start instruction sent by the target processing object and respond to the face recognition start instruction;

[0138] The information return module is configured to collect the current target face image of the target processing object through the image acquisition device, input the target face image into the face recognition model, and return resource processing confirmation information to the target processing object when the recognition result meets the resource processing conditions.

[0139] Optionally, the device further includes:

[0140] The resource processing module is configured to receive a resource processing instruction sent by the target processing object based on the resource processing confirmation information, and process the resources of the target processing object based on the account information carried in the resource processing instruction.

[0141] Optionally, the device further includes:

[0142] a model training module configured to use the human body time series training graph and / or the human face time series training graph of the at least one target object as training samples, input them into a classification model, and output sample labels of the human body time series training graph and / or the human face time series training graph;

[0143] An image prediction model is trained based on the training samples and the sample labels.

[0144] The image processing device provided in the embodiments of this specification inputs the collected user body frame timing diagram and / or face frame timing diagram into the image prediction model to predict whether the user has the willingness to pay by face recognition. This allows the user to avoid clicking on the face recognition machine screen. The server can predict the target user who is willing to pay by face recognition based on the real-time video of the user in front of the face recognition machine, thereby reducing the actual user interaction process and improving the user experience.

[0145] The above is a schematic diagram of an image processing device according to this embodiment. It should be noted that the technical solution of the image processing device and the technical solution of the above-mentioned image processing method are based on the same concept. For details not described in detail in the technical solution of the image processing device, please refer to the description of the technical solution of the above-mentioned image processing method.

[0146] Figure 5 The block diagram of a computing device 500 according to one embodiment of the present disclosure is shown. Components of the computing device 500 include, but are not limited to, a memory 510 and a processor 520. The processor 520 is connected to the memory 510 via a bus 530, and a database 550 is used to store data.

[0147] The computing device 500 also includes an access device 540 that enables the computing device 500 to communicate via one or more networks 560. Examples of such networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 540 may include one or more of any type of network interface (e.g., a network interface card (NIC)), whether wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0148] In one embodiment of the present specification, the above components of the computing device 500 and Figure 5 Other components not shown in the figure may also be connected to each other, for example, via a bus. Figure 5 The computing device structure block diagram shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art may add or replace other components as needed.

[0149] Computing device 500 can be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, personal digital assistant, laptop computer, notebook computer, netbook computer, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smartwatch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. Computing device 500 can also be a mobile or stationary server.

[0150] The processor 520 is configured to execute the following computer-executable instructions, which implement the steps of the above-mentioned image processing method when executed by the processor.

[0151] The above is a schematic diagram of a computing device according to this embodiment. It should be noted that the technical solution of the computing device and the technical solution of the above-mentioned image processing method are based on the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the above-mentioned image processing method.

[0152] An embodiment of the present specification further provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the above-mentioned image processing method.

[0153] The above is a schematic diagram of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of the storage medium and the technical solution of the above-mentioned image processing method are based on the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the above-mentioned image processing method.

[0154] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0155] The computer instructions include computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. It should be noted that the content of the computer-readable medium may be appropriately increased or decreased based on the requirements of legislation and patent practice within a jurisdiction. For example, in some jurisdictions, based on legislation and patent practice, computer-readable media does not include electric carrier signals and telecommunication signals.

[0156] It should be noted that for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0157] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0158] The preferred embodiments disclosed above are intended only to help illustrate this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made based on the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that those skilled in the art can better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A method for processing an image, comprising: Inputting the acquired at least one to-be-detected picture into the picture detection model, obtaining a human body frame and / or a human face frame of at least one initial object in the at least one to-be-detected picture, and an initial object identifier corresponding to the human body frame and / or the human face frame of the at least one initial object, wherein the at least one to-be-detected picture is acquired by a picture acquisition device; Determine an association relationship between initial object identifiers corresponding to a human body frame and / or a face frame of at least one initial object in the at least one picture to be detected, and determine at least one target object in the at least one picture to be detected based on the association relationship; Processing a human body frame and / or a human face frame of at least one target object in the at least one image to be detected to obtain a human body timing diagram and / or a human face timing diagram of the at least one target object; The human body timing diagram and / or face timing diagram of the at least one target object is input into the image prediction model, the target intention of the at least one target object is determined, and according to the target intention, the image acquisition device is jumped to the shooting interface, and the shooting interface is used to execute the subsequent face-swiping payment process. The target intention is the face-swiping intention of the target object.

2. The image processing method according to claim 1, wherein determining the association relationship between the initial object identifiers corresponding to the human body frame and / or face frame of the at least one initial object in the at least one image to be detected comprises: Determine the similarity between the human body frame and / or the face frame of the at least one initial object in the at least one picture to be detected, and when the similarity meets a preset threshold, determine that there is an association relationship between the initial object identifiers corresponding to the human body frame and / or the face frame of the at least one initial object.

3. The image processing method according to claim 1, wherein the processing of the human body frame and / or face frame of at least one target object in the at least one image to be detected to obtain the human body timing diagram and / or face timing diagram of the at least one target object comprises: Performing cutout processing on a human body frame and / or a human face frame of at least one target object in at least one picture to be detected, to obtain a frame image of a human body frame and / or a human face frame of at least one target object in the at least one picture to be detected; The framework images of the human body frame and / or face frame of the at least one target object that have an associated relationship between the at least one image to be detected are spliced ​​to obtain the human body timing image and / or face timing image of the at least one target object.

4. The image processing method according to claim 3, after inputting the human body timing diagram and / or face timing diagram of the at least one target object into the image prediction model and determining the target intention of the at least one target object, further comprising: Determine the location information of each target object within the area captured by the image acquisition device; A target processing object is screened out from the at least one target object based on the location information, and a target processing intention of the target processing object is determined, where the target processing intention is the face recognition payment intention of the target processing object.

5. The image processing method according to claim 4, after determining the target processing intention of the target processing object, further comprising: Based on the target processing intention of the target processing object, switching the display interface of the image acquisition device to a video recording mode; A target face of the target processing object is obtained based on the video recording mode of the display interface.

6. The image processing method according to claim 5, after acquiring the target face of the target processing object based on the video recording mode of the display interface, further comprising: Receiving a face recognition start instruction sent by the target processing object, and responding to the face recognition start instruction; The current target face image of the target processing object is collected by the image collection device, the target face image is input into the face recognition model, and when the recognition result meets the resource processing condition, the resource processing confirmation information is returned to the target processing object.

7. The image processing method according to claim 6, after returning the resource processing confirmation information to the target processing object, further comprising: A resource processing instruction sent by the target processing object based on the resource processing confirmation information is received, and resources of the target processing object are processed based on the account information carried in the resource processing instruction.

8. According to the image processing method according to any one of claims 1 to 3, the pre-training method of the image prediction model is as follows: Using the human body time series training graph and / or the human face time series training graph of the at least one target object as training samples, inputting them into a classification model, and outputting sample labels of the human body time series training graph and / or the human face time series training graph; A picture prediction model is trained based on the training samples and the sample labels.

9. A picture processing device, comprising: The human body detection module is configured to input the acquired at least one to-be-detected picture into the picture detection model, obtain the human body frame and / or human face frame of at least one initial object in the at least one to-be-detected picture, and the initial object identifier corresponding to the human body frame and / or human face frame of the at least one initial object, wherein the at least one to-be-detected picture is acquired by the picture acquisition device; a target object determination module, configured to determine an association relationship between initial object identifiers corresponding to a human body frame and / or a face frame of at least one initial object in the at least one image to be detected, and determine at least one target object in the at least one image to be detected based on the association relationship; A timing diagram generating module is configured to process a human body frame and / or a human face frame of at least one target object in the at least one image to be detected to obtain a human body timing diagram and / or a human face timing diagram of the at least one target object; The image prediction module is configured to input the human body timing diagram and / or face timing diagram of the at least one target object into the image prediction model, determine the target intention of the at least one target object, and according to the target intention, jump the image acquisition device to the shooting interface, and the shooting interface is used to execute the subsequent face-swiping payment process, and the target intention is the face-swiping intention of the target object.

10. According to the image processing device of claim 9, the target object determination module is further configured to determine the similarity between the human body frame and / or face frame of the at least one initial object in the at least one image to be detected, and when the similarity meets a preset threshold, determine that there is an association relationship between the initial object identifiers corresponding to the human body frame and / or face frame of the at least one initial object.

11. The image processing device according to claim 9, wherein the timing diagram generating module is further configured to perform a cutout process on a human frame and / or a face frame of at least one target object in at least one image to be detected, so as to obtain a frame diagram of the human frame and / or the face frame of at least one target object in the at least one image to be detected; The framework images of the human body frame and / or face frame of the at least one target object that have an associated relationship between the at least one image to be detected are spliced ​​to obtain the human body timing image and / or face timing image of the at least one target object.

12. The image processing device according to claim 11, further comprising: A position determination module is configured to determine the position information of each target object within the area captured by the image acquisition device; The intention determination module is configured to filter out a target processing object from the at least one target object based on the location information, and determine the target processing intention of the target processing object, wherein the target processing intention is the face-swiping payment intention of the target processing object.

13. The image processing device according to claim 12, further comprising: A switching module, configured to switch the display interface of the image acquisition device to a video recording mode based on the target processing intention of the target processing object; The face acquisition module is configured to acquire the target face of the target processing object based on the recording mode of the display interface.

14. A computing device comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the image processing method described in any one of claims 1 to 8 are implemented.

15. A computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the image processing method according to any one of claims 1 to 8.

Images