Terminal camera pose self-adaptive adjustment method and device and financial self-service terminal

Abstract

The invention provides a terminal camera pose self-adaptive adjustment method and device and a financial self-service terminal, and the method comprises the steps: recording the current pose of a camera under the condition that a target activation signal is monitored, and obtaining the initial position information; controlling a camera to shoot to obtain a plurality of first target images, and based on the first target images, performing classification and recognition through a face detection model to obtain a first recognition result; when the first recognition result is that the camera monitors the face image, determining an offset parameter of the face image based on the first recognition result, performing face tracking on the user based on the offset parameter and controlling the camera to shoot to obtain a second target image, and completing a business process according to the second target image; and under the condition that the target standby signal is monitored, controlling the camera to enter a standby state and performing position recovery. According to the method, the problem that a salesman consumes a long time to handle the process due to the adoption of a fixed-angle camera or a manually adjustable camera in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the field of image acquisition technology, and in particular to a method and device for adaptively adjusting the posture of a terminal camera, a computer-readable storage medium, and a financial self-service terminal. Background Art

[0002] In the field of smart finance and smart government affairs that provide convenient services to customers, there are more and more large-scale self-service smart terminals equipped with face recognition cameras and video cameras. However, the camera modules of these self-service smart terminals are currently composed of fixed angles and manually adjustable modules. When customers of different heights use self-service devices to handle business without the guidance of guides, even if the terminal equipment has detailed voice and animation guidance, customers still need to spend a lot of time adjusting their bodies and accurately aligning their heads and faces with the camera's recognition frame to perform image acquisition and identity comparison required by the business. When some customers are too tall or too short, they even need to manually adjust the camera's orientation angle to meet the recognition requirements, which makes the business process more cumbersome and consumes a lot of unnecessary adjustment time.

[0003] In summary, the camera position adjustment method in the prior art requires customers to adjust the position or manually adjust the camera's posture to achieve the customer's business processing, which is relatively cumbersome camera adjustment and consumes a lot of unnecessary adjustment time. Summary of the invention

[0004] The main purpose of the present application is to provide a terminal camera posture adaptive adjustment method, device, computer-readable storage medium and financial self-service terminal, so as to at least solve the problem in the prior art that the use of fixed-angle cameras or manually adjustable cameras causes the salesperson to spend a long time in the processing process.

[0005] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a method for adaptively adjusting the position and posture of a terminal camera is provided, comprising: when a target activation signal is detected, recording the current position and posture of the camera to obtain initial position information, wherein the target activation signal is used to instruct the camera to exit the standby state; controlling the camera to shoot with the initial position information to obtain multiple first target images, classifying and identifying the first target images through a face detection model to obtain a first recognition result, wherein the first recognition result is at least used to characterize whether the camera has detected the user's face image; when the first recognition result is that the camera has detected the face image, determining the offset parameter of the face image based on the first recognition result, tracking the user's face based on the offset parameter so that the face image is located at the center of the image, controlling the camera to shoot to obtain a second target image, and completing the business process according to the second target image; when a target standby signal is detected, controlling the camera to enter the standby state and restore the position according to the initial position information, wherein the target standby signal is used to instruct the camera to enter the standby state.

[0006] Optionally, the terminal includes an operating screen, a distance sensor, a light intensity sensor and a temperature sensor. Before recording the current posture of the camera, the method also includes: generating a first signal when the distance sensor detects that the distance between the user and the operating screen is less than or equal to a preset distance and lasts for a first preset time; generating a second signal when the temperature sensor detects that the temperature in a preset direction of the operating screen exceeds a preset temperature and lasts for a second preset time; generating a third signal when the light intensity sensor detects that the light intensity in a preset direction of the operating screen is lower than a preset light intensity and lasts for a third preset time; generating the target activation signal when the first signal, the second signal and the third signal are detected at the same time.

[0007] Optionally, before classifying and identifying the first target image through a face detection model, the method also includes: acquiring the face images of the user at different angles to obtain a first data set; preprocessing based on the sample data in the first data set to obtain a second data set, the preprocessing at least including flipping, scaling and rotation; training a deep learning model based on a convolutional neural network through a stochastic gradient descent method based on the second data set, and correcting the training process based on a momentum term to obtain the face detection model, and the face detection model is used to detect the face image according to the input image.

[0008] Optionally, classification and recognition are performed based on the first target image through a face detection model to obtain a first recognition result, including: extracting features of the first target image based on a convolutional neural network to obtain target features, sliding a window based on the target features to obtain multiple candidate areas, where the candidate areas are areas where the face image may exist; performing classification and bounding box regression in turn based on each of the candidate areas; obtaining a face detection bounding box, and determining the face detection bounding box as the first recognition result.

[0009] Optionally, the offset parameters of the facial image are determined based on the first recognition result, and the face of the user is tracked based on the offset parameters so that the facial image is located at the center of the image, including: taking the facial image in the first recognition result as a positive sample, and taking the area other than the facial image in the first recognition result as a negative sample, training a classifier through a machine learning method to obtain a target classifier; calculating the offset parameters of the facial image through the classifier based on the first recognition results corresponding to the first target image in two adjacent frames; decomposing based on the offset parameters to obtain the pitch angle adjustment amount and yaw angle adjustment amount of the camera; and adjusting the posture of the camera based on the pitch angle adjustment amount and the yaw angle adjustment amount until the facial image is located at the center of the image.

[0010] Optionally, the method also includes: when the first recognition result is that the camera does not detect the facial image, storing the corresponding first target image, obtaining a second target image, and recording the corresponding initial position information; when face tracking fails, recording the first target image during the face tracking process, obtaining a third target image, and recording the posture changes of the camera during the face tracking process to obtain target parameters; training the face detection model based on the second target image and the initial position information; and training the classifier based on the third target image and the target parameters.

[0011] Optionally, after completing the business process according to the second target image, the method further includes: recording the posture of the camera after tracking the user's face based on the offset parameters so that the face image is located at the center of the image, and obtaining alternative position information; at every preset time interval, updating the initial position information based on the alternative position information with the highest corresponding frequency.

[0012] According to another aspect of the present application, a terminal camera posture adaptive adjustment device is provided, the device comprising: a first storage unit, for recording the current posture of the camera to obtain initial position information when a target activation signal is detected, the target activation signal is used to instruct the camera to exit the standby state; a first control unit, for controlling the camera to shoot with the initial position information to obtain multiple first target images, and classifying and identifying the first target images through a face detection model to obtain a first recognition result, the first recognition result is at least used to characterize whether the camera has detected the user's face image; a second control unit, for determining an offset parameter of the face image based on the first recognition result when the first recognition result is that the camera has detected the face image, tracking the user's face based on the offset parameter so that the face image is located at the center of the image, and controlling the camera to shoot to obtain a second target image, and completing the business process according to the second target image; a third control unit, for controlling the camera to enter the standby state and restore the position according to the initial position information when a target standby signal is detected, the target standby signal is used to instruct the camera to enter the standby state.

[0013] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute any one of the methods described.

[0014] According to another aspect of the present application, a financial self-service terminal is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include methods for executing any one of the methods described.

[0015] Applying the technical solution of the present application, in the above-mentioned terminal camera posture adaptive adjustment method, first, when a target activation signal is detected, the current posture of the camera is recorded to obtain initial position information, and the above-mentioned target activation signal is used to instruct the camera to exit the standby state; then, the camera is controlled to shoot with the above-mentioned initial position information to obtain multiple first target images, and the face detection model is used to classify and identify the first target images to obtain a first recognition result, and the above-mentioned first recognition result is at least used to characterize whether the camera has detected the user's face image; thereafter, when the above-mentioned first recognition result is that the camera has detected the face image, the offset parameter of the face image is determined based on the above-mentioned first recognition result, the user's face is tracked based on the above-mentioned offset parameter so that the face image is located at the center of the image, and the camera is controlled to shoot to obtain a second target image, and the business process is completed according to the above-mentioned second target image; finally, when a target standby signal is detected, the camera is controlled to enter the standby state and restore the position according to the above-mentioned initial position information, and the above-mentioned target standby signal is used to instruct the camera to enter the standby state. The present application provides a camera adaptive adjustment method, which performs face recognition and face tracking based on computer vision and machine learning technology, and realizes automatic camera tracking of users, thereby avoiding the trouble of users adjusting their own positions to complete image acquisition during the business processing flow. This method solves the problem in the prior art that the use of fixed-angle cameras or manually adjustable cameras causes the business staff to consume a long time in the processing flow. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A hardware structure block diagram of a mobile terminal according to a terminal camera posture adaptive adjustment method provided in an embodiment of the present application is shown;

[0017] Figure 2 A schematic diagram of a process of adaptively adjusting the position and posture of a terminal camera provided in accordance with an embodiment of the present application is shown;

[0018] Figure 3 A structural block diagram of a terminal camera posture adaptive adjustment device provided according to an embodiment of the present application is shown.

[0019] The above drawings include the following reference numerals:

[0020] 102, processor; 104, memory; 106, transmission device; 108, input and output devices. DETAILED DESCRIPTION

[0021] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0022] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0023] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0024] As introduced in the background technology, the camera position adjustment method in the prior art requires the customer to adjust the position, or manually adjust the camera posture to realize the customer's business processing, which is relatively cumbersome camera adjustment and consumes a lot of unnecessary adjustment time. In order to solve the problem that the use of fixed-angle cameras or manually adjustable cameras in the prior art causes the salesperson's processing process to consume a long time, the embodiments of the present application provide a terminal camera posture adaptive adjustment method, device, computer-readable storage medium and financial self-service terminal.

[0025] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0026] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 1 is a hardware structure block diagram of a mobile terminal of a terminal camera posture adaptive adjustment method according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0027] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the display method of device information in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The transmission device 106 is used to receive or send data via a network. The above-mentioned specific examples of the network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0028] In this embodiment, a method for adaptively adjusting the posture of a terminal camera running on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0029] Figure 2 : is a flow chart of a method for adaptively adjusting the position and posture of a terminal camera according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0030] Step S201, when a target activation signal is detected, the current position of the camera is recorded to obtain initial position information, wherein the target activation signal is used to instruct the camera to exit the standby state;

[0031] Specifically, the present application is configured to monitor the user's position to determine whether to generate the target activation signal to turn on the camera. The adaptive power-on / off mechanism saves energy and reduces the speed of camera wear. Furthermore, when the camera is activated, the position of the camera before adjustment is recorded to obtain the initial position information.

[0032] Step S202, controlling the camera to shoot with the initial position information to obtain a plurality of first target images, and classifying and identifying the first target images through a face detection model to obtain a first recognition result, wherein the first recognition result is at least used to indicate whether the camera has detected the user's face image;

[0033] Specifically, when the camera is located at the initial position, the camera is controlled to shoot, and a face detection model is used to detect based on the image to determine whether there is a customer within the camera's field of view, thereby obtaining the first recognition result.

[0034] Step S203, when the first recognition result is that the camera detects the face image, determine the offset parameter of the face image based on the first recognition result, track the face of the user based on the offset parameter so that the face image is located at the center of the image, control the camera to shoot, obtain a second target image, and complete the business process according to the second target image;

[0035] Specifically, when the camera recognizes the customer at the initial position, the offset between the user's face and the image center is calibrated based on the first recognition result to obtain the offset parameters, and then the camera's posture is adjusted through an image tracking algorithm based on the offset parameters to realize automatic tracking of the user's face by the camera in the business process.

[0036] Step S204, when a target standby signal is detected, the camera is controlled to enter a standby state and the position is restored according to the initial position information. The target standby signal is used to instruct the camera to enter a standby state.

[0037] Specifically, after detecting that the customer has left, the target standby signal is generated, the camera is controlled to be turned off, and the camera is restored to its initial position, waiting for the next business process.

[0038] Through this embodiment, first, in the case of detecting a target activation signal, the current position of the camera is recorded to obtain initial position information, and the target activation signal is used to instruct the camera to exit the standby state; then, the camera is controlled to shoot with the initial position information to obtain multiple first target images, and the first target images are classified and identified through the face detection model to obtain a first recognition result, and the first recognition result is at least used to characterize whether the camera has detected the user's face image; thereafter, in the case that the first recognition result is that the camera has detected the face image, the offset parameter of the face image is determined based on the first recognition result, the user's face is tracked based on the offset parameter so that the face image is located at the center of the image, and the camera is controlled to shoot to obtain a second target image, and the business process is completed according to the second target image; finally, in the case of detecting a target standby signal, the camera is controlled to enter the standby state and restore the position according to the initial position information, and the target standby signal is used to instruct the camera to enter the standby state. The present application provides a camera adaptive adjustment method, which performs face recognition and face tracking based on computer vision and machine learning technology, and realizes automatic camera tracking of users, thereby avoiding the trouble of users adjusting their own positions to complete image acquisition during the business processing flow. This method solves the problem in the prior art that the use of fixed-angle cameras or manually adjustable cameras causes the business staff to consume a long time in the processing flow.

[0039] In order to monitor whether there is a customer who needs to handle business in front of the terminal, in an optional implementation, before recording the current position of the camera, the method further includes:

[0040] Step S301, generating a first signal when the distance sensor detects that the distance between the user and the operation screen is less than or equal to a preset distance and lasts for a first preset time period;

[0041] It is understandable that in order to ensure the accuracy of the monitoring results, the present application is configured to sense whether there is a user in a preset area through a combination of sensors of the terminal.

[0042] In one embodiment, the sensor combination includes a distance sensor, a light intensity sensor, and a temperature sensor.

[0043] In a specific implementation, when the distance sensor located on the same side of the operation screen detects that the customer has arrived at a preset distance in front of the terminal and continues for the first preset time, the user's arrival is determined based on the distance and the first signal is generated. Optionally, the preset distance is ≤40cm and the first preset time is 2s.

[0044] Step S302, generating a second signal when the temperature sensor detects that the temperature in the preset direction of the operation screen exceeds the preset temperature and lasts for a second preset time period;

[0045] In a specific implementation, when the temperature sensor located on the upper side of the operation screen detects that the temperature in the preset direction begins to rise and lasts for the second preset time, the arrival of the customer is determined based on the temperature perception, and the second signal is generated. Optionally, the preset direction is the front of the operation screen, and the second preset time is 2s.

[0046] Step S303, generating a third signal when the light intensity sensor detects that the light intensity in the preset direction of the operation screen is lower than the preset light intensity and lasts for a third preset time period;

[0047] In a specific implementation, when the light intensity sensor located at the lower side of the operation screen detects that the brightness in front is reduced due to the occlusion of the customer's body and lasts for the third preset time, the customer's arrival is determined based on the light intensity perception, and the third signal is generated. Optionally, the preset direction is the front of the operation screen, and the third preset time is 2s.

[0048] Step S304, generating the target activation signal when the first signal, the second signal and the third signal are monitored simultaneously.

[0049] Specifically, when it is determined that there is a customer in the preset area based on distance perception, temperature perception, and light intensity perception, the arrival of the customer is determined, the target activation signal is generated, the camera is activated, and angle adjustment is prepared.

[0050] In order to train the face detection model, in an optional implementation, before classifying and identifying the first target image through the face detection model, the method further includes:

[0051] Step S401, acquiring the above-mentioned facial images of the user at different angles to obtain a first data set;

[0052] Specifically, based on the deep learning algorithm, the customer's facial information is identified. Data preparation is first performed before model training. The main and backup data include face images at different angles such as the front, side, front to side, looking up, and looking down, as the initial data set. The above-mentioned first data set is obtained to ensure the applicability of the face detection model.

[0053] Step S402, preprocessing the sample data in the first data set to obtain a second data set, wherein the preprocessing at least includes flipping, scaling and rotating;

[0054] Specifically, based on the above-mentioned first data set, data enhancement means such as flipping, rotating, scaling, etc. are used to further expand the diversity of the above-mentioned first data set to obtain the above-mentioned second data set.

[0055] Step S403, based on the above-mentioned second data set, the deep learning model based on the convolutional neural network is trained by the stochastic gradient descent method, and the training process is corrected based on the momentum term to obtain the above-mentioned face detection model, and the above-mentioned face detection model is used to detect the above-mentioned face image according to the input image.

[0056] Specifically, the above-mentioned face detection model of the present application is designed based on the TensorFlow framework and convolutional neural network. In the specific implementation, a deep learning model is first built based on the convolutional neural network (CNN). When training the model, the stochastic gradient descent method is used to calculate the gradient of the loss function relative to the model. At the same time, in order to solve the problem of small errors in the stochastic gradient descent method, the present application introduces a momentum term to accelerate the convergence process, and uses a predefined learning rate to adjust the learning strategy.

[0057] It can be understood that in the above process, a small batch of samples is used each time to calculate the gradient, and after multiple rounds of training, the prediction work can be completed accurately and quickly, and the above face detection model is obtained.

[0058] In order to obtain the first recognition result, in an optional implementation, the step S203 includes:

[0059] Step S2021, extracting features of the first target image based on a convolutional neural network to obtain target features, sliding a window based on the target features to obtain multiple candidate regions, where the candidate regions are regions where the facial image may exist;

[0060] Specifically, after the above-mentioned face detection model is deployed, images are collected by a camera at preset intervals to obtain the above-mentioned first target image, and the above-mentioned first target image is input into the above-mentioned face detection model, and features are extracted through the model to obtain the above-mentioned target features, and based on the target features of the image, candidate areas are generated by sliding the window at different positions of the image to obtain the above-mentioned candidate areas.

[0061] Step S2022, classifying and regressing the bounding boxes based on the candidate regions in sequence; obtaining a face detection bounding box, and determining the face detection bounding box as the first recognition result.

[0062] Specifically, based on the candidate areas, each candidate area is classified (whether it is a face image), and based on the classification, bounding box regression is performed (by fine-tuning the position and size of the bounding box, the face is framed in a smaller range to make the recognition result more accurate), to obtain the above-mentioned first recognition result.

[0063] It is understandable that if no facial image is detected in multiple consecutive images, it is determined that the customer's position in the current preset area cannot be recognized. At this time, a voice announcement is made to prompt the customer to adjust the position and move the face as close to the camera as possible, and the first target image is captured again until recognition is successful.

[0064] In order to perform face tracking, in an optional implementation, the above step S203 includes:

[0065] Step S2031, using the face image in the first recognition result as a positive sample and the area other than the face image in the first recognition result as a negative sample, training a classifier by a machine learning method to obtain a target classifier;

[0066] Specifically, the present application sets up a classifier for face tracking, specifically, using a face image as a positive sample and a corresponding background area as a negative sample, training a classifier that can support parameter adjustment, and obtaining the above-mentioned target classifier.

[0067] Step S2032, calculating the offset parameter of the face image through the classifier based on the first recognition results corresponding to the first target images of two adjacent frames;

[0068] Specifically, based on the above-mentioned target classifier, a classifier is used to calculate the change value of the face image in the above-mentioned first recognition result at the current moment and the face image in the above-mentioned first recognition result at the previous image acquisition moment, that is, the above-mentioned offset parameter is determined according to the relative position offset of the face in two consecutive images.

[0069] Step S2033, performing decomposition based on the above offset parameters to obtain a pitch angle adjustment amount and a yaw angle adjustment amount of the above camera;

[0070] Specifically, the above offset parameters are decomposed based on the space coordinate system, that is, the above offset parameters are decomposed into the pitch angle and the yaw angle respectively, so as to determine the above pitch angle adjustment amount and the yaw angle adjustment amount.

[0071] Step S2034: Adjust the position of the camera based on the pitch angle adjustment amount and the yaw angle adjustment amount until the face image is located at the center of the image.

[0072] Specifically, the position and posture of the camera is adjusted based on the pitch angle adjustment amount and the yaw angle adjustment amount until the face image is located at the center of the image.

[0073] In order to optimize the initial position of the camera and the face detection model, in an optional implementation, the method further includes:

[0074] Step S501, when the first recognition result is that the camera does not detect the face image, the corresponding first target image is stored, a second target image is obtained, and the corresponding initial position information is recorded;

[0075] Specifically, in the event that face recognition fails, the image corresponding to the face recognition and the initial position information for face recognition are recorded to optimize the initial position and the face detection model.

[0076] Step S502, in the case where face tracking fails, the first target image during the face tracking process is recorded to obtain a third target image, and the position change of the camera during the face tracking process is recorded to obtain target parameters;

[0077] Specifically, in the case of face tracking failure, all images in the face tracking process are saved to obtain the third target image, and the angle change of the camera in the face tracking process is recorded to obtain the target parameters.

[0078] Step S503, training the face detection model based on the second target image and the initial position information;

[0079] Specifically, when a certain amount of the second target image and initial position information is accumulated, the variable parameters of face detection are adjusted based on the second target image and the initial position information, and the model is updated.

[0080] Step S504: training the classifier based on the third target image and the target parameters.

[0081] Specifically, when a certain amount of the third target image and the target parameters are accumulated, the variable parameters of the face tracking algorithm are adjusted based on the third target image and the target parameters, and the classifier is updated.

[0082] In order to reduce the time for the customer to adjust the position and the time for the camera to perform adaptive tracking, in an optional implementation, after completing the business process according to the second target image, the method further includes:

[0083] Step S601, recording the position of the camera after tracking the face of the user based on the offset parameters so that the face image is located at the center of the image, and obtaining candidate position information;

[0084] Specifically, the position of the camera when the camera performs face tracking during the business process and makes the face image located at the center for the first time is recorded to obtain the candidate position information.

[0085] Step S602: updating the initial position information based on the candidate position information with the highest corresponding frequency at every preset time interval.

[0086] Specifically, when the accumulated amount of candidate position information is large enough, the candidate position information with the highest majority and probability among the candidate position information is used as the initial position information for subsequent camera regression.

[0087] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the implementation process of the terminal camera posture adaptive adjustment method of the present application will be described in detail below in combination with specific embodiments.

[0088] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0089] The embodiment of the present application also provides a terminal camera posture adaptive adjustment device. It should be noted that the terminal camera posture adaptive adjustment device of the embodiment of the present application can be used to execute the terminal camera posture adaptive adjustment method provided by the embodiment of the present application. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0090] The following is an introduction to the terminal camera posture adaptive adjustment device provided in an embodiment of the present application.

[0091] Figure 3 is a structural block diagram of a terminal camera posture adaptive adjustment device according to an embodiment of the present application. Figure 3 As shown, the device comprises:

[0092] The first storage unit 10 is used to record the current position of the camera and obtain initial position information when a target activation signal is detected, wherein the target activation signal is used to instruct the camera to exit the standby state;

[0093] Specifically, the present application is configured to monitor the user's position to determine whether to generate the target activation signal to turn on the camera. The adaptive power-on / off mechanism saves energy and reduces the speed of camera wear. Furthermore, when the camera is activated, the position of the camera before adjustment is recorded to obtain the initial position information.

[0094] A first control unit 20 is used to control the camera to shoot with the initial position information to obtain a plurality of first target images, and to classify and recognize the first target images through a face detection model to obtain a first recognition result, wherein the first recognition result is at least used to indicate whether the camera detects the user's face image;

[0095] Specifically, when the camera is located at the initial position, the camera is controlled to shoot, and a face detection model is used to detect based on the image to determine whether there is a customer within the camera's field of view, thereby obtaining the first recognition result.

[0096] The second control unit 30 is used to determine the offset parameter of the facial image based on the first recognition result when the first recognition result is that the camera monitors the facial image, track the face of the user based on the offset parameter so that the facial image is located at the center of the image, control the camera to shoot, obtain a second target image, and complete the business process according to the second target image;

[0097] Specifically, when the camera recognizes the customer at the initial position, the offset between the user's face and the image center is calibrated based on the first recognition result to obtain the offset parameters, and then the camera's posture is adjusted through an image tracking algorithm based on the offset parameters to realize automatic tracking of the user's face by the camera in the business process.

[0098] The third control unit 40 is used to control the camera to enter the standby state and restore the position according to the initial position information when a target standby signal is detected. The target standby signal is used to instruct the camera to enter the standby state.

[0099] Specifically, after detecting that the customer has left, the target standby signal is generated, the camera is controlled to be turned off, and the camera is restored to its initial position, waiting for the next business process.

[0100] Through this embodiment, the first storage unit records the current position of the camera and obtains initial position information when detecting a target activation signal, and the target activation signal is used to instruct the camera to exit the standby state; the first control unit controls the camera to shoot with the initial position information to obtain multiple first target images, and classifies and identifies the first target images through a face detection model to obtain a first recognition result, and the first recognition result is at least used to characterize whether the camera has detected the user's face image; the second control unit determines the offset parameter of the face image based on the first recognition result when the first recognition result is that the camera has detected the face image, tracks the user's face based on the offset parameter so that the face image is located at the center of the image, controls the camera to shoot, obtains the second target image, and completes the business process according to the second target image; the third control unit controls the camera to enter the standby state and restores the position according to the initial position information when detecting a target standby signal, and the target standby signal is used to instruct the camera to enter the standby state. The present application provides a camera adaptive adjustment method, which performs face recognition and face tracking based on computer vision and machine learning technology, and realizes automatic camera tracking of users, thereby avoiding the trouble of users adjusting their own positions to complete image acquisition during the business processing flow. This method solves the problem in the prior art that the use of fixed-angle cameras or manually adjustable cameras causes the business staff to consume a long time in the processing flow.

[0101] In order to monitor whether there is a customer who needs to handle business in front of the terminal, in an optional implementation, the above device also includes:

[0102] A first generating unit is configured to generate a first signal before recording the current posture of the camera when the distance sensor detects that the distance between the user and the operation screen is less than or equal to a preset distance and lasts for a first preset time period;

[0103] It is understandable that in order to ensure the accuracy of the monitoring results, the present application is configured to sense whether there is a user in a preset area through a combination of sensors of the terminal.

[0104] In one embodiment, the sensor combination includes a distance sensor, a light intensity sensor, and a temperature sensor.

[0105] In a specific implementation, when the distance sensor located on the same side of the operation screen detects that the customer has arrived at a preset distance in front of the terminal and continues for the first preset time, the user's arrival is determined based on the distance and the first signal is generated. Optionally, the preset distance is ≤40cm and the first preset time is 2s.

[0106] A second generating unit, configured to generate a second signal when the temperature sensor detects that the temperature in a preset direction of the operation screen exceeds a preset temperature and lasts for a second preset time period;

[0107] In a specific implementation, when the temperature sensor located on the upper side of the operation screen detects that the temperature in the preset direction begins to rise and lasts for the second preset time, the arrival of the customer is determined based on the temperature perception, and the second signal is generated. Optionally, the preset direction is the front of the operation screen, and the second preset time is 2s.

[0108] A third generating unit, configured to generate a third signal when the light intensity sensor detects that the light intensity in the preset direction of the operation screen is lower than a preset light intensity and lasts for a third preset time period;

[0109] In a specific implementation, when the light intensity sensor located at the lower side of the operation screen detects that the brightness in front is reduced due to the occlusion of the customer's body and lasts for the third preset time, the customer's arrival is determined based on the light intensity perception, and the third signal is generated. Optionally, the preset direction is the front of the operation screen, and the third preset time is 2s.

[0110] The fourth generating unit is used to generate the target activation signal when the first signal, the second signal and the third signal are monitored simultaneously.

[0111] Specifically, when it is determined that there is a customer in the preset area based on distance perception, temperature perception, and light intensity perception, the arrival of the customer is determined, the target activation signal is generated, the camera is activated, and angle adjustment is prepared.

[0112] In order to train the above face detection model, in an optional implementation, the above device further includes:

[0113] A first acquisition unit is used to acquire the face images of the user at different angles to obtain a first data set before classification and recognition is performed based on the first target image through a face detection model;

[0114] Specifically, based on the deep learning algorithm, the customer's facial information is identified. Data preparation is first performed before model training. The main and backup data include face images at different angles such as the front, side, front to side, looking up, and looking down, as the initial data set. The above-mentioned first data set is obtained to ensure the applicability of the face detection model.

[0115] A first processing unit, configured to perform preprocessing based on the sample data in the first data set to obtain a second data set, wherein the preprocessing at least includes flipping, scaling and rotating;

[0116] Specifically, based on the above-mentioned first data set, data enhancement means such as flipping, rotating, scaling, etc. are used to further expand the diversity of the above-mentioned first data set to obtain the above-mentioned second data set.

[0117] An input unit is used to train a deep learning model based on a convolutional neural network by a stochastic gradient descent method based on the second data set, and to correct the training process based on a momentum term to obtain the face detection model, which is used to detect the face image based on the input image.

[0118] Specifically, the above-mentioned face detection model of the present application is designed based on the TensorFlow framework and convolutional neural network. In the specific implementation, a deep learning model is first built based on the convolutional neural network (CNN). When training the model, the stochastic gradient descent method is used to calculate the gradient of the loss function relative to the model. At the same time, in order to solve the problem of small errors in the stochastic gradient descent method, the present application introduces a momentum term to accelerate the convergence process, and uses a predefined learning rate to adjust the learning strategy.

[0119] It can be understood that in the above process, a small batch of samples is used each time to calculate the gradient, and after multiple rounds of training, the prediction work can be completed accurately and quickly, and the above face detection model is obtained.

[0120] In order to obtain the first recognition result, in an optional implementation, the first control unit includes:

[0121] A first processing module is used to extract features of the first target image based on a convolutional neural network to obtain target features, and to slide a window based on the target features to obtain multiple candidate regions, where the candidate regions are regions where the face image may exist;

[0122] Specifically, after the above-mentioned face detection model is deployed, images are collected by a camera at preset intervals to obtain the above-mentioned first target image, and the above-mentioned first target image is input into the above-mentioned face detection model, and features are extracted through the model to obtain the above-mentioned target features, and based on the target features of the image, candidate areas are generated by sliding the window at different positions of the image to obtain the above-mentioned candidate areas.

[0123] The second processing module is used to perform classification and bounding box regression in sequence based on each of the above candidate areas; obtain a face detection bounding box, and determine the above face detection bounding box as the above first recognition result.

[0124] Specifically, based on the candidate areas, each candidate area is classified (whether it is a face image), and based on the classification, bounding box regression is performed (by fine-tuning the position and size of the bounding box, the face is framed in a smaller range to make the recognition result more accurate), to obtain the above-mentioned first recognition result.

[0125] It is understandable that if no facial image is detected in multiple consecutive images, it is determined that the customer's position in the current preset area cannot be recognized. At this time, a voice announcement is made to prompt the customer to adjust the position and move the face as close to the camera as possible, and the first target image is captured again until recognition is successful.

[0126] In order to perform face tracking, in an optional implementation, the second control unit further includes:

[0127] A training module, configured to train a classifier by a machine learning method using the face image in the first recognition result as a positive sample and the area other than the face image in the first recognition result as a negative sample to obtain a target classifier;

[0128] Specifically, the present application sets up a classifier for face tracking, specifically, using a face image as a positive sample and a corresponding background area as a negative sample, training a classifier that can support parameter adjustment, and obtaining the above-mentioned target classifier.

[0129] A first calculation module, used for calculating the offset parameter of the face image through the classifier based on the first recognition results corresponding to the first target images of two adjacent frames;

[0130] Specifically, based on the above-mentioned target classifier, a classifier is used to calculate the change value of the face image in the above-mentioned first recognition result at the current moment and the face image in the above-mentioned first recognition result at the previous image acquisition moment, that is, the above-mentioned offset parameter is determined according to the relative position offset of the face in two consecutive images.

[0131] A second calculation module is used to decompose the above-mentioned offset parameters to obtain the pitch angle adjustment amount and the yaw angle adjustment amount of the above-mentioned camera;

[0132] Specifically, the above offset parameters are decomposed based on the space coordinate system, that is, the above offset parameters are decomposed into the pitch angle and the yaw angle respectively, so as to determine the above pitch angle adjustment amount and the yaw angle adjustment amount.

[0133] The control module is used to adjust the position and posture of the camera based on the pitch angle adjustment amount and the yaw angle adjustment amount until the face image is located at the center of the image.

[0134] Specifically, the position and posture of the camera is adjusted based on the pitch angle adjustment amount and the yaw angle adjustment amount until the face image is located at the center of the image.

[0135] In order to optimize the initial position of the camera and the face detection model, in an optional implementation, the device further includes:

[0136] A second storage unit is used for storing the corresponding first target image, obtaining a second target image, and recording the corresponding initial position information when the first recognition result is that the camera does not monitor the face image;

[0137] Specifically, in the event that face recognition fails, the image corresponding to the face recognition and the initial position information for face recognition are recorded to optimize the initial position and the face detection model.

[0138] A third storage unit is used to record the first target image during the face tracking process to obtain a third target image when face tracking fails, and to record the position change of the camera during the face tracking process to obtain target parameters;

[0139] Specifically, in the case of face tracking failure, all images in the face tracking process are saved to obtain the third target image, and the angle change of the camera in the face tracking process is recorded to obtain the target parameters.

[0140] A first training unit, used for training the face detection model based on the second target image and the initial position information;

[0141] Specifically, when a certain amount of the second target image and initial position information is accumulated, the variable parameters of face detection are adjusted based on the second target image and the initial position information, and the model is updated.

[0142] The second training unit is used to train the classifier based on the third target image and the target parameters.

[0143] Specifically, when a certain amount of the third target image and the target parameters are accumulated, the variable parameters of the face tracking algorithm are adjusted based on the third target image and the target parameters, and the classifier is updated.

[0144] In order to reduce the time for the client to adjust the position and the time for the camera to perform adaptive tracking, in an optional implementation, the above device further includes:

[0145] a fourth storage unit, for recording, after completing the business process according to the second target image, the position and posture of the camera after performing face tracking on the user based on the offset parameter so that the face image is located at the center of the image, to obtain candidate position information;

[0146] Specifically, the position of the camera when the camera performs face tracking during the business process and makes the face image located at the center for the first time is recorded to obtain the candidate position information.

[0147] The updating unit is used to update the above-mentioned initial position information based on the above-mentioned candidate position information with the highest corresponding frequency at intervals of a preset time length.

[0148] Specifically, when the accumulated amount of candidate position information is large enough, the candidate position information with the highest majority and probability among the candidate position information is used as the initial position information for subsequent camera regression.

[0149] The terminal camera posture adaptive adjustment device comprises a processor and a memory, wherein the first storage unit, the first control unit, the second control unit and the third control unit are stored in the memory as program units, and the processor executes the program units stored in the memory to implement corresponding functions. The modules are all located in the same processor; or, the modules are located in different processors in any combination.

[0150] The processor contains a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the speed of the business processing process can be increased by adjusting the kernel parameters.

[0151] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0152] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the terminal camera posture adaptive adjustment method.

[0153] Specifically, the terminal camera posture adaptive adjustment method includes:

[0154] Step S201, when a target activation signal is detected, the current position of the camera is recorded to obtain initial position information, wherein the target activation signal is used to instruct the camera to exit the standby state;

[0155] Specifically, the present application is configured to monitor the user's position to determine whether to generate the target activation signal to turn on the camera. The adaptive power-on / off mechanism saves energy and reduces the speed of camera wear. Furthermore, when the camera is activated, the position of the camera before adjustment is recorded to obtain the initial position information.

[0156] Step S202, controlling the camera to shoot with the initial position information to obtain a plurality of first target images, and classifying and identifying the first target images through a face detection model to obtain a first recognition result, wherein the first recognition result is at least used to indicate whether the camera has detected the user's face image;

[0157] Specifically, when the camera is located at the initial position, the camera is controlled to shoot, and a face detection model is used to detect based on the image to determine whether there is a customer within the camera's field of view, thereby obtaining the first recognition result.

[0158] Step S203, when the first recognition result is that the camera detects the face image, determine the offset parameter of the face image based on the first recognition result, track the face of the user based on the offset parameter so that the face image is located at the center of the image, control the camera to shoot, obtain a second target image, and complete the business process according to the second target image;

[0159] Specifically, when the camera recognizes the customer at the initial position, the offset between the user's face and the image center is calibrated based on the first recognition result to obtain the offset parameters, and then the camera's posture is adjusted through an image tracking algorithm based on the offset parameters to realize automatic tracking of the user's face by the camera in the business process.

[0160] Step S204, when a target standby signal is detected, the camera is controlled to enter a standby state and the position is restored according to the initial position information. The target standby signal is used to instruct the camera to enter a standby state.

[0161] Specifically, after detecting that the customer has left, the target standby signal is generated, the camera is controlled to be turned off, and the camera is restored to its initial position, waiting for the next business process.

[0162] An embodiment of the present invention provides a processor, which is used to run a program, wherein the terminal camera posture adaptive adjustment method is executed when the program is running.

[0163] Specifically, the terminal camera posture adaptive adjustment method includes:

[0164] Step S201, when a target activation signal is detected, the current position of the camera is recorded to obtain initial position information, wherein the target activation signal is used to instruct the camera to exit the standby state;

[0165] Specifically, the present application is configured to monitor the user's position to determine whether to generate the target activation signal to turn on the camera. The adaptive power-on / off mechanism saves energy and reduces the speed of camera wear. Furthermore, when the camera is activated, the position of the camera before adjustment is recorded to obtain the initial position information.

[0166] Step S202, controlling the camera to shoot with the initial position information to obtain a plurality of first target images, and classifying and identifying the first target images through a face detection model to obtain a first recognition result, wherein the first recognition result is at least used to indicate whether the camera has detected the user's face image;

[0167] Specifically, when the camera is located at the initial position, the camera is controlled to shoot, and a face detection model is used to detect based on the image to determine whether there is a customer within the camera's field of view, thereby obtaining the first recognition result.

[0168] Step S203, when the first recognition result is that the camera detects the face image, determine the offset parameter of the face image based on the first recognition result, track the face of the user based on the offset parameter so that the face image is located at the center of the image, control the camera to shoot, obtain a second target image, and complete the business process according to the second target image;

[0169] Specifically, when the camera recognizes the customer at the initial position, the offset between the user's face and the image center is calibrated based on the first recognition result to obtain the offset parameters, and then the camera's posture is adjusted through an image tracking algorithm based on the offset parameters to realize automatic tracking of the user's face by the camera in the business process.

[0170] Step S204, when a target standby signal is detected, the camera is controlled to enter a standby state and the position is restored according to the initial position information. The target standby signal is used to instruct the camera to enter a standby state.

[0171] Specifically, after detecting that the customer has left, the target standby signal is generated, the camera is controlled to be turned off, and the camera is restored to its initial position, waiting for the next business process.

[0172] An embodiment of the present invention provides a business system, the business system includes a processor, a memory, and a program stored in the memory and executable on the processor, and when the processor executes the program, at least the following steps are implemented:

[0173] Step S201, when a target activation signal is detected, the current position of the camera is recorded to obtain initial position information, wherein the target activation signal is used to instruct the camera to exit the standby state;

[0174] Specifically, the present application is configured to monitor the user's position to determine whether to generate the target activation signal to turn on the camera. The adaptive power-on / off mechanism saves energy and reduces the speed of camera wear. Furthermore, when the camera is activated, the position of the camera before adjustment is recorded to obtain the initial position information.

[0175] Step S202, controlling the camera to shoot with the initial position information to obtain a plurality of first target images, and classifying and identifying the first target images through a face detection model to obtain a first recognition result, wherein the first recognition result is at least used to indicate whether the camera has detected the user's face image;

[0176] Specifically, when the camera is located at the initial position, the camera is controlled to shoot, and a face detection model is used to detect based on the image to determine whether there is a customer within the camera's field of view, thereby obtaining the first recognition result.

[0177] Step S203, when the first recognition result is that the camera detects the face image, determine the offset parameter of the face image based on the first recognition result, track the face of the user based on the offset parameter so that the face image is located at the center of the image, control the camera to shoot, obtain a second target image, and complete the business process according to the second target image;

[0178] Specifically, when the camera recognizes the customer at the initial position, the offset between the user's face and the image center is calibrated based on the first recognition result to obtain the offset parameters, and then the camera's posture is adjusted through an image tracking algorithm based on the offset parameters to realize automatic tracking of the user's face by the camera in the business process.

[0179] Step S204, when a target standby signal is detected, the camera is controlled to enter a standby state and the position is restored according to the initial position information. The target standby signal is used to instruct the camera to enter a standby state.

[0180] Specifically, after detecting that the customer has left, the target standby signal is generated, the camera is controlled to be turned off, and the camera is restored to its initial position, waiting for the next business process.

[0181] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program for initializing at least the following method steps:

[0182] Step S201, when a target activation signal is detected, the current position of the camera is recorded to obtain initial position information, wherein the target activation signal is used to instruct the camera to exit the standby state;

[0183] Specifically, the present application is configured to monitor the user's position to determine whether to generate the target activation signal to turn on the camera. The adaptive power-on / off mechanism saves energy and reduces the speed of camera wear. Furthermore, when the camera is activated, the position of the camera before adjustment is recorded to obtain the initial position information.

[0184] Step S202, controlling the camera to shoot with the initial position information to obtain a plurality of first target images, and classifying and identifying the first target images through a face detection model to obtain a first recognition result, wherein the first recognition result is at least used to indicate whether the camera has detected the user's face image;

[0185] Specifically, when the camera is located at the initial position, the camera is controlled to shoot, and a face detection model is used to detect based on the image to determine whether there is a customer within the camera's field of view, thereby obtaining the first recognition result.

[0186] Step S203, when the first recognition result is that the camera detects the face image, determine the offset parameter of the face image based on the first recognition result, track the face of the user based on the offset parameter so that the face image is located at the center of the image, control the camera to shoot, obtain a second target image, and complete the business process according to the second target image;

[0187] Specifically, when the camera recognizes the customer at the initial position, the offset between the user's face and the image center is calibrated based on the first recognition result to obtain the offset parameters, and then the camera's posture is adjusted through an image tracking algorithm based on the offset parameters to realize automatic tracking of the user's face by the camera in the business process.

[0188] Step S204, when a target standby signal is detected, the camera is controlled to enter a standby state and the position is restored according to the initial position information. The target standby signal is used to instruct the camera to enter a standby state.

[0189] Specifically, after detecting that the customer has left, the target standby signal is generated, the camera is controlled to be turned off, and the camera is restored to its initial position, waiting for the next business process.

[0190] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0191] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0192] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0193] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0194] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0195] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0196] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0197] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0198] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0199] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0200] 1) The terminal camera posture adaptive adjustment method of the present application, first, in the case of detecting a target activation signal, the current posture of the camera is recorded to obtain initial position information, and the above-mentioned target activation signal is used to instruct the camera to exit the standby state; then, the camera is controlled to shoot with the above-mentioned initial position information to obtain multiple first target images, and the face detection model is used to classify and identify the first target images to obtain a first recognition result, and the above-mentioned first recognition result is at least used to characterize whether the camera has detected the user's face image; thereafter, in the case that the above-mentioned first recognition result is that the above-mentioned camera has detected the above-mentioned face image, the offset parameter of the above-mentioned face image is determined based on the above-mentioned first recognition result, the user's face is tracked based on the above-mentioned offset parameter so that the above-mentioned face image is located at the center of the image, and the camera is controlled to shoot to obtain a second target image, and the business process is completed according to the above-mentioned second target image; finally, in the case of detecting a target standby signal, the camera is controlled to enter the standby state and restore the position according to the above-mentioned initial position information, and the above-mentioned target standby signal is used to instruct the camera to enter the standby state. The present application provides a camera adaptive adjustment method, which performs face recognition and face tracking based on computer vision and machine learning technology, and realizes automatic camera tracking of users, thereby avoiding the trouble of users adjusting their own positions to complete image acquisition during the business processing flow. This method solves the problem in the prior art that the use of fixed-angle cameras or manually adjustable cameras causes the business staff to consume a long time in the processing flow.

[0201] 2) The terminal camera posture adaptive adjustment device of the present application, the first storage unit records the current posture of the camera and obtains the initial position information when detecting the target activation signal, and the above-mentioned target activation signal is used to instruct the control of the above-mentioned camera to exit the standby state; the first control unit controls the above-mentioned camera to shoot with the above-mentioned initial position information to obtain multiple first target images, and classifies and recognizes them based on the above-mentioned first target images through the face detection model to obtain a first recognition result, and the above-mentioned first recognition result is at least used to characterize whether the above-mentioned camera has detected the user's face image; the second control unit determines the offset parameter of the above-mentioned face image based on the above-mentioned first recognition result when the above-mentioned first recognition result is that the above-mentioned camera has detected the above-mentioned face image, tracks the user's face based on the above-mentioned offset parameter so that the above-mentioned face image is located at the center of the image, and controls the above-mentioned camera to shoot to obtain the second target image, and completes the business process according to the above-mentioned second target image; the third control unit controls the above-mentioned camera to enter the standby state and restores the position according to the above-mentioned initial position information when detecting the target standby signal, and the above-mentioned target standby signal is used to instruct the control of the above-mentioned camera to enter the standby state. The present application provides a camera adaptive adjustment method, which performs face recognition and face tracking based on computer vision and machine learning technology, and realizes automatic camera tracking of users, thereby avoiding the trouble of users adjusting their own positions to complete image acquisition during the business processing flow. This method solves the problem in the prior art that the use of fixed-angle cameras or manually adjustable cameras causes the business staff to consume a long time in the processing flow.

[0202] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A terminal camera posture adaptive adjustment method, characterized in that: include: When a target activation signal is detected, the current position of the camera is recorded to obtain initial position information, wherein the target activation signal is used to instruct the camera to exit the standby state; Control the camera to shoot with the initial position information to obtain a plurality of first target images, and classify and recognize the first target images through a face detection model to obtain a first recognition result, wherein the first recognition result is at least used to indicate whether the camera detects a face image of the user; In the case where the first recognition result is that the camera monitors the facial image, determining an offset parameter of the facial image based on the first recognition result, tracking the face of the user based on the offset parameter so that the facial image is located at the center of the image, controlling the camera to shoot, obtaining a second target image, and completing the business process according to the second target image; In the case of monitoring a target standby signal, the camera is controlled to enter a standby state and the position is restored according to the initial position information, wherein the target standby signal is used to instruct the camera to enter a standby state.

2. The method according to claim 1, characterized in that The terminal includes an operation screen, a distance sensor, a light intensity sensor, and a temperature sensor. Before recording the current position of the camera, the method further includes: When the distance sensor detects that the distance between the user and the operation screen is less than or equal to a preset distance and lasts for a first preset time period, generating a first signal; When the temperature sensor detects that the temperature in a preset direction of the operation screen exceeds a preset temperature and lasts for a second preset time period, generating a second signal; When the light intensity sensor detects that the light intensity in the preset direction of the operating screen is lower than the preset light intensity and lasts for a third preset time period, generating a third signal; The target activation signal is generated when the first signal, the second signal and the third signal are monitored simultaneously.

3. The method according to claim 1, characterized in that: Before classifying and identifying the first target image through a face detection model, the method further includes: Acquire the facial images of the user at different angles to obtain a first data set; Performing preprocessing based on the sample data in the first data set to obtain a second data set, wherein the preprocessing at least includes flipping, scaling and rotating; Based on the second data set, a deep learning model based on a convolutional neural network is trained by a stochastic gradient descent method, and the training process is corrected based on a momentum term to obtain the face detection model, which is used to detect the face image according to the input image.

4. The method according to claim 3, characterized in that Classifying and identifying the first target image through a face detection model to obtain a first recognition result includes: Extracting features of the first target image based on a convolutional neural network to obtain target features, sliding a window based on the target features to obtain multiple candidate regions, where the candidate regions are regions where the face image may exist; Classification and bounding box regression are performed in sequence based on each of the candidate regions; a face detection bounding box is obtained, and the face detection bounding box is determined as the first recognition result.

5. The method according to claim 1, characterized in that Determining an offset parameter of the face image based on the first recognition result, and tracking the face of the user based on the offset parameter so that the face image is located at the center of the image, includes: Taking the face image in the first recognition result as a positive sample and the area other than the face image in the first recognition result as a negative sample, training a classifier by a machine learning method to obtain a target classifier; Calculating the offset parameter of the face image by the classifier based on the first recognition results corresponding to the first target images in two adjacent frames; Decomposing the offset parameters to obtain a pitch angle adjustment amount and a yaw angle adjustment amount of the camera; The position and posture of the camera is adjusted based on the pitch angle adjustment amount and the yaw angle adjustment amount until the face image is located at the center of the image.

6. The method according to claim 5, characterized in that The method further comprises: When the first recognition result is that the camera does not detect the face image, storing the corresponding first target image, obtaining a second target image, and recording the corresponding initial position information; In the case of face tracking failure, the first target image in the face tracking process is recorded to obtain a third target image, and the position change of the camera in the face tracking process is recorded to obtain target parameters; Training the face detection model based on the second target image and the initial position information; The classifier is trained based on the third target image and the target parameters.

7. The method according to claim 1, characterized in that After completing the business process according to the second target image, the method further includes: Recording the position of the camera after tracking the face of the user based on the offset parameter so that the face image is located at the center of the image, and obtaining candidate position information; The initial position information is updated at intervals of a preset duration based on the candidate position information with the highest corresponding frequency.

8. A terminal camera posture adaptive adjustment device, characterized in that: The device comprises: A first storage unit is used to record the current position of the camera and obtain initial position information when a target activation signal is detected, wherein the target activation signal is used to instruct the camera to exit the standby state; A first control unit is used to control the camera to shoot with the initial position information to obtain a plurality of first target images, and to classify and recognize the first target images through a face detection model to obtain a first recognition result, wherein the first recognition result is at least used to indicate whether the camera has detected a face image of the user; a second control unit, configured to, when the first recognition result is that the camera monitors the facial image, determine an offset parameter of the facial image based on the first recognition result, perform face tracking on the user based on the offset parameter so that the facial image is located at the center of the image, control the camera to shoot, obtain a second target image, and complete the business process according to the second target image; The third control unit is used to control the camera to enter a standby state and restore the position according to the initial position information when a target standby signal is detected. The target standby signal is used to instruct the camera to enter a standby state.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 7.

10. A financial self-service terminal, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include methods for executing any one of claims 1 to 7.

Images