Method and device for adjusting height of machine body of self-service terminal equipment and electronic equipment

Abstract

The invention discloses a self-service terminal equipment body height adjusting method and device and electronic equipment, and relates to the field of financial science and technology. The method comprises the following steps: acquiring depth information including a user and a background through image acquisition equipment installed on the self-service terminal equipment, and generating an original depth map of the user and the background; the original depth map is preprocessed, a target depth map is obtained, and preprocessing is used for removing background information of non-users and drawing the edges and shapes of the users; posture information of the user is determined according to the target depth map, and the posture information is at least used for representing that the user is in a standing posture or in a wheelchair posture; and according to the posture information of the user, controlling a lifting device of the self-service terminal equipment to adjust the machine body height of the self-service terminal equipment. The technical problem that in the prior art, due to the fact that self-service terminal equipment of a financial institution needs to manually adjust the height of a machine body, the timeliness of machine body height adjustment is poor is solved.

Description

Technical Field

[0001] The present application relates to the field of financial technology, and specifically to a method, device and electronic device for adjusting the height of a self-service terminal device. Background Art

[0002] With the increasing trend towards smart and self-service banking services, self-service terminals have become an indispensable component of bank branches. These devices can significantly improve the efficiency of banking transactions. However, traditional self-service terminals are relatively rigid in design and often fail to effectively accommodate customers with disabilities and mobility impairments, as they may be difficult to operate at a high enough level.

[0003] However, in the prior art, the self-service terminal equipment of financial institutions requires manual adjustment of the height of the body, which has a technical problem of poor timeliness in adjusting the height of the body.

[0004] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0005] The embodiments of the present application provide a method, device, and electronic device for adjusting the height of a self-service terminal device, so as to at least solve the technical problem in the prior art that the height of the self-service terminal devices of financial institutions needs to be manually adjusted, resulting in poor timeliness of the height adjustment.

[0006] According to one aspect of an embodiment of the present application, a method for adjusting the height of a self-service terminal device is provided, comprising: acquiring depth information including a user and a background by an image acquisition device installed on the self-service terminal device to generate an original depth map of the user and the background; preprocessing the original depth map to obtain a target depth map, wherein the preprocessing is used to remove background information other than the user and outline the edges and shape of the user; determining the user's posture information based on the target depth map, wherein the posture information is used to at least characterize whether the user is in a standing posture or in a wheelchair posture; and controlling the lifting device of the self-service terminal device to adjust the height of the self-service terminal device based on the user's posture information.

[0007] Optionally, preprocessing includes: denoising the original depth map to obtain an intermediate depth map; dividing the intermediate depth map into a foreground sub-image and a background sub-image based on a depth threshold, wherein the distance between the pixels in the foreground sub-image and the image acquisition device is less than the depth threshold; the distance between the pixels in the background sub-image and the image acquisition device is greater than or equal to the depth threshold; detecting the edge information of the user in the foreground sub-image, and outlining the user image area based on the edge information; and using the image block where the user image area is located as the target depth map.

[0008] Optionally, after the intermediate depth map is divided into a foreground sub-map and a background sub-map based on a depth threshold, if it is detected that the foreground sub-map includes images of multiple users, the depth pixel information of the target user closest to the image acquisition device is retained, and the depth pixel information of other users except the target user is deleted from the foreground sub-map.

[0009] Optionally, determining the user's posture information based on the target depth map includes: extracting the user's key point features from the target depth map through a convolutional layer of a neural network; after inputting the key point features into multiple feature processing channels of the neural network, compressing the key point features in each feature processing channel into a single numerical value through a global average pooling layer of the neural network to represent the global features of the channel; scaling and activating each feature processing channel through a fully connected layer of the neural network to obtain a weight for each feature processing channel; multiplying the weight of each feature processing channel by the user's key point features to obtain target key point features; and determining the user's posture information based on the target key point features.

[0010] Optionally, based on the user's posture information, the lifting device of the self-service terminal device is controlled to adjust the body height of the self-service terminal device, including: if it is detected that the user is in a wheelchair posture, determining the spatial coordinates of the user's target key points based on the user's posture information; and adjusting the body height of the self-service terminal device based on the spatial coordinates.

[0011] Optionally, adjusting the body height of the self-service terminal device according to the spatial coordinates includes: determining a target height of the self-service terminal device according to the spatial coordinates; determining a trapezoidal speed curve according to the current height and the target height of the self-service terminal device, wherein the trapezoidal speed curve is used to represent a change curve of the height adjustment speed of the self-service terminal device; and adjusting the body height of the self-service terminal device to the target height according to the trapezoidal speed curve.

[0012] Optionally, based on the user's posture information, controlling the lifting device of the self-service terminal device to adjust the body height of the self-service terminal device includes: if it is detected that the user is in a standing posture, generating a prompt message, wherein the prompt message is used to prompt the user to choose whether to adjust the body height of the self-service terminal device; when it is detected that feedback information of the prompt message indicates that the user chooses to adjust the body height of the self-service terminal device, adjusting the body height of the self-service terminal device according to the user's posture information; when it is detected that feedback information of the prompt message indicates that the user chooses not to adjust the body height of the self-service terminal device, keeping the body height of the self-service terminal device unchanged.

[0013] According to another aspect of an embodiment of the present application, a device for adjusting the height of a self-service terminal device is also provided, including: a generation unit, used to collect depth information including a user and a background through an image acquisition device installed on the self-service terminal device, and generate an original depth map of the user and the background; a first processing unit, used to preprocess the original depth map to obtain a target depth map, wherein the preprocessing is used to remove background information other than the user and outline the edges and shape of the user; a determination unit, used to determine the user's posture information based on the target depth map, wherein the posture information is at least used to characterize whether the user is in a standing posture or in a wheelchair posture; a second processing unit, used to control the lifting device of the self-service terminal device to adjust the height of the self-service terminal device according to the user's posture information.

[0014] According to another aspect of an embodiment of the present application, a computer-readable storage medium is further provided, in which a computer program is stored. When the computer program is executed, the device where the computer-readable storage medium is located executes the above-mentioned method for adjusting the height of the self-service terminal device.

[0015] According to another aspect of an embodiment of the present application, an electronic device is also provided, including one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors execute the above-mentioned method for adjusting the height of the self-service terminal device body.

[0016] From the above content, it can be seen that the present application collects depth information including the user and the background through an image acquisition device installed on the self-service terminal device to generate an original depth map of the user and the background; the original depth map is preprocessed to obtain a target depth map, wherein the preprocessing is used to remove non-user background information and outline the edges and shapes of the user; the user's posture information is determined based on the target depth map, wherein the posture information is at least used to characterize whether the user is in a standing posture or in a wheelchair posture; according to the user's posture information, the lifting device of the self-service terminal device is controlled to adjust the body height of the self-service terminal device.

[0017] In an embodiment of the present application, an image acquisition and depth information processing method is adopted. Depth information including the user and the background is collected by an image acquisition device installed on the self-service terminal device to generate an original depth map. The original depth map is preprocessed to remove non-user background information and outline the user's edge and shape. The user posture information is then determined based on the target depth map, and the lifting device is controlled to adjust the body height based on the user posture information. The purpose of automatically adjusting the body height of the self-service terminal device according to the actual posture of the user is achieved, thereby realizing the technical effect of automated and intelligent height adjustment of the self-service terminal device, and thus solving the technical problem in the prior art that the self-service terminal equipment of financial institutions needs to manually adjust the body height, resulting in poor timeliness of the body height adjustment. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 This is a flow chart of an optional method for adjusting the height of a self-service terminal device according to an embodiment of the present application;

[0020] Figure 2 is a flowchart of an optional depth image acquisition module according to an embodiment of the present application;

[0021] Figure 3 is a workflow diagram of an optional data preprocessing module according to an embodiment of the present application;

[0022] Figure 4 is a workflow diagram of an optional user posture classification module according to an embodiment of the present application;

[0023] Figure 5 is a schematic structural diagram of an optional user posture classification module according to an embodiment of the present application;

[0024] Figure 6 is a schematic diagram of an optional device body height adjustment module according to an embodiment of the present application;

[0025] Figure 7 is a flowchart of another optional method for adjusting the height of a self-service terminal device according to an embodiment of the present application;

[0026] Figure 8 This is a schematic diagram of an optional device for adjusting the height of a self-service terminal device according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] It should also be noted that the information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) collected by this application are information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data comply with the relevant laws, regulations and standards of the relevant regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse. For example, an interface is set up between this system and relevant users or institutions. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or institution through the interface, and obtain relevant information after receiving the consent information fed back by the aforementioned user or institution.

[0030] According to an embodiment of the present application, an embodiment of a method for adjusting the height of a self-service terminal device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0031] Optionally, according to an embodiment of the present application, a self-service terminal device body height adjustment system (hereinafter referred to as the system) is provided as the execution subject of the self-service terminal device body height adjustment method based on the embodiment of the present application, wherein the system can be a software system or an embedded system combining software and hardware. Of course, the method execution subject in the embodiment of the present application can also be other forms of execution subjects, such as devices, equipment, etc. Those skilled in the art should know that this application does not specifically limit the specific form of expression of the method execution subject.

[0032] Figure 1 This is a flow chart of an optional method for adjusting the height of a self-service terminal device according to an embodiment of the present application. Figure 1 As shown, the method includes the following steps:

[0033] Step S101 : collecting depth information including the user and the background through an image acquisition device installed on the self-service terminal device to generate an original depth map of the user and the background.

[0034] Optionally, before collecting user data (such as image data and posture data), explicit user authorization must be obtained. This includes informing the user of the purpose of data collection, how it will be used, and how their privacy will be protected. The user is also clearly informed of the privacy policy, including policies regarding data collection, use, storage, and deletion, as well as how users can exercise their rights, such as accessing, correcting, or deleting their data. Users have the right to choose whether to consent to the collection and use of their data. Furthermore, when collecting user data, the system uses encryption technology to ensure data security during transmission and storage. User data will be used strictly in accordance with relevant laws and regulations and within the scope of user authorization. For example, if a user authorizes data to be used only for height adjustment of a self-service terminal, the data will not be used for other purposes. User data will be stored in a secure database and undergo regular security audits to ensure data security and the effectiveness of privacy protection measures. When the user data is no longer needed, the system securely deletes it according to user requirements and legal requirements to ensure data cannot be recovered.

[0035] Alternatively, depth information may refer to data describing the distance between an object (user) in a scene and an image acquisition device. Depth information may be presented in the form of a depth map, where the value of each pixel represents the distance between that point and the camera.

[0036] Alternatively, the raw depth map refers to an unprocessed depth map directly captured by the image acquisition device, which includes all depth information of the user and background. The raw depth map may contain unwanted information such as noise and background interference, which needs to be optimized through subsequent preprocessing steps.

[0037] Optionally, the system can complete the above steps through a depth image acquisition module. Figure 2 This is a workflow diagram of an optional depth image acquisition module according to an embodiment of the present application, such as Figure 2 As shown, the depth image acquisition module uses sensors to detect when a user approaches the self-service terminal and automatically activates a high-precision depth camera. The high-precision depth camera then captures the user's image in real time, obtaining depth information about the user and their background, and generates a depth map. Finally, the depth camera controller transmits the captured depth map via the network to a backend server for processing.

[0038] For example, when customer A approaches a self-service kiosk, the device's sensors detect their presence and automatically activate the device's installed depth camera. The depth camera begins collecting three-dimensional spatial information, encompassing both customer A and the background. This includes the contours of customer A's body, their distance from the kiosk, and objects in the background. The camera converts this 3D information into digital signals, generating a raw depth map. This map contains depth information for both customer A and the background, with each pixel representing its distance from the camera. In the raw depth map, parts of customer A's body (such as the head, shoulders, and waist) are displayed in varying grayscale. Generally, objects closer to the camera appear as brighter pixels, while objects farther away appear as darker pixels. The generated raw depth map is then transmitted to the device's processing unit for subsequent image processing and analysis, such as background subtraction, foreground extraction, and edge detection, to identify customer A's posture and determine whether the kiosk's height needs to be adjusted.

[0039] Optionally, the depth image acquisition module uses an image acquisition device to capture depth information, generating raw depth maps of the user and background. This provides an accurate data foundation for subsequent depth image preprocessing and gesture recognition, achieving the goal of accurately perceiving the distance difference between the user and the background. This process ensures the accuracy of subsequent processing steps, thereby providing a reliable basis for height adjustment of the self-service terminal device, thereby paving the way for resolving the existing problem of untimely and inaccurate height adjustment of self-service terminals.

[0040] Step S102: pre-process the original depth map to obtain a target depth map.

[0041] In step S102 , pre-processing is used to remove non-user background information and outline the edge and shape of the user.

[0042] Optionally, during pre-processing of the raw depth image, the system desensitizes the data, removing or replacing potentially identifiable information. For example, facial features in the image may be blurred or user data may be anonymized using techniques. Furthermore, when processing and analyzing user data, the system ensures that all operations are performed in a secure environment to prevent data leakage or unauthorized access.

[0043] Optionally, the target depth map is a pre-processed depth map that primarily contains the user's depth information, with background information removed and the user's edges and shape clearly outlined. This makes the target depth map more concise and clear, facilitating subsequent user gesture recognition and analysis.

[0044] For example, the original depth map contains all the depth information of customer A and the background, but in order to more accurately analyze the posture of customer A, the background information needs to be removed and only the depth information related to customer A needs to be retained. The preprocessing algorithm first identifies and removes the background information. This can be achieved by setting a depth threshold. All pixels exceeding this threshold (i.e., objects farther away from the camera) are considered to be the background and removed from the depth map. Next, the algorithm can outline the edges and shape of customer A. This is usually achieved through an edge detection algorithm, such as the Canny edge detection algorithm, which can identify places in the image where pixel values change dramatically, i.e., the edges of the outline of customer A. After background removal and edge outlining, the resulting target depth map only contains the depth information of customer A, clearly showing the edges and shape of customer A, providing accurate data for subsequent posture analysis and equipment height adjustment.

[0045] Optionally, by preprocessing the original depth map, the system can effectively remove background information irrelevant to the user's posture and accurately outline the user's edges and shape, thereby generating a clear target depth map. This process can significantly improve the quality of the depth map, making subsequent user posture recognition more accurate and efficient. This preprocessing step not only reduces the interference of background noise on the recognition process but also optimizes the structure of the image data, providing a solid data foundation for intelligent height adjustment of self-service terminal equipment.

[0046] Step S103: determining the user's posture information according to the target depth map.

[0047] In step S103 , the posture information is at least used to indicate whether the user is in a standing posture or a wheelchair-sitting posture.

[0048] For example, the system can analyze customer A's posture by utilizing a target depth map. Using deep learning algorithms or computer vision techniques, the system identifies customer A's body outline and key points, such as the head, shoulders, and knees. The system analyzes the relative position and depth information of these key points to determine whether customer A is standing or in a wheelchair. For example, if the knees are significantly lower than the shoulders, and the head and shoulders are relatively stable, the system may determine that the customer is in a wheelchair. Conversely, if the head, shoulders, and knees display normal standing human proportions, the system determines that the customer is standing.

[0049] Optionally, the system automatically adjusts the height of the device, making it comfortable for both standing and wheelchair users to use the kiosk. This reduces the need for users or staff to manually adjust the device, thereby improving efficiency. Furthermore, personalized height adjustment can enhance the user's interactive experience with the device.

[0050] Step S104 : controlling the lifting device of the self-service terminal device to adjust the height of the self-service terminal device according to the user's posture information.

[0051] For example, once the system determines customer A's posture, it will automatically adjust the height of the kiosk based on this information. For example, if customer A is in a wheelchair, the kiosk will be lowered to a suitable height for the wheelchair user; if customer A is standing, the kiosk will maintain its original height or make minor adjustments based on customer A's feedback and key points.

[0052] Optionally, the system automatically adjusts the height of the device, making it easy for users of different heights, including those in wheelchairs, to use the self-service terminal. This expands the device's applicability, reduces manual intervention, and enhances the device's intelligence. Furthermore, by providing personalized height adjustment, the above process enhances the user's interactive experience with the device, reduces user inconvenience and fatigue, and ensures the safety of the device and user during the adjustment process, preventing accidental injuries caused by improper height adjustment.

[0053] From the above content, it can be seen that the present application adopts the method of image acquisition and depth information processing. The image acquisition device installed on the self-service terminal device collects depth information including the user and the background, generates an original depth map, and pre-processes the original depth map to remove non-user background information and outline the user's edge and shape. Then, the user posture information is determined based on the target depth map, and the lifting device is controlled to adjust the body height according to the user posture information, thereby achieving the purpose of automatically adjusting the body height of the self-service terminal device according to the actual posture of the user, thereby realizing the technical effect of automation and intelligent height adjustment of the self-service terminal device, and thus solving the technical problem in the prior art that the self-service terminal equipment of financial institutions needs to manually adjust the body height, resulting in poor timeliness of the body height adjustment.

[0054] In an optional embodiment, preprocessing includes: denoising the original depth map to obtain an intermediate depth map; dividing the intermediate depth map into a foreground sub-image and a background sub-image based on a depth threshold, wherein the distance between the pixels in the foreground sub-image and the image acquisition device is less than the depth threshold; the distance between the pixels in the background sub-image and the image acquisition device is greater than or equal to the depth threshold; detecting the edge information of the user in the foreground sub-image, and outlining the user image area based on the edge information; and using the image block where the user image area is located as the target depth map.

[0055] Optionally, the system completes the above process through a data preprocessing module, wherein: Figure 3 This is a workflow diagram of an optional data preprocessing module according to an embodiment of the present application, such as Figure 3 As shown in the figure, first, the data preprocessing module removes noise from the image, thereby improving image quality. The data preprocessing module can use median filtering technology for denoising. The sliding window sorts the neighborhood values of each pixel and replaces the original pixel value with the median value, thereby effectively removing random noise and preserving image details.

[0056] Next, the data preprocessing module performs depth map segmentation, dividing the image into two parts: the foreground (user) sub-image and the background sub-image. As shown in formula (1), the data preprocessing module sets a depth threshold T. Pixels exceeding this threshold can be regarded as background sub-images, and pixels below this threshold can be regarded as foreground sub-images. Based on this threshold T, the system can create a binary image and divide the depth map into foreground sub-images and background sub-images:

[0057]

[0058] Wherein, in formula (1), M(x, y) is the foreground mask, and D′(x, y) represents the depth value after denoising at the pixel position (x, y). As shown in formula (1), the foreground pixels are marked as 1 (white) and the background pixels are marked as 0 (black).

[0059] For example, when customer B approaches a self-service terminal, the terminal's depth camera automatically activates, capturing 3D information of customer B and their surroundings, generating a raw depth map. Due to ambient lighting variations or device noise, this raw depth map may contain noise. The data preprocessing module uses median filtering to remove noise. For example, if the depth values of a 3x3 pixel region are ranked as [900, 950, 1000, 1050, 1100, 1150, 1200, 1250, 1300], the median filter selects the middle value, 1100, as the new depth value for the center pixel of the region, thereby reducing noise and preserving image detail. The denoised depth map enters the segmentation stage, where the data preprocessing module sets a depth threshold, T, for example, T = 1200 mm. All pixels exceeding 1200 mm are considered background sub-images, while pixels below 1200 mm are considered foreground sub-images (customer B). Based on this threshold, the system creates a binary image, where foreground pixels are labeled 1 (white) and background pixels are labeled 0 (black). Based on the segmentation results, the system creates a foreground mask M(x,y), where M(x,y) = 1 indicates that the pixel at position (x,y) belongs to the foreground (customer B), and M(x,y) = 0 indicates that it belongs to the background. This mask is used in subsequent processing steps, such as pose recognition and keypoint detection, to ensure that only the image portions relevant to the customer are analyzed.

[0060] The data preprocessing module can then extract the foreground sub-image portion, i.e., the area where the user is located, from the original depth map. In addition, using the generated foreground mask, only the depth information of the user closest to the device is retained, ignoring other users or the background, ensuring that the extracted information is relevant to the current user.

[0061] Finally, the data preprocessing module can perform edge detection to obtain the user's outline. The data preprocessing module can apply the Canny edge detection algorithm. First, Gaussian blurring is performed on the image to further reduce noise and provide a smoother image for edge detection. The module then calculates the gradient strength and direction of each pixel in the image to find the initial edge location. Next, non-maximum suppression is used to refine the edge, ensuring that the edge is continuous and that each edge pixel is a local maximum within its neighborhood. Finally, dual-threshold detection is used to identify and connect the true edges, connecting weak and strong edges to form a complete edge outline.

[0062] Through the above steps, the system can accurately extract the user outline from the depth image through the data preprocessing module, and provide an intelligent height adjustment basis for the self-service terminal device, thereby improving the user experience and the adaptability of the device.

[0063] In an optional embodiment, after the intermediate depth map is divided into a foreground sub-map and a background sub-map based on a depth threshold, if it is detected that the foreground sub-map includes images of multiple users, the depth pixel information of the target user closest to the image acquisition device is retained, and the depth pixel information of other users except the target user is deleted from the foreground sub-map.

[0064] Optionally, the data preprocessing module can use the generated foreground mask to isolate the foreground region from the original depth map. This process is similar to using a template to crop an image, retaining only the portion where the user is located. However, multiple users may be present at the kiosk at the same time. In this case, this method only retains the depth pixel information related to the user closest to the device to ensure that only data related to that user is extracted.

[0065] Optionally, deleting the depth pixel information of other users helps reduce interference during the analysis process, allowing the system to focus more on the most relevant user at the moment. By focusing on the closest user, the system can more accurately identify and analyze that user's posture, thereby improving recognition accuracy. This also allows the self-service terminal to better adapt to multi-user environments, accurately identifying and responding to the needs of the target user even in crowded situations.

[0066] In an optional embodiment, determining the user's posture information based on the target depth map includes: extracting the user's key point features from the target depth map through a convolutional layer of a neural network; after inputting the key point features into multiple feature processing channels of the neural network, compressing the key point features in each feature processing channel into a single numerical value through a global average pooling layer of the neural network to represent the global features of the channel; scaling and activating each feature processing channel through a fully connected layer of the neural network to obtain a weight for each feature processing channel; multiplying the weight of each feature processing channel by the user's key point features to obtain target key point features; and determining the user's posture information based on the target key point features.

[0067] Optionally, the system may perform the above process through a user gesture classification module, wherein: Figure 4 This is a workflow diagram of an optional user posture classification module according to an embodiment of the present application, such as Figure 4 As shown in Figure 1, the user posture classification module first receives input data, namely the depth map after data preprocessing. These depth maps have undergone necessary preprocessing, such as denoising and binarization, to improve the accuracy and efficiency of subsequent processing.

[0068] Next, the user pose classification module can use a 3D convolutional layer for feature extraction. 3D convolution involves multiple layers of convolution, for example, with a kernel size of 5*5*5 in each layer. Larger kernels can capture a wider range of contextual information and reduce the number of layers required in the network, lowering computational complexity. The user pose classification module can then use the ReLU activation function. This helps reduce the computational effort during backpropagation and improve model training efficiency.

[0069] Next, the user posture classification module can use the SE module (Squeeze-and-Excitation Module) as the attention mechanism layer to enhance the key features of the character. First, the SE module can use the global average pooling layer to compress the spatial information of each channel into a single numerical value, representing the global features of the channel. Then, two fully connected layers can be used for scaling and activation. The first fully connected layer reduces the number of channels, and the second fully connected layer restores the original number of channels and generates weights for each channel through the Sigmoid activation function. These weights represent the importance of each channel. Finally, the generated weights are multiplied by the original feature map to obtain a weighted feature map, which can enhance the key features.

[0070] The user posture classification module can then use the posture estimation layer to identify wheelchair users and standing users. Specifically, the user posture classification module can use the lightweight network HRNet (High-Resolution Network) to extract 3D key points (such as head, shoulders, knees, etc.) and output the coordinates of these key points. HRNet can extract key point features through a series of convolutions, and then the user posture classification module can use shallow convolution and residual blocks to learn the key point features of the human body. After the convolution processing, the user posture classification module can output the coordinates of the user's key points in space through the fully connected layer.

[0071] Finally, the user posture classification module can combine the feature maps and posture key points obtained in the above process and input them into the fully connected layer for classification, so as to judge the user's posture (such as standing, sitting in a wheelchair, etc.), and finally output the posture classification result.

[0072] Through these steps, the user posture classification module extracts key features from the depth map, identifies the user's posture, and outputs the corresponding classification results. This automated posture recognition technology can provide intelligent height adjustment for self-service terminals, thereby improving device accessibility and user experience.

[0073] Optionally, Figure 5 is a schematic structural diagram of an optional user posture classification module according to an embodiment of the present application, such as Figure 5As shown in the figure, the input layer serves as the starting point of the user posture classification module and receives input data. Next, the input data is subjected to feature extraction through multiple 3D convolutional layers (i.e., 3D convolutional layers * N shown in the figure), and the extracted features are enhanced by the attention mechanism layer. The purpose of this layer is to enhance important features and suppress unimportant features, thereby improving the performance of the model. Next, the features processed by the attention mechanism layer will be sent to the posture estimation layer to detect key points in the image, such as key parts of the human body (such as the head, shoulders, knees, etc.). The key point information output by the posture estimation layer will be sent to the fully connected layer for further processing. Finally, the information processed by the fully connected layer is ultimately used to output the posture classification result, that is, to determine the posture of the person in the input data.

[0074] In an optional embodiment, the lifting device of the self-service terminal device is controlled to adjust the body height of the self-service terminal device according to the user's posture information, including: if it is detected that the user is in a wheelchair posture, determining the spatial coordinates of the user's target key point according to the user's posture information; and adjusting the body height of the self-service terminal device according to the spatial coordinates.

[0075] Optionally, the system can enable the terminal device to detect the key point information of the person through the device body height adjustment module, and then control the drive to adjust the height of the device lifting device according to the calculation result.

[0076] Specifically, when the system detects that the target user is a wheelchair user, it uses the posture estimation layer to extract key points of the user, such as the head, shoulders, and knees. The spatial coordinates of these key points are precisely calculated, providing a basis for adjusting the device's height. Based on the spatial coordinates of the user's key points, particularly the user's eye level or the height of the wheelchair, the system calculates the required height adjustment. The self-service terminal's lifting mechanism then automatically adjusts the device's height based on this calculation, ensuring the device screen or interactive interface is in the most comfortable and accessible position for the user.

[0077] Optionally, Figure 6 is a schematic diagram of an optional device body height adjustment module according to an embodiment of the present application, such as Figure 6As shown, first, the system can determine whether the user is using a wheelchair based on the posture classification results output by the user posture classification module. If the user is identified as a standing user, the system can display on the smart terminal device whether the body height adjustment is required; if the user is identified as a wheelchair user, the system will combine multiple frames of data for averaging to improve the accuracy of key point detection. Among them, key points include shoulders, top of the head, etc., and this information will be used for subsequent height adjustment calculations. Based on the obtained coordinate information, the system will calculate an appropriate height by comparing the user's height information with the preset standard distance, where the preset standard distance can refer to the appropriate height distance between the top of the user's head and the device screen.

[0078] For example, when Customer C, in a wheelchair, approaches a self-service terminal and wishes to use an ATM, the device's onboard depth camera automatically activates, capturing a depth image of Customer C. The device's internal deep learning model analyzes the image information to identify Customer C's posture in the wheelchair and extract the coordinates of key points on Customer C's head and shoulders. Based on these coordinates and a preset standard distance, the system calculates Customer C's eye level and determines that the device screen needs to be lowered 15 cm for optimal eye level. The device's lifting mechanism then automatically lowers the screen 15 cm, enabling Customer C to comfortably view and operate the screen.

[0079] In an optional embodiment, adjusting the height of the self-service terminal device according to the spatial coordinates includes: determining a target height of the self-service terminal device according to the spatial coordinates; determining a trapezoidal speed curve according to the current height and the target height of the self-service terminal device, wherein the trapezoidal speed curve is used to represent a curve of change in the height adjustment speed of the self-service terminal device; and adjusting the height of the self-service terminal device to the target height according to the trapezoidal speed curve.

[0080] Optionally, when controlling the height adjustment of the autonomous terminal device, the system may use a trapezoidal speed curve to control the smooth movement of the autonomous terminal device body, and reduce sudden changes in speed by controlling acceleration and deceleration, thereby reducing wear and vibration of the mechanical system; at the same time, during the height adjustment process, the system may add a safety detection mechanism to prevent the autonomous terminal device body from causing pinching or other injuries to the user when it is raised or lowered.

[0081] In an optional embodiment, controlling a lifting device of a self-service terminal device to adjust a body height of the self-service terminal device according to user posture information includes: if it is detected that the user is in a standing posture, generating a prompt message, wherein the prompt message is used to prompt the user to select whether to adjust the body height of the self-service terminal device; if feedback information of the prompt message indicates that the user has selected to adjust the body height of the self-service terminal device, adjusting the body height of the self-service terminal device according to the user posture information; if feedback information of the prompt message indicates that the user has selected not to adjust the body height of the self-service terminal device, maintaining the body height of the self-service terminal device unchanged.

[0082] Optionally, if the user posture classification module identifies the user as standing, the system will generate a prompt asking the user whether to adjust the height of the autonomous terminal device for a more comfortable interactive experience. The system awaits user feedback, which can be provided through touchscreen selection, voice commands, or other interactive methods. Depending on the user's feedback, the system will take different actions: If the user chooses to adjust the height, the system will automatically adjust the height of the autonomous terminal device based on the user's posture information. If the user chooses not to adjust the height, the system will maintain the current height of the autonomous terminal device. If the user chooses to adjust the height, the system will calculate an appropriate height based on the obtained key point coordinate information by comparing the user's height information with a preset standard distance. The preset standard distance can be the appropriate height distance between the user's head and the autonomous terminal device screen. The system will then adjust the autonomous terminal device's height to ensure that the autonomous terminal device screen or interactive interface is in the most comfortable and accessible position for the user.

[0083] For example, customer D approaches a self-service terminal in a bank lobby to withdraw cash or perform other banking operations. As customer D approaches the terminal, the depth camera on the terminal automatically captures a depth image of the customer. A deep learning model within the terminal analyzes the image and recognizes that customer D is standing. The system prompts customer D to adjust the height of the terminal to ensure the user interface is within their most comfortable field of view. Customer D responds to their selection by touching a button on the screen or using a voice command.

[0084] If Customer D selects "Yes," indicating that they wish to adjust the height of the autonomous terminal, the system will proceed with the autonomous terminal height adjustment process. If Customer D selects "No," indicating that they do not wish to adjust the height, the system will maintain the current height of the autonomous terminal. If Customer D chooses to adjust the height, the system will automatically calculate and adjust the height of the autonomous terminal based on the coordinates of Customer D's key points. The autonomous terminal's lifting mechanism will smoothly adjust the height of the device, ensuring that the user interface is within Customer D's most comfortable viewing area, for example, approximately 50 cm from the eye.

[0085] Customer D can now enjoy a comfortable user experience without having to manually adjust the terminal, improving the convenience and satisfaction of self-service. Furthermore, by providing this intelligent, personalized service, the bank enhances customer satisfaction and loyalty while also reducing the burden on bank staff.

[0086] Optionally, Figure 7 is a flow chart of another optional method for adjusting the height of the self-service terminal device according to an embodiment of the present application, such as Figure 7 As shown in the figure, the process for adjusting the height of a self-service terminal based on the user's posture consists of four main modules: a depth image acquisition module, a data preprocessing module, a user posture classification module, and a device height adjustment module. First, the depth image acquisition module begins operation. When a user is in front of the self-service terminal, the system automatically captures an image of the user through the depth image acquisition module and generates a depth map of the user based on the captured image data.

[0087] Next, the system uses the data preprocessing module to read the depth map generated by the depth image acquisition module and perform necessary preprocessing. This module removes background information from the preprocessed depth map, extracts the user image from the foreground subimage, and performs edge contour detection on the extracted foreground image to identify the user's shape and posture. It then extracts key features from the processed image to prepare for subsequent user posture classification.

[0088] The system then uses an attention mechanism to enhance key features in the image through the user posture classification module, thereby improving classification accuracy. The user posture classification module can output the user's posture classification results, such as standing or sitting in a wheelchair, based on the extracted and enhanced features.

[0089] Finally, the system uses the device body height adjustment module to calculate the height using the key feature points (such as head, shoulders, knees, etc.) extracted from the user posture classification module, and based on the calculation results, uses a trapezoidal velocity curve to smoothly adjust the height of the device body to adapt to the user's posture.

[0090] The entire process begins with capturing a user image. Through depth map generation and preprocessing, key user features are extracted and used to classify the user's posture. Finally, based on the posture classification results, the device's height is automatically adjusted to provide an optimal user experience. This process ensures that the self-service terminal can intelligently adapt to the needs of different users, providing a more user-friendly and convenient service, especially for wheelchair users.

[0091] According to another aspect of the present application, a device for adjusting the height of a self-service terminal device is provided, wherein: Figure 8 Schematic diagram of an optional device for adjusting the height of a self-service terminal device according to an embodiment of the present application. Figure 8 As shown, the device for adjusting the height of the self-service terminal device includes: a generating unit 801 , a first processing unit 802 , a determining unit 803 , and a second determining unit 804 .

[0092] Optionally, the generation unit 801 is used to collect depth information including the user and the background through an image acquisition device installed on the self-service terminal device to generate an original depth map of the user and the background; the first processing unit 802 is used to preprocess the original depth map to obtain a target depth map, wherein the preprocessing is used to remove background information other than the user and outline the edges and shape of the user; the determination unit 803 is used to determine the user's posture information based on the target depth map, wherein the posture information is at least used to characterize the user as being in a standing posture or in a wheelchair posture; the second processing unit 804 is used to control the lifting device of the self-service terminal device to adjust the body height of the self-service terminal device according to the user's posture information.

[0093] Optionally, the first processing unit 802 includes: a first processing subunit, a second processing subunit, a detection subunit, and a first determination subunit. The first processing subunit is configured to denoise the original depth map to obtain an intermediate depth map; the second processing subunit is configured to segment the intermediate depth map into a foreground subimage and a background subimage based on a depth threshold, wherein the distance between the pixels in the foreground subimage and the image acquisition device is less than the depth threshold; and the distance between the pixels in the background subimage and the image acquisition device is greater than or equal to the depth threshold; the detection subunit is configured to detect edge information of the user in the foreground subimage and to delineate the user image area based on the edge information; and the first determination subunit is configured to use the image block where the user image area is located as the target depth map.

[0094] Optionally, the first processing unit 802 further includes: a third processing sub-unit. The third processing sub-unit is configured to, after segmenting the intermediate depth map into a foreground sub-image and a background sub-image based on the depth threshold, retain the depth pixel information of the target user closest to the image acquisition device if it is detected that the foreground sub-image includes images of multiple users, and delete the depth pixel information of other users except the target user from the foreground sub-image.

[0095] Optionally, the determination unit 803 includes: an extraction subunit, a fourth processing subunit, a fifth processing subunit, a sixth processing subunit, and a second determination subunit. The extraction subunit is used to extract the key point features of the user from the target depth map through the convolution layer of the neural network; the fourth processing subunit is used to compress the key point features in each feature processing channel into a single numerical value representing the global features of the channel through the global average pooling layer of the neural network after inputting the key point features into multiple feature processing channels of the neural network; the fifth processing subunit is used to scale and activate each feature processing channel through the fully connected layer of the neural network to obtain the weight of each feature processing channel; the sixth processing subunit is used to multiply the weight of each feature processing channel by the key point features of the user to obtain the target key point features; and the second determination subunit is used to determine the user's posture information based on the target key point features.

[0096] Optionally, the second processing unit 804 includes: a third determining subunit and a seventh processing subunit. The third determining subunit is configured to determine the spatial coordinates of the user's target key point based on the user's posture information if the user is detected to be in a wheelchair; and the seventh processing subunit is configured to adjust the height of the self-service terminal device based on the spatial coordinates.

[0097] Optionally, the seventh processing subunit includes: a first determining module, a second determining module, and a processing module. The first determining module is configured to determine a target height of the self-service terminal device based on spatial coordinates; the second determining module is configured to determine a trapezoidal speed curve based on the current height of the self-service terminal device and the target height, wherein the trapezoidal speed curve is configured to represent a curve of a change in the height adjustment speed of the self-service terminal device; and the processing module is configured to adjust the height of the self-service terminal device to the target height based on the trapezoidal speed curve.

[0098] Optionally, the second processing unit 804 further includes: a generating subunit, an eighth processing subunit, and a ninth processing subunit. The generating subunit is configured to generate prompt information if it is detected that the user is in a standing posture, wherein the prompt information is used to prompt the user to select whether to adjust the height of the self-service terminal device; the eighth processing subunit is configured to adjust the height of the self-service terminal device according to the user's posture information if feedback information from the prompt information indicates that the user has selected to adjust the height of the self-service terminal device; and the ninth processing subunit is configured to maintain the height of the self-service terminal device unchanged if feedback information from the prompt information indicates that the user has selected not to adjust the height of the self-service terminal device.

[0099] According to another aspect of the present application, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium. When the computer program runs, the device where the computer-readable storage medium is located executes the above-mentioned method for adjusting the height of the self-service terminal device.

[0100] According to another aspect of the present application, an electronic device is also provided, comprising one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the above-mentioned method for adjusting the height of the self-service terminal device body.

[0101] The above-mentioned embodiments or examples disclosed in this application are not exhaustive, but are only illustrations of some embodiments or examples, and are not intended to be specific limitations on the scope of protection disclosed in this application. In the absence of contradiction, each step in a certain embodiment or example in this application can be implemented as an independent example, and the steps can be arbitrarily combined. For example, the solution after removing some steps in a certain embodiment or example can also be implemented as an independent example, and the order of the steps in a certain embodiment or example can be arbitrarily exchanged. In addition, the optional methods or optional examples in a certain embodiment or example can be arbitrarily combined; in addition, the various embodiments or examples can be arbitrarily combined. For example, some or all of the steps in different embodiments or examples can be arbitrarily combined, and a certain embodiment or example can be arbitrarily combined with the optional methods or optional examples of other embodiments or examples.

[0102] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0103] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0104] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0105] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected to achieve the purpose of the present embodiment according to actual needs.

[0106] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0107] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program code.

[0108] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for adjusting the height of a self-service terminal device, characterized in that: include: The image acquisition device installed on the self-service terminal device collects depth information including the user and the background, and generates an original depth map of the user and the background; Preprocessing the original depth map to obtain a target depth map, wherein the preprocessing is used to remove background information other than the user and outline the edge and shape of the user; Determining posture information of the user based on the target depth map, wherein the posture information is at least used to indicate whether the user is in a standing posture or a wheelchair-sitting posture; According to the user's posture information, the lifting device of the self-service terminal device is controlled to adjust the height of the body of the self-service terminal device.

2. The method according to claim 1, characterized in that The pretreatment comprises the following steps: Denoising the original depth map to obtain an intermediate depth map; Segmenting the intermediate depth map into a foreground sub-map and a background sub-map according to a depth threshold, wherein the distance between pixels in the foreground sub-map and the image acquisition device is less than the depth threshold; and the distance between pixels in the background sub-map and the image acquisition device is greater than or equal to the depth threshold; Detecting edge information of the user in the foreground sub-image, and outlining a user image region based on the edge information; The image block where the user image area is located is used as the target depth map.

3. The method according to claim 2, characterized in that The method further comprises: After the intermediate depth map is divided into a foreground sub-map and a background sub-map according to the depth threshold, if it is detected that the foreground sub-map includes images of multiple users, the depth pixel information of the target user closest to the image acquisition device is retained, and the depth pixel information of other users except the target user is deleted from the foreground sub-map.

4. The method according to claim 1, wherein Determining the user's posture information based on the target depth map includes: Extracting key point features of the user from the target depth map through a convolutional layer of a neural network; After inputting the key point features into multiple feature processing channels of the neural network, the key point features in each feature processing channel are compressed into a single value representing the global feature of the channel through a global average pooling layer of the neural network; Scaling and activating each feature processing channel through a fully connected layer of the neural network to obtain a weight for each feature processing channel; Multiplying the weight of each feature processing channel by the key point feature of the user to obtain the target key point feature; The user's posture information is determined according to the target key point features.

5. The method according to claim 1, characterized in that Controlling the lifting device of the self-service terminal device to adjust the height of the self-service terminal device according to the user's posture information includes: If it is detected that the user is in a wheelchair-sitting posture, determining the spatial coordinates of the target key point of the user according to the posture information of the user; The height of the self-service terminal device is adjusted according to the spatial coordinates.

6. The method according to claim 5, characterized in that Adjusting the height of the self-service terminal device according to the spatial coordinates includes: determining a target height of the self-service terminal device according to the spatial coordinates; Determining a trapezoidal speed curve according to the current height of the self-service terminal device and the target height, wherein the trapezoidal speed curve is used to represent a change curve of the height adjustment speed of the self-service terminal device; The height of the self-service terminal device is adjusted to the target height according to the trapezoidal speed curve.

7. The method according to claim 1, characterized in that Controlling the lifting device of the self-service terminal device to adjust the height of the self-service terminal device according to the user's posture information includes: If it is detected that the user is in a standing posture, generating a prompt message, wherein the prompt message is used to prompt the user to choose whether to adjust the height of the self-service terminal device; When it is detected that the feedback information of the prompt information indicates that the user has chosen to adjust the height of the self-service terminal device, adjusting the height of the self-service terminal device according to the posture information of the user; When it is detected that the feedback information of the prompt information indicates that the user chooses not to adjust the height of the self-service terminal device, the height of the self-service terminal device is kept unchanged.

8. A method for adjusting the height of a self-service terminal device, characterized in that: include: A generating unit, configured to collect depth information including the user and the background by using an image acquisition device installed on the self-service terminal device, and generate an original depth map of the user and the background; a first processing unit, configured to preprocess the original depth map to obtain a target depth map, wherein the preprocessing is used to remove background information other than the user and outline the edge and shape of the user; a determining unit, configured to determine posture information of the user based on the target depth map, wherein the posture information is at least used to indicate whether the user is in a standing posture or a wheelchair-sitting posture; The second processing unit is configured to control the lifting device of the self-service terminal device to adjust the height of the self-service terminal device according to the posture information of the user.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located executes the method for adjusting the height of the self-service terminal device according to any one of claims 1 to 7.

10. An electronic device, characterized in that: The device comprises one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the method for adjusting the height of the self-service terminal device according to any one of claims 1 to 7.

Images