Humanized elevator taking method and system based on machine vision and mechanical arm interaction

Abstract

The invention belongs to the technical field of robots, and particularly relates to a humanization elevator taking method and system based on machine vision and mechanical arm interaction. The method comprises the steps that S1, the robot recognizes current floor information and calls an elevator; s2, the robot enters the elevator by adopting a humanization method; s3, after the robot enters the elevator, the target floor is pressed down; and S4, the robot intelligently leaves the elevator after judging that the robot arrives at the target floor. The robot has the advantages that the robot can take an elevator like a common person, the requirement of the robot for taking the elevator autonomously can be met, and manpower and economic cost are saved.

Description

Technical Field

[0001] The present invention belongs to the technical field of robots, and particularly relates to a humanoid elevator-riding method and system based on machine vision and robotic arm interaction. Background Art

[0002] With the development of society and the progress of technology, robots have been widely used in various industries. In modern buildings, the movement between floors mainly relies on elevators. However, currently, the movement of mainstream robots mainly focuses on planar path planning, and robots lack the ability to autonomously ride elevators, which limits their application scenarios. Therefore, it is of great significance for robots to have the ability to autonomously ride elevators. At present, only a small number of robots use the method of implanting chips connected to elevators to achieve elevator rides through scheduling. However, this method requires separate customization of each elevator and robot, resulting in waste of economy and manpower, and it is not applicable to most robots.

[0003] Therefore, it is very important to design a humanoid elevator-riding method and system based on machine vision and robotic arm interaction that enables robots to perform the function of riding elevators like ordinary humans, meets the needs of robots for autonomous elevator rides, and saves manpower and economic costs. Summary of the Invention

[0004] The present invention aims to overcome the problem that most current robots cannot achieve autonomous elevator rides in the prior art, and provides a humanoid elevator-riding method and system based on machine vision and robotic arm interaction that enables robots to perform the function of riding elevators like ordinary humans, meets the needs of robots for autonomous elevator rides, and saves manpower and economic costs.

[0005] To achieve the above invention objectives, the present invention adopts the following technical solutions:

[0006] The humanoid elevator-riding method based on machine vision and robotic arm interaction includes the following steps:

[0007] S1, the robot identifies the current floor information and summons the elevator;

[0008] S2, the robot enters the elevator using a humanoid method;

[0009] S3, after the robot enters the elevator, it presses the target floor;

[0010] S4, after the robot determines that it has reached the target floor, it intelligently leaves the elevator.

[0011] Preferably, step S1 includes the following steps:

[0012] S11. First, the robot creates a map of and identifies the current floor using lidar. Based on the mapping function, the robot makes a preliminary judgment on the current floor environment to find the location of the elevator on the current floor and autonomously navigates to the elevator lobby.

[0013] S12. The robot identifies the up and down button information in the elevator lobby and determines whether to press the button in combination with the target floor information. If necessary, the robot uses the four-axis robotic arm adaptive pressing method to autonomously summon the elevator by pressing the elevator lobby button.

[0014] Preferably, step S11 includes the following steps:

[0015] When the robot obtains the target position, the movement process starts immediately. The robot uses the high-precision distance obtained by lidar and the short-range detection of ultrasonic sensors to ensure that the robot understands the surrounding dynamic environment and static obstacles.

[0016] In a dynamic environment, the robot uses a global path planning algorithm to formulate an optimal route. When an obstacle appears in the environment, the robot updates the path in real time and uses a local obstacle avoidance algorithm to actively avoid the obstacle to ensure continuous passage.

[0017] Preferably, in step S12, the four-axis robotic arm adaptive pressing method for pressing the elevator lobby button includes the following steps:

[0018] S121. Coarse positioning:

[0019] The robot uses a camera to capture image information related to the elevator buttons. Based on the obtained image information, it quickly identifies the approximate position of the elevator buttons and performs coarse positioning with an accuracy controlled within plus or minus 5 centimeters.

[0020] S122. Precise adjustment:

[0021] The robotic arm of the robot interacts with the camera and continuously adjusts the position of the robotic arm through real-time feedback to achieve a target accuracy within 2 centimeters.

[0022] S123. Judgment program:

[0023] After the precise adjustment stage, the last step is to implement the judgment program, which is used to make a judgment based on the current position information of the robotic arm and the target position to ensure that the robotic arm accurately presses the elevator button.

[0024] Preferably, step S2 includes the following steps:

[0025] S21. After calling the elevator, the robot first judges the opening and closing of the elevator door. When the elevator door opens, the robot makes an intelligent judgment on whether to enter the elevator. The robot uses a camera to identify the situation inside the elevator. If it is identified that the number of people inside the elevator reaches or exceeds the set threshold, or the remaining space is insufficient, the robot automatically judges to give up entering the elevator. When the robot judges that the number of people inside the elevator is appropriate and the remaining space is sufficient, the robot enters the elevator to complete the operations of the robot entering the elevator and giving way to pedestrians.

[0026] Preferably, in step S21, the robot uses a camera to identify the situation inside the elevator, specifically referring to using a method for identifying the number of people inside the elevator and a method for estimating the remaining space inside the elevator based on Bayesian classification. The method for identifying the number of people inside the elevator based on Bayesian classification includes the following steps:

[0027] S211. Extraction of image features inside the elevator:

[0028] First, determine the features of the target area and adopt Histogram of Oriented Gradients (HOG) features. The HOG features are composed by calculating and statistically analyzing the gradient direction histogram of the local area of the image.

[0029] S212. Recognition of the number of people based on Bayesian classification:

[0030] Based on the existing image features, establish a classifier for predicting unknown image types. Bayesian classification is based on Bayes' theorem and estimates the posterior probability by training a large number of samples.

[0031] Among them, the established classifier needs to select training samples and sample features. The training samples need to be manually made. The positive samples are the set of images of the occupied areas, which are the areas with people inside the elevator. The negative samples are the set of images of the unoccupied areas. The HOG features of the regional images are used as the feature vectors for Bayesian classification.

[0032] The method for estimating the remaining space inside the elevator includes the following steps:

[0033] S213. After determining the number of people in the elevator, since the area occupied by each person in the elevator is about 0.2 to 0.3 square meters. The common passenger elevator size on the market is 1.1 meters × 1.4 meters, and the common freight elevator size is 1.5 meters × 1.5 meters, with the height between 2 meters and 4 meters. Based on this, we can estimate the remaining area and volume in the elevator, establish a set with the same size as the number of people in the elevator. Set the number of people in the elevator as n, then establish Zn, where Zn is a set with n elements. Calculate the number of blocks occupied by normal people according to the normal distribution. After counting the number of blocks obtained after identifying each person in the elevator and then assigning values respectively, store them in Z1, Z2, Z3, ……, Zn; if the number of blocks of person 1 obtained after counting is much less than the normal value, then set Z1 equal to 0.2, and similarly if the number of blocks of person 2 obtained after counting is much more, then set Z1 equal to 0.3. Then the total floor space G occupied by the people in the elevator is:

[0034]

[0035] Compare the obtained proportion G of the people in the elevator with the total floor area F occupied by the elevator (for the freight elevator, it is 2.25 square meters, and for the passenger elevator, it is 1.54 square meters, which can be calculated according to the specific elevator) to obtain the remaining space A in the elevator, that is, A = F - G; when the value of A is less than the set threshold, it is judged that the remaining space in the elevator is insufficient, and the robot gives up entering the elevator.

[0036] Preferably, step S3 includes the following steps:

[0037] S31. When the robot enters the elevator, rotate the camera through the servo pan-tilt for one week. First, identify the digital matrix through edge recognition. When it is recognized, stop rotating and move the digital matrix to the center of the screen;

[0038] S32. By identifying the digital information and color information, judge whether the target floor button has been pressed. If not, the robot operates the robotic arm to press the target floor button to complete the operation of the robot independently pressing the target floor button after entering the elevator.

[0039] Preferably, in step S32, the specific process of judging whether the target floor button has been pressed is as follows:

[0040] In the operation interface of the elevator, after the button is pressed by the user, it emits a light signal of a specific color to confirm the successful reception of the instruction; if the target floor button has been pressed, that is, the target floor button emits light, no operation is required. If the target floor button has not been pressed, that is, the target floor button does not emit light, the robot operates the robotic arm to press the target floor button.

[0041] Preferably, step S4 includes the following steps:

[0042] S41. After the robot determines that the button for the target floor has been pressed, it continuously identifies the button for the target floor. When the button for the target floor changes from glowing to non - glowing, the robot determines that the target floor has been reached.

[0043] S42. After reaching the target floor, the robot makes a voice announcement to prompt pedestrians that the target floor has been reached. Subsequently, the robot uses an ultrasonic sensor to judge the opening and closing of the elevator door. When the elevator door opens, the robot leaves the elevator to complete the operation of arriving at the target floor.

[0044] The present invention also provides a human - like elevator - riding system based on the interaction between machine vision and a robotic arm, including:

[0045] An identification and summons module for the robot to identify the current floor information and summon the elevator.

[0046] An interaction module for the robot to enter the elevator using a human - like method.

[0047] An operation module for the robot to press the target floor button after entering the elevator.

[0048] A judgment and movement module for the robot to intelligently leave the elevator after determining that it has reached the target floor.

[0049] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) The present invention improves the degree of automation and efficiency. Unmanned operation: Without manual intervention, the robotic arm can autonomously complete the entire process of elevator calling, door opening and closing, floor selection, etc., and is applicable to scenarios such as unmanned factories, warehousing logistics, etc.; Time optimization: By visually identifying the elevator status in real time (such as floor display, door opening and closing status), the waiting time is reduced, the action sequence is dynamically adjusted, and the transportation efficiency is improved; Multi-task collaboration: It can be linked with other automation systems (such as AGV, conveyor belt) to form an end-to-end material transportation closed loop; (2) The present invention enhances environmental adaptability. Visual generalization ability: It can adapt to complex scenarios such as different elevator models, button layouts, and lighting changes. Dynamic obstacle avoidance: By using lidar to monitor the elevator door area in real time, it can avoid pinching or hitting people or objects, ensuring the safety of actions; (3) The present invention reduces the deployment and maintenance costs. No elevator modification required: Traditional solutions require physical modification of the elevator (such as installing RFID or communication modules), while the visual solution only requires external installation of a robotic arm and a camera, reducing the hardware cost; Software scalability: The algorithm model can be updated through OTA to adapt to new elevator models or functional requirements, avoiding repeated deployment; (4) The present invention expands the application scenarios. Cross-domain applicability: The technology can be migrated to other enclosed space operations (such as subway turnstiles, automatic access control), forming a general mobile operation ability; Service robot integration: It provides a "last mile" vertical transportation solution for delivery robots and medical transportation robots, breaking through the bottleneck of building automation; (5) The present invention has certain social and economic values. Promote smart buildings: Provide core technical support for building intelligence, and help new business forms such as unmanned logistics and intelligent warehousing. Energy conservation and emission reduction: By optimizing the transportation path and elevator usage efficiency, the no-load energy consumption is reduced; Standardization promotion: Promote the formulation of robot-elevator communication protocols (such as ROS-Industrial standard), and promote industry interconnection. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 FIG. is a flowchart of a humanoid elevator riding method based on machine vision and robotic arm interaction in the present invention;

[0051] Figure 2 FIG. is a schematic diagram of the structure of the robot in the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0052] In order to more clearly illustrate the embodiments of the present invention, the specific implementation manners of the present invention will be described below with reference to the accompanying drawings. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings, and other implementation manners can also be obtained.

[0053] As Figure 1As shown, the present invention provides a humanoid elevator-riding method based on machine vision and robotic arm interaction, including the following steps:

[0054] S1, the robot identifies the current floor information and summons the elevator;

[0055] S2, the robot enters the elevator using a humanoid method;

[0056] S3, after the robot enters the elevator, it presses the target floor;

[0057] S4, after the robot determines that it has reached the target floor, it intelligently leaves the elevator.

[0058] Regarding step S1, the main principle is as follows:

[0059] The robot will first use lidar to map and identify the current floor. Based on the mapping function, the robot can make a preliminary judgment on the current floor environment to find the location of the elevator on the current floor, so as to realize the function of autonomous navigation to the elevator lobby. The robot will identify the up and down floor button information in the elevator lobby, and combine the target floor information to judge whether it is necessary to press the button. If necessary, the robot will use the four-axis robotic arm adaptive pressing method to control the elevator lobby button to realize the function of autonomous elevator summoning.

[0060] Among them, the method for identifying the current floor information based on lidar to reach the elevator lobby on the current floor is as follows:

[0061] In an indoor environment, using lidar for SLAM mapping can better obtain information. The positioning problem and map addressing problem are the two main problems solved by SLAM. When the robot enters a strange environment, SLAM can help the robot avoid obstacles in the environment during the running process in this environment, and timely perceive the surrounding environment and spatial relationship. Finally, it is ensured that the robot can safely run to the destination in a strange environment.

[0062] The robot designed by the present invention adopts four-wheel independent drive, and can accurately control the robot to complete corresponding movements according to the planned route.

[0063] When the robot obtains the target position, the movement process starts immediately. At this time, the robot not only relies on lidar and ultrasonic ranging modules to collect observation data of the surrounding environment, but also combines these data with its own movement information. The collaborative work of these two sensors can effectively construct a real-time model of the environment. Through the high-precision distance obtained by lidar and the close-range detection of ultrasonic sensors, it can be ensured that the robot fully understands the surrounding dynamic and static obstacles.

[0064] In a dynamic environment, real-time path planning is particularly important. The robot will use global path planning algorithms (such as A* or Dijkstra algorithms) to formulate an optimal route. This route is not only the shortest path from the starting point to the target location, but also takes into account safety and feasibility. When obstacles appear in the environment, the robot can update the path in real time and use local obstacle avoidance algorithms (such as the dynamic window method or the Velocity Obstacles method) to actively avoid obstacles, ensuring continuous passage. Through precise path calculation and frequent data updates, the robot achieves flexible dynamic navigation.

[0065] The system uses serial communication to transmit the speed information required for navigation and obstacle avoidance to the lower-level main control module. At the beginning, the lower-level computer adjusts the movement direction and speed of the robot chassis by controlling the motors, including forward, backward, left turn, and right turn. The robot always remains vigilant about the surrounding environment during this process and can respond to any obstacles in a timely manner to achieve highly safe navigation.

[0066] The method for the robot to autonomously summon the elevator in the present invention includes the following ways:

[0067] 1) The method for recognizing the up and down buttons in the elevator vestibule based on image processing:

[0068] When the robot arrives at the elevator vestibule, its first step is to identify whether the currently lit up and down floor buttons on the elevator vestibule panel meet the up and down floor requirements of the target floor. This recognition process relies on the robot's visual processing system, usually equipped with a high-resolution camera and image recognition software. The status of the elevator buttons (i.e., the currently lit and unpressed buttons) depends not only on their physical state but also on the light signals they emit.

[0069] (1) The method for recognizing the position of the elevator vestibule buttons based on edge detection:

[0070] The recognition of matrix buttons is mainly achieved through edge recognition. When recognizing matrix buttons, edge detection is first performed on the matrix buttons. The Canny operator is used for edge detection experiments. Due to its excellent performance in noise suppression ability and edge detection accuracy, it is widely used in various scenarios in image processing, especially in edge extraction in complex environments, showing extremely high robustness. The working process of the Canny operator has been strictly theoretically deduced and practically verified, and includes a series of steps, such as Gaussian smoothing, gradient calculation, non-maximum suppression, and hysteresis thresholding method, etc. These steps not only ensure that the detected edges have good continuity, but also make their positioning accuracy reach a relatively high level, and can effectively reduce the influence of noise interference on edge detection. The core process of its algorithm can be divided into the following steps:

[0071] Gaussian smoothing

[0072] During the edge detection process, images are often affected by noise, which can interfere with edge detection. Therefore, the Canny operator first performs Gaussian filtering on the input image I(x,y) to smooth the image and reduce the impact of noise. The two-dimensional function of the Gaussian filter is expressed as:

[0073]

[0074] to the smoothed image I G (x,y):

[0075] I G (x,y) = I(x,y) * G(x,y);

[0076] This step effectively reduces the impact of high-frequency noise and lays the foundation for subsequent gradient calculations.

[0077] Gradient calculation:

[0078] Edges usually manifest as sharp changes in the image grayscale value. Therefore, the Canny operator detects edges by calculating the gradient of the image. Perform a difference operation on the smoothed image I G (x,y) to obtain the gradient images in the x and y directions:

[0079]

[0080] The gradient magnitude M(x,y) and the gradient direction θ(x,y) are calculated as follows respectively:

[0081]

[0082]

[0083] The gradient magnitude M(x,y) represents the intensity of the edge, while the gradient direction θ(x,y) provides the direction information of the edge. This step lays the foundation for accurately locating the edge.

[0084] Double-threshold detection:

[0085] After non-maximum suppression, the edge image may still contain some weak edges caused by noise or other factors. To further refine the edges, the Canny operator introduces the double-threshold technique. By setting a high threshold T h and a low threshold T l , the edge pixel points (x,y) are divided into three categories:

[0086] Strong Edge: M(x,y) > T h ;

[0087] Weak Edge: Tl <M(x,y)≤T h ;

[0088] Non - Edge: M(x,y)≤T l ;

[0089] Strong edge points are directly retained. Weak edge points are only retained when connected to strong edges, while non - edge points are suppressed. This step helps to eliminate pseudo - edges caused by noise and enhance the detection effect of real edges.

[0090] Edge connection and result generation:

[0091] In the last step, through edge connection operations, the retained weak edges are connected to strong edges to form continuous edges. This step ensures the integrity of the edges, avoiding edge breaks and incoherence. The finally generated binary image clearly shows the edge structure in the image, where 1 represents the edge and 0 represents non - edge. Thus, the coordinate information of the elevator lobby buttons is obtained.

[0092] Character template matching algorithm:

[0093] After identifying the positions of the matrix buttons, the image and numbers inside the buttons are recognized and segmented. Character template matching is a key step in character recognition. All possible text templates need to be created and stored before matching. The principle is to compare and pair the features of the text to be recognized with the standard character templates stored in the computer before, and compare their similarity. If the similarity between the two characters being compared is very high, theoretically, they can be determined to be the same character, and the matching is successful. The principle of the template matching algorithm is as follows:

[0094] First, the size of the segmented target character is normalized to adjust it to a predefined size unified with the images in the template library. Then, the binary character image is divided regionally, that is, the target character image is divided into a grid pattern according to the dividing lines and abstracted into a matrix structure denoted as M kl . The number of valid pixels of the binary character image falling into the corresponding grid regions is counted, and the value of each element is denoted as X ij . For the matrix M kl , the value X ij counted in the previous step is assigned to the specified elements of the target matrix, and finally a complete set of image grid feature vectors is obtained. The above algorithm for extracting the grid feature vectors of a single character image will be applied to the input character image and the template image.

[0095] The grid feature extraction of the input character image occurs in the last stage of the preprocessing module. The grid feature extraction of the template image occurs in the initialization stage when the program runs for the first time, and the processed result data of the template image is stored in the file system with characters as the file unit.

[0096] In order to match each segmented character image with the template, it is necessary to operate on the feature matrix of the character image obtained in the previous section to obtain the correlation between the two. Here, it is defined that the greater the correlation, the closer the two are, so as to classify the target character image into the current template character set. The calculation method of the correlation is shown in the following formula:

[0097]

[0098] Among them, X represents the normalized character matrix in memory to be recognized, and P represents a template character matrix in the template library for matching. X m,n represents the data in the m-th row and n-th column of the character feature matrix to be recognized. Similarly, P m,n represents the data in the m-th row and n-th column of a template character feature matrix in the template library for matching. Finally, the calculated r represents the correlation between the character image to be recognized and the matching character in the character template. The range of r is (0, 1]; the larger r is, the more matching the two images are.

[0099] (2) Recognition method of the color information of the front hall buttons

[0100] After the elevator button is pressed, it usually emits a light signal of a specific color for confirmation. In most cases, the button emits blue light. The luminous characteristic of this blue light provides a visual sign for the recognition of the floor. The robot will monitor this blue light with an optical sensor and extract its feature information.

[0101] The common color space model is the RGB model, but the RGB model is commonly used for image description and is generally not used in image recognition because the RGB model is a color channel model of the three primary colors of red, green, and blue, which combines the three parameters of hue, brightness, and saturation commonly used in color recognition and is difficult to separate. Color spaces such as HSV can distinguish colors more intuitively. Here, H represents hue, S represents saturation, and V represents value.

[0102] The HSV space model is similar to an inverted cone model. The value of value V is measured along the vertical line of the cone, and the range is from 0 to 1. 0 represents black and 1 represents white. The value of hue H is measured by the angle value of the bottom surface of the cone, and the range is 0° - 360°. The H values of red, green, and blue are each divided by 120° counterclockwise, and the corresponding complementary colors differ by 180°. Saturation S is a proportional value.

[0103] The proportional formula for saturation S is:

[0104]

[0105] The value range of S is from 0 to 1. When S = 0, it is only gray.

[0106] First, collect a large number of pictures of the illuminated elevator buttons, and then use an image processing tool to extract the RGB values of the pictures of the illuminated elevator buttons. Use the following formula:

[0107] max = Max[R, G, B];

[0108] min = Min[R, G, B];

[0109]

[0110] For the determination range of the illuminated state, when the HSV feature values of the pictures captured by the robot's camera are within the range set in the present invention, it is determined that the button is illuminated.

[0111] After that, character recognition is performed on the illuminated part again. Similar to the recognition of the elevator lobby buttons, referring to the character recognition algorithm in button recognition, let the similarity obtained this time be r0. When it is determined that the values of r and r0 are close to 1, the character set data is obtained as Xn. At the same time, the target floor data Z is compared with Xn. When the known floor data Z is included in Xn, it is determined that the floor has been pressed and no further action is required.

[0112] In addition, the specific method for the four-axis robotic arm of the robot to adaptively press the elevator lobby buttons is as follows:

[0113] When it is determined that the robot needs to perform the step of pressing the elevator lobby buttons, an adaptive robotic arm pressing elevator button control method is adopted. In this design, an adaptive robotic arm control method is adopted, and the specific control method is divided into three stages. In the first step, after the camera locates the approximate position of the elevator button, a rough positioning is performed, and the specific accuracy is within plus or minus 5 cm. Then, a fine adjustment is performed through the interaction between the robotic arm and the camera, and the required target accuracy is within 2 cm. Finally, a judgment program is performed to achieve the function of the robotic arm accurately pressing the elevator button.

[0114] Such as Figure 2As shown, the present invention designs a robotic arm dedicated to elevator buttons. The end of the robotic arm adopts a spherical contact interface (with a diameter of 15 mm and made of yellow silicone with a Shore hardness of A40). By using the spherical end to reduce the contact area (allowing an angular deviation of ±3°), and in cooperation with the coordinated control of four degrees of freedom, a three-level positioning strategy is achieved: rough positioning - precise alignment - program confirmation, significantly improving the success rate of button operations. In terms of four-degree-of-freedom control, the robotic arm is driven by four servo pan-tilts in coordination, capable of achieving multi-directional movements such as front-back, left-right, up-down, etc., enhancing the flexibility of operation. The specific control method is as follows:

[0115] Servo 1 and Servo 3 jointly complete the positioning of front-back and up-down;

[0116] Servo 2 is responsible for the adjustment in the left-right direction;

[0117] Servo 4 controls the small ball at the end to achieve delicate movements and fine adjustments.

[0118] This design scheme improves the operation accuracy and adaptability of the robotic arm, especially suitable for accurate touch control of elevator buttons.

[0119] In addition, the present invention designs an adaptive robotic arm control method aiming to achieve precise control of the robotic arm for elevator buttons. The specific control method can be divided into three main stages, as detailed below:

[0120] The first stage: rough positioning

[0121] In this stage, the system uses a camera to capture image information related to elevator buttons. The positioning algorithm of the camera can quickly identify the approximate position of the elevator buttons and perform rough positioning. The goal of this stage is to determine the preliminary position of the robotic arm relative to the buttons, with the accuracy controlled within plus or minus 5 centimeters. In this way, the robotic arm can roughly narrow down the activity range, laying a foundation for subsequent precise adjustments.

[0122] The second stage: precise adjustment

[0123] After completing the rough positioning, the system enters the second stage, which focuses on achieving higher positioning accuracy. At this time, the robotic arm interacts with the camera and continuously adjusts the position of the robotic arm through real-time feedback to reach a target accuracy within 2 centimeters. This stage relies on high-precision sensors and control algorithms to ensure that the robotic arm can accurately identify the exact position of the elevator buttons, thereby completing the positioning with a smaller error.

[0124] The third stage: judgment program

[0125] After the precise adjustment phase, the final step is to implement a judgment program that can make a judgment based on the current position information of the robotic arm and the target position to ensure that the robotic arm can accurately press the elevator button. This judgment process may include various logical judgments to ensure that when the robotic arm touches the button, it can achieve the optimal force application and angle, thus avoiding operation failures caused by excessive or insufficient force.

[0126] Regarding step S2, the main principle is as follows:

[0127] After calling the elevator, the robot will first judge whether to open or close the elevator door. When the elevator door opens, the robot will make an intelligent judgment on whether to enter the elevator. The robot will identify the situation inside the elevator through a camera. If it identifies that the number of people inside the elevator reaches or exceeds the set threshold (for example, 3 people), or there is insufficient space, the robot will automatically judge to give up entering the elevator. When the robot judges that the number of people inside the elevator is appropriate and the space is sufficient, the robot will enter the elevator to realize the functions of the robot entering the elevator and giving way to pedestrians.

[0128] Step S2 mainly realizes the corresponding function in the following way:

[0129] 1. Ultrasonic-based elevator door opening and closing detection method:

[0130] To implement this monitoring function, the robot uses an ultrasonic module as a key sensor. It emits high-frequency ultrasonic signals and receives reflected waves to detect obstacles in the surrounding environment in real time. This process involves two basic components: a transmitter and a receiver. The transmitter generates ultrasonic waves and sends them to the target in front; when these waves encounter obstacles (such as elevator doors, walls, or other objects), they are reflected back to the receiver. By measuring the time difference between the transmitted signal and the received signal, the ultrasonic module can calculate the distance to the obstacle.

[0131] When the elevator door is in the closed state, the ultrasonic module will generate a stable reflected wave signal and transmit it to the robot's control system. The waveform, intensity, and return time of the reflected signal remain within a fixed range. When the elevator door starts to open, these parameters will change significantly. The opening of the elevator door not only changes the reflection path of the ultrasonic signal but also changes the intensity and arrival time of the reflected wave. Specifically, the opening of the door causes the waveform of some ultrasonic signals to distort, and the return time of the reflected wave increases accordingly.

[0132] By performing real-time data analysis on these changes, the robot system can quickly judge whether the elevator door has been fully opened. Once the system confirms that the state of the elevator door has changed to open, it can continue to execute subsequent tasks

[0133] 2. Intelligent Judgment Method for Whether a Robot Enters the Elevator:

[0134] After the elevator door opens, the robot will make an intelligent judgment on entering the elevator. If it recognizes that the number of people in the elevator reaches or exceeds the set threshold (for example, 3 people), or there is insufficient space, the robot will automatically judge to give up entering the elevator. This decision-making mechanism ensures that the operation of the robot will not interfere with passengers and avoids potential safety hazards.

[0135] Conversely, when the number of people in the elevator is moderate, the robot will further confirm whether all the surrounding people have entered the elevator. During this process, the robot must consider the surrounding dynamics to ensure that no pedestrians are obstructed during the process of entering the elevator. This "giving way to pedestrians" function not only reflects the intelligence of the robot but also provides a more comfortable and safe experience for passengers.

[0136] When the robot arrives at the elevator door smoothly, it will detect the number of people in the elevator by means of a camera installed on the robot. This process usually involves the application of visual recognition algorithms, such as convolutional neural networks (CNNs), to accurately identify and count the passengers in the elevator. The specific method is as follows:

[0137] 1) Method for Recognizing the Number of People in the Elevator Based on Bayesian Classification

[0138] Feature Extraction of Images in the Elevator:

[0139] First, determine the features of the target area. In this design, the Histogram of Oriented Gradients (HOG) features are adopted. The Histogram of Oriented Gradients features are constructed by calculating and statistically analyzing the gradient direction histograms of local regions of the image. In one image, the appearance and shape of the local target can be well described by the distribution of the gradient intensity in the gradient direction. The HOG features focus on the distribution of the shape information of the target, and they can maintain good invariance to geometric and optical deformations of the image, with the advantages of being insensitive to illumination, direction, and size. The HOG features of the image can effectively classify whether there are people in the elevator area, and have greater advantages compared with other feature descriptors in the area detection applied in the present invention.

[0140] The basic steps of HOG feature extraction are as follows:

[0141] (1) Use the Gamma correction method to standardize (normalize) the color space of the input image, aiming to adjust the contrast of the image, reduce the influence caused by local shadows and illumination changes of the image, and at the same time suppress the interference of noise.

[0142] (2) Calculation of Gradient: The gradient of the pixel point (x, y) in the image is:

[0143] G x(x, y) = H(x + 1, y) - H(x - 1, y);

[0144] G y (x, y) = H(x, y + 1) - H(x, y - 1);

[0145] In the formula: G x (x, y), G y (x, y), H(x, y) represent the horizontal direction gradient, vertical direction gradient and pixel value at the input pixel point (x, y) respectively. The gradient magnitude G(x, y) and gradient direction a(x, y) at the pixel point (x, y) are respectively:

[0146]

[0147] Use a one-dimensional discrete differential template to process the image in the horizontal and vertical directions, filter out the data with drastic color changes in the image, and obtain the gradient magnitude and gradient direction of the pixel points.

[0148] Construct a histogram of the gradient direction: First, divide a 50pix×70pix detection image into 7×5 cell units, with the unit size of 10pix×10pix. The gradient direction is quantized into 9, that is, there are 9 histogram channels. Each pixel point in the cell unit takes a weighted vote for the histogram channel based on a certain direction, and the weight value is calculated according to the gradient magnitude of this pixel point.

[0149] (3) Interval normalization: Due to the changes in local illumination and the foreground-background contrast, the variation range of the gradient intensity is very large. Therefore, it is necessary to normalize the gradient intensity. Combine every adjacent 4 units into a normalized block, arrange the histogram vectors of different units in the block in order to form a large feature vector, and normalize the elements of the feature vector. There is a certain degree of overlap between blocks. The gradient direction histograms of 4 cells in each block form a block vector, which can effectively weaken the influence of local image changes after normalization.

[0150] (4) Arrange the feature vectors of different blocks in order to obtain the HOG feature descriptor of the image. In this way, there are a total of 6×4 blocks, and the gradient direction is quantized into 9. Therefore, the dimension of the obtained HOG feature vector is 4×9×6×4 = 864.

[0151] Number recognition based on Bayesian classification:

[0152] The purpose of classification is to establish a classifier based on the existing image features, which can predict the unknown image type. Bayesian classification is based on Bayes' theorem and estimates the posterior probability by training a large number of samples. Two conditions must be met when using a Bayesian classifier:

[0153] (1) The number of categories to be decided for classification is fixed;

[0154] (2) The probability distributions of the overall populations of each category are known.

[0155] In condition (1), assume that there are c pattern classes in the classification problem to be studied, which are represented by w i (i = 1, 2, 3... c). The content studied in the present invention is to identify whether there is a person in the target area under condition (1). In condition (2), assume that the posterior probability P(w i |x) corresponding to the feature vector value x of the object to be identified is known; or the prior probability P(w i ) of the occurrence of each category w i and the class-conditional probability density function P(x|w i ) are known. In this problem, the prior probabilities are P(w1) = P(w2) = 0.5, indicating that according to past experience and analysis, whether there is a person in the seating area occurs with equal probability. The class-conditional probability density function can be obtained through the statistics of the training samples, and condition (2) is satisfied.

[0156] For a two-class classification problem (c = 2), the guiding ideology of the minimum error rate Bayesian classification is: for the feature x, if the probability that it belongs to the pattern class w1 is greater than the probability that it belongs to the pattern class w2, then it is decided that the pattern belongs to the pattern class w1; conversely, it is decided that the pattern belongs to the pattern class w2. Described in mathematical language as follows:

[0157] If P(w1|x) > P(w2|x), then x ∈ w1;

[0158] If P(w1|x) < P(w2|x), then x ∈ w2.

[0159] Among them: P(w1|x) and P(w2|x) are called posterior probabilities.

[0160] The Bayesian formula is:

[0161]

[0162] Considering that P(x) > 0, the above decision rule can be rewritten as:

[0163] If P(x|w1)P(w1) > P(x|w2)P(w2), then x ∈ w1;

[0164] If P(x|w1)P(w1) < P(x|w2)P(w2), then x ∈ w2.

[0165] Where: P(x|w1) and P(x|w2) are the class-conditional probability densities of classes w1 and w2 respectively. The Bayesian classifier is applied to the classification of the regional images in this article, and the classes are the set C = {w1, w2}. Among them, w1 represents the area with people, and w2 represents the area without people.

[0166] The naive Bayesian classifier first selects the training samples and sample features. The training samples need to be manually made. The positive samples are the set of images of the area with people, which is the area with people in the elevator; the negative samples are the set of images of the area without people. The characteristic attributes of the samples determine the quality of the classifier. The HOG features of the regional images are used as the feature vectors for Bayesian classification. In order to simplify the probability distribution model of the feature vectors, a threshold t is set, then:

[0167]

[0168] Among them, x is the returned block, j is the label of the returned block, and j is greater than 0 and less than 864. The HOG features are simplified to 0-1 encoding with 864 dimensions. Calculate the conditional probability estimation of each feature attribute in each class and record the data. Statistically calculate the conditional probability estimations of each feature component under the two classes, that is, P(x1|w1), P(x2|w1), … P(x n |w1), P(x1|w2), P(x2|w2), … P(x n |w2). The above probabilities can be estimated from the training samples:

[0169]

[0170] Among them: S k is the number of training samples in which the j-th feature component has the value x i under the class w j , and S i is the number of samples of w i . x j takes values of 0 or 1, then the conditional probability estimations under the two classes are shown in Table 1 and Table 2 below. P i , T i respectively represent the probabilities that when the sample belongs to classes w1 and w2, the value of the feature x i is 0.

[0171] Table 1 Conditional Probability Estimation Table for Class w1

[0172] Eigenvalue <![CDATA[X1]]> <![CDATA[X2]]> <![CDATA[X3]]> … <![CDATA[X 864 > 0 <![CDATA[P1]]> <![CDATA[P2]]> <![CDATA[P3]]> … <![CDATA[P 864 > 1 <![CDATA[1-P1]]> <![CDATA[1-P2]]> <![CDATA[1-P3]]> … <![CDATA[1-P 864 >

[0173] Table 2 Conditional Probability Estimation Table for Class w2

[0174] Eigenvalue <![CDATA[X1]]> <![CDATA[X2]]> <![CDATA[X3]]> … <![CDATA[X 864 > 0 <![CDATA[T1]]> <![CDATA[T2]]> <![CDATA[T3]]> … <![CDATA[T 864 > 1 <![CDATA[1-T1]]> <![CDATA[1-T2]]> <![CDATA[1-T3]]> … <![CDATA[1-T 864 >

[0175] Use the classifier to classify the item to be classified. Let x = [x1, x2, …, xn T is an item to be classified. According to the decision rule:

[0176] If \(P(x|w_1)P(w_1)>P(x|w_2)P(w_2)\), then \(x\in w_1\);

[0177] If \(P(x|w_1)P(w_1)<P(x|w_2)P(w_2)\), then \(x\in w_2\).

[0178] The key point of modeling lies in the solution of \(P(x|w\) i ). Given \(x = [x_1,x_2,\cdots,x\) n T , then

[0179] \(P(x|w\) i ) = \(P(x_1,x_2,\cdots,x\) n |w\) i );

[0180] Assuming that each feature attribute is conditionally independent, at this time, the class-conditional probability can be simplified as:

[0181]

[0182] The final decision rule is:

[0183] If then \(x\in w_1\); otherwise, \(x\in w_2\).

[0184] When implementing in programming, due to the limited training sample set, if the class-conditional probability of a certain dimension component in the feature vector is 0, it will cause the product in the above formula to be 0, and in this case, the meaning of comparing the posterior probabilities is lost. Therefore, the product of class-conditional probabilities can be changed to summation, and it is modified to the following form:

[0185]

[0186] where \(n = 864\) represents the feature dimension. Substituting each feature component of the item to be classified into the conditional probability distributions of the two categories and comparing the probability values can effectively classify the unknown pictures, determine the part of the picture with people, and thus realize the recognition and statistics of the number of people in the elevator.

[0187] 2) Method for estimating the remaining space in the elevator

[0188] ​​After determining the number of people in the elevator, since the area occupied by each person in the elevator is about 0.2 to 0.3 square meters. The common passenger elevator size on the market is 1.1 meters × 1.4 meters, and the common freight elevator size is 1.5 meters × 1.5 meters, with the height between 2 meters and 4 meters. On this basis, we can estimate the remaining area and volume in the elevator, establish a set with the same size as the number of people in the elevator. Set the number of people in the elevator as n, then establish Zn, and Zn is a set with n elements. Calculate the number of blocks occupied by normal people according to the normal distribution. After counting the number of blocks obtained after identifying each person in the elevator, assign values respectively and store them in Z1, Z2, Z3, ……, Zn; if the number of blocks of person 1 obtained after counting is much less than the normal value, then set Z1 equal to 0.2, and similarly if the number of blocks of person 2 obtained after counting is much more, then set Z1 equal to 0.3. Then the total floor space G occupied by the people in the elevator is:

[0189]

[0190] Compare the obtained proportion G of the people in the elevator with the total floor area F occupied by the elevator (the floor area of the freight elevator is 2.25 square meters, and that of the passenger elevator is 1.54 square meters, which can be calculated according to the specific elevator) to obtain the remaining space A in the elevator, that is, A = F - G; when the value of A is less than the set threshold, it is judged that the remaining space in the elevator is insufficient, and the robot gives up entering the elevator.

[0191] Regarding step S3, the main principle is as follows:

[0192] When the robot enters the elevator, it will rotate the camera through the servo pan-tilt for one week. First, identify the digital matrix through edge recognition. When it is recognized, it will stop rotating and move the digital matrix to the center of the screen. By recognizing the digital information and color information, judge whether the target floor button has been pressed. If not, the robot operates the robotic arm to press the target floor button. Realize the function that the robot can independently press the target floor button after entering the elevator.

[0193] Step S3 mainly realizes the corresponding function in the following way:

[0194] 1. Method for the robot to independently find the matrix button:

[0195] 1) Method for the pan-tilt to drive the vision module to find the button

[0196] In modern robot technology, the application of the multi-modal perception fusion system provides more efficient environment understanding ability for intelligent mobile devices. When the robot recognizes that it has entered the elevator, it will automatically start a preset program. This program first rotates the camera bracket of the robot by 360 degrees to comprehensively detect the environment in the elevator.

[0197] During this rotation process, since the height of the camera is equivalent to that of the matrix keyboard inside the elevator, the position of the keyboard is inevitably covered within its visual range. When the camera recognizes the exact position of the matrix keys, the rotation process will stop immediately. At this time, the robot will perform a fine-tuning operation to adjust the camera's perspective to ensure that the matrix keyboard is centered on the display screen. This process not only improves the operation accuracy of the robot but also further enhances the convenience of human-machine interaction, providing a good foundation for subsequent intelligent operations.

[0198] 2) Method for identifying the position of digital matrix keys based on image processing

[0199] During the rotation of the servo pan-tilt, the vision module will continuously detect the digital matrix keys. When it is determined that the digital key matrix of the elevator is recognized, the servo pan-tilt will stop rotating. The method for identifying the digital matrix keys inside the elevator is similar to the method for identifying the up and down keys in the elevator lobby based on image processing. The specific coordinates of the matrix keys are obtained through edge recognition, and then the coordinate information of the digital keys in the matrix keys is determined through character recognition. For the specific method, refer to the method for identifying the position of elevator lobby keys based on edge detection.

[0200] 2. Method for the robot to press the target floor button:

[0201] 1) Method for judging whether the target floor button is pressed based on color recognition

[0202] In the operation interface of the elevator, after the button is pressed by the user, it usually emits a light signal of a specific color to confirm the successful reception of the instruction. In common cases, these buttons emit blue light. This blue light is not only a signal for feedback status but also provides an important visual mark for floor recognition and interaction of the user interface. The robot judges whether the target floor button is pressed based on color recognition. If the target floor button has been pressed (emitting light), no operation is required. If the target floor button has not been pressed (not emitting light), the robot operates the robotic arm to press the target floor button. For the specific judgment method, refer to the method for identifying the color information of the lobby buttons.

[0203] 2) Method for the robotic arm to press the target floor button

[0204] When it is judged that the target floor button has not been pressed, the robot will operate the robotic arm to press the target floor button according to the coordinates of the matrix button, and then judge the color of the target floor button again, referring to the method for judging whether the target floor button is pressed based on color recognition, to ensure that the target floor has been pressed. The specific method for the robot to operate the robotic arm to press the target floor button refers to the method for the robot to use a four-axis robotic arm to adaptively press the elevator lobby button control method.

[0205] Regarding step S4, the main principle is as follows:

[0206] After the robot determines that the target floor button has been pressed, it will continuously identify the target floor button. When the target floor button changes from glowing to non-glowing, the robot will determine that the target floor has been reached. After that, it will first give a voice prompt to the pedestrian that the target floor has been reached. Then the robot will judge the opening and closing of the elevator door. When the elevator door opens, the robot will leave the elevator to achieve the function of reaching the target floor.

[0207] Step S4 mainly realizes the corresponding function in the following way:

[0208] 1. Judgment method for reaching the target floor based on machine vision:

[0209] The judgment method for the robot to judge reaching the target floor is realized by real-time detection of the target button. After the robot determines that the target floor button has been pressed, it will perform real-time detection on the target floor button. When it is found that the button changes from glowing to non-glowing, it is judged that the target floor has been reached. The recognition of the button refers to the method of recognizing the up and down buttons in the elevator vestibule based on image processing.

[0210] 2. Method for leaving the elevator after reaching the target floor:

[0211] 1) Method for giving a voice prompt to the pedestrian that the target floor has been reached

[0212] When the robot judges that it has reached the destination floor during the process of going to the target floor, the system will trigger the voice broadcast module to work. First, the robot will input specific voice prompt information in advance, such as "Hello, I have reached the target floor. Please pay attention to avoiding." This information can clearly inform the surrounding pedestrians of the upcoming action and prompt them to be prepared to avoid. Subsequently, the robot will send a command to the voice broadcast module, and the command module will call the TTS engine to convert the text into an audio signal. This audio signal is played through the speaker to ensure that the conveyed information can be clearly heard in environments such as the elevator hall or corridor. The volume and intonation are optimized to adapt to the noise level of the surrounding environment, so that pedestrians can easily understand and respond.

[0213] Through this voice prompt mechanism, the robot effectively increases the interaction with pedestrians and improves the safety of the environment. After hearing the prompt that the robot is about to perform an action, pedestrians can better respond and avoid potential collisions or misunderstandings. This precise communication improves the smoothness of the operation and enables pedestrians to feel more at ease when using the elevator or passing through.

[0214] 2) Judgment method for the opening and closing of the elevator door

[0215] After arriving at the target floor, the robot will judge the elevator door again. The specific judgment method refers to the method for detecting the opening and closing of the elevator door based on ultrasonic waves. Finally, after judging that the elevator door has been opened, the robot leaves the elevator and arrives at the target floor, realizing the complete function of autonomous elevator riding.

[0216] In addition, the present invention also provides a humanoid elevator-riding system based on the interaction between machine vision and a robotic arm, including;

[0217] An identification and summons module, used for the robot to identify the current floor information and summon the elevator;

[0218] An interaction module, used for the robot to enter the elevator in a humanoid way;

[0219] An operation module, used for the robot to press the target floor after entering the elevator;

[0220] A judgment and movement module, used for the robot to judge and then intelligently leave the elevator after arriving at the target floor.

[0221] The above is only a detailed description of the preferred embodiments and principles of the present invention. For those of ordinary skill in the art, according to the idea provided by the present invention, there will be changes in the specific implementation manners, and these changes should also be regarded as the protection scope of the present invention.

Claims

1. A humanoid elevator-riding method based on machine vision and robotic arm interaction, characterized in that It includes the following steps: S1. The robot identifies the current floor information and summons the elevator; S2. The robot enters the elevator using a humanoid method; S3. After the robot enters the elevator, it presses the target floor; S4. After the robot determines that it has reached the target floor, it intelligently leaves the elevator.

2. The anthropomorphic elevator-riding method based on machine vision and robotic arm interaction according to claim 1, wherein Step S1 includes the following steps: S11. First, the robot uses lidar to map and identify the current floor. Based on the mapping function, the robot makes a preliminary judgment on the current floor environment to find the location of the elevator on the current floor and autonomously navigates to the elevator lobby; S12. The robot identifies the up and down button information in the elevator lobby, and combines the target floor information to determine whether to press the button. If necessary, the robot uses a four-axis robotic arm adaptive pressing method to autonomously summon the elevator.

3. The anthropomorphic elevator-riding method based on machine vision and robotic arm interaction according to claim 2, wherein Step S11 includes the following steps: When the robot obtains the target position, the movement process starts immediately; the robot uses the high-precision distance obtained by lidar and the short-range detection of ultrasonic sensors to ensure that the robot understands the surrounding dynamic environment and static obstacles; In a dynamic environment, the robot uses a global path planning algorithm to formulate an optimal route; when an obstacle appears in the environment, the robot updates the path in real time and uses a local obstacle avoidance algorithm to actively avoid the obstacle to ensure continuous passage.

4. The anthropomorphic elevator-riding method based on machine vision and robotic arm interaction according to claim 3, wherein In step S12, the four-axis robotic arm adaptive pressing method for pressing the elevator lobby button includes the following steps: S121. Coarse positioning: The robot uses a camera to capture image information related to the elevator button; based on the obtained image information, it quickly identifies the approximate position of the elevator button and performs coarse positioning, with the accuracy controlled within plus or minus 5 centimeters; S122. Precise adjustment: The robotic arm of the robot interacts with the camera and continuously adjusts the position of the robotic arm through real-time feedback to achieve a target accuracy within 2 centimeters; S123. Judgment program: After the precise adjustment stage, the last step is to implement the judgment program, which is used to judge based on the current position information of the robotic arm and the target position to ensure that the robotic arm accurately presses the elevator button.

5. The anthropomorphic elevator-riding method based on machine vision and robotic arm interaction according to claim 4, characterized in that, Step S2 includes the following steps: S21. After summoning the elevator, the robot first judges the opening and closing of the elevator door. When the elevator door opens, the robot makes an intelligent judgment on whether to enter the elevator. The robot uses a camera to identify the situation inside the elevator. If it is identified that the number of people inside the elevator reaches or exceeds the set threshold, or the space margin is insufficient, the robot automatically judges to give up entering the elevator; when the robot judges that the number of people inside the elevator is appropriate and the space margin is sufficient, the robot enters the elevator to complete the operation of the robot entering the elevator and giving way to pedestrians.

6. The anthropomorphic elevator-riding method based on machine vision and robotic arm interaction according to claim 5, wherein In step S21, when the robot uses a camera to identify the situation inside the elevator, it specifically refers to using a method for identifying the number of people inside the elevator based on Bayesian classification and a method for estimating the space margin inside the elevator; the method for identifying the number of people inside the elevator based on Bayesian classification includes the following steps: S211. Extraction of image features inside the elevator: First, determine the characteristics of the target area and adopt the Histogram of Oriented Gradients (HOG) features. The HOG features are constructed by calculating and statistically analyzing the gradient direction histogram of local regions of the image. S212, Person count recognition based on Bayesian classification: Based on the existing image features, establish a classifier for predicting unknown image types. Bayesian classification is based on Bayes' theorem and estimates the posterior probability by training a large number of samples. Among them, the established classifier requires the selection of training samples and sample features. The training samples need to be manually made. The positive samples are the image set of the occupied area, which is the area with people in the elevator. The negative samples are the image set of the unoccupied area. The HOG features of the regional images are used as the feature vectors for Bayesian classification. The method for estimating the remaining space in the elevator includes the following steps: S213, After determining the number of people in the elevator, since the area occupied by each person in the elevator is about 0.2 to 0.3 square meters. The common passenger elevator sizes on the market are 1.1 meters × 1.4 meters, and the common freight elevator sizes are 1.5 meters × 1.5 meters, with the height between 2 meters and 4 meters. Based on this, we can estimate the remaining area and volume in the elevator, establish a set with the same size as the number of people in the elevator. Set the number of people in the elevator as n, then establish Zn, where Zn is a set with n elements. Calculate the number of blocks occupied by normal people according to the normal distribution. After counting the number of blocks obtained by identifying each person in the elevator and assigning values respectively, store them in Z1, Z2, Z3, ……, Zn. Then the total floor space G occupied by the people in the elevator is: Compare the obtained proportion G of the people in the elevator with the total floor area F occupied by the elevator to get the remaining space A in the elevator, that is, A = F - G. When the value of A is less than the set threshold, it is judged that the remaining space in the elevator is insufficient, and the robot gives up entering the elevator.

7. The anthropomorphic elevator-riding method based on machine vision and robotic arm interaction according to claim 6, wherein, Step S3 includes the following steps: S31, When the robot enters the elevator, rotate the camera through the servo pan-tilt for one week. First, identify the digital matrix through edge recognition. When it is recognized, stop rotating and move the digital matrix to the center of the screen. S32, By identifying the digital information and color information, judge whether the target floor button has been pressed. If not, the robot operates the robotic arm to press the target floor button to complete the operation of the robot autonomously pressing the target floor button after entering the elevator.

8. The anthropomorphic elevator-riding method based on machine vision and robotic arm interaction according to claim 7, characterized in that In step S32, the specific process of judging whether the target floor button has been pressed is as follows: In the operation interface of the elevator, after the button is pressed by the user, it emits a light signal of a specific color to confirm the successful reception of the command. If the target floor button has been pressed, that is, the target floor button is lit, no operation is required. If the target floor button has not been pressed, that is, the target floor button is not lit, the robot operates the robotic arm to press the target floor button.

9. The anthropomorphic elevator-riding method based on machine vision and robotic arm interaction according to claim 8, wherein Step S4 includes the following steps: S41, After the robot judges that the target floor button has been pressed, continuously identify the target floor button. When the target floor button changes from lit to unlit, the robot judges that the target floor has arrived. S42. After reaching the target floor, the robot makes a voice announcement to prompt pedestrians that the target floor has been reached. Subsequently, the robot uses an ultrasonic sensor to judge the opening and closing of the elevator door. When the elevator door opens, the robot leaves the elevator to complete the operation of arriving at the target floor.

10. A humanoid elevator system based on machine vision and robotic arm interaction, for implementing the humanoid elevator method based on machine vision and robotic arm interaction according to any one of claims 1-9, characterized in that, The humanoid elevator-riding system based on machine vision and robotic arm interaction includes: An identification and summons module for the robot to identify the current floor information and summon the elevator; An interaction module for the robot to enter the elevator using humanoid methods; An operation module for the robot to press the target floor after entering the elevator; A judgment and movement module for the robot to intelligently leave the elevator after judging that it has reached the target floor.

Images