Elevator floor positioning method, device, terminal equipment and storage medium

Abstract

The present application is applicable to the technical field of intelligent identification of elevator floors, and provides an elevator floor positioning method, device, terminal equipment and storage medium. In the embodiment of the present application, image information is acquired through an image acquisition device built into the elevator, and a display panel image and a floor button image are extracted from the above image information; image recognition is performed on the above display panel image and the above button image respectively, and the recognition results of the above display panel image and the above button image are determined; the floor positioning result is determined according to the recognition results of the above display panel image and the above button image, and a positioning request sent by a robot is acquired, and the corresponding floor positioning result is sent to the above robot according to the above positioning request, thereby improving the accuracy of the elevator floor positioning result.

Description

Technical Field

[0001] The present application belongs to the technical field of intelligent identification of elevator floors, and in particular, relates to a method, device, terminal equipment and storage medium for positioning elevator floors. Background Art

[0002] With the development of society, robots are becoming more and more common in people's lives, and people are becoming more and more accustomed to using robots to assist people in their work. As the demand increases, robots need to have the ability to take the elevator autonomously. Therefore, how to accurately obtain the current floor information of the elevator, so as to enable the robot to take the elevator autonomously based on this information, has become an important factor in determining whether the robot can take the elevator autonomously.

[0003] At present, an image recognition unit is set on the robot shell to recognize the elevator data, so that the robot can perform the operation of entering and exiting the elevator according to the recognized data. However, since the image recognition unit is always in motion, the deviation of the image information obtained at different times is too large, and the image information is blocked due to the height of the robot and the occlusion of the elevator personnel, resulting in the inaccurate recognized data, which leads to the low accuracy of the elevator floor positioning result. Summary of the invention

[0004] The embodiments of the present application provide an elevator floor positioning method, apparatus, terminal device and storage medium, which can solve the problem of low accuracy of elevator floor positioning results.

[0005] In a first aspect, an embodiment of the present application provides an elevator floor positioning method, comprising:

[0006] Acquire image information through an image acquisition device built into the elevator, and extract a display panel image and a floor button image from the image information;

[0007] Performing image recognition on the display panel image and the button image respectively, and determining recognition results of the display panel image and the button image;

[0008] Determining a floor positioning result according to the recognition results of the display panel image and the button image;

[0009] A positioning request sent by the robot is obtained, and a corresponding floor positioning result is sent to the robot according to the positioning request.

[0010] In a second aspect, an embodiment of the present application provides an elevator floor positioning device, comprising:

[0011] An extraction module, used to obtain image information through an image acquisition device built into the elevator, and extract a display panel image and a floor button image from the image information;

[0012] a recognition module, configured to perform image recognition on the display panel image and the button image respectively, and determine recognition results of the display panel image and the button image;

[0013] A result determination module, used to determine a floor positioning result according to the recognition results of the display panel image and the button image;

[0014] The acquisition module is used to acquire the positioning request sent by the robot, and send the corresponding floor positioning result to the robot according to the positioning request.

[0015] In a third aspect, an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the above-mentioned elevator floor positioning methods when executing the computer program.

[0016] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned elevator floor positioning methods are implemented.

[0017] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device executes any one of the elevator floor positioning methods in the first aspect above.

[0018] In the embodiment of the present application, image information is obtained by an image acquisition device built into the elevator to avoid the problem of excessive deviation of image information obtained at different times, and the display panel image and the floor button image are extracted from the above image information to exclude the interference image in the image information to improve the accuracy of image recognition. By performing image recognition on the above display panel image and the above button image respectively, a double detection of the current floor is achieved. After determining the recognition results of the above display panel image and the above button image, the floor positioning result is determined according to the recognition results of the above display panel image and the above button image, thereby further improving the accuracy of the elevator floor positioning result. After obtaining the positioning request sent by the robot, the corresponding floor positioning result is sent to the robot according to the above positioning request, so that the robot performs corresponding operations according to the elevator floor positioning result with higher accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0020] Figure 1 This is a schematic diagram of a first flow chart of an elevator floor positioning method provided in an embodiment of the present application;

[0021] Figure 2 is a schematic diagram of the structure of a positioning device provided in an embodiment of the present application;

[0022] Figure 3 This is a second flow chart of the elevator floor positioning method provided in the embodiment of the present application;

[0023] Figure 4 This is a schematic diagram of the first structure of the elevator floor display panel provided in the embodiment of the present application;

[0024] Figure 5 This is a second structural schematic diagram of an elevator floor display panel provided in an embodiment of the present application;

[0025] Figure 6 It is a structural schematic diagram of an elevator floor positioning device provided in an embodiment of the present application;

[0026] Figure 7 It is a structural diagram of the terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0028] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0029] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0030] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0031] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0032] Figure 1 The figure shows a flow chart of an elevator floor positioning method in an embodiment of the present application. The execution subject of the method may be a terminal device, such as a positioning device. This embodiment is described by taking the positioning device as an example. Figure 1 As shown, the above elevator floor positioning method may include the following steps:

[0033] Step S101: Acquire image information through an image acquisition device built into the elevator, and extract a display panel image and a floor button image from the image information.

[0034] In this embodiment, if Figure 2As shown, the positioning device 10 is arranged in the elevator car 12, and the image information is obtained by the image acquisition device 101 installed at the elevator ceiling 13 in the device. The image acquisition device 101 can use a digital camera module to capture the image information in the visual field. The image processing device 102 in the positioning device 10 processes the acquired image information, and extracts the display panel 14 image and the floor button 15 image from the image information, so that the positioning device 10 can realize floor positioning according to the acquired display panel 14 image and the floor button 15 image. Among them, the above-mentioned image information includes a combination of at least two of the display panel 14 image of the elevator floor, the floor button 15 image and the elevator car door 16 image; the above-mentioned positioning device 10 includes an image acquisition device 101, an image processing device 102, an image recognition device 103, a storage device 104 and a wireless communication device 105. By setting the positioning device 10 in the elevator car 12, it is possible to avoid construction on the outside of the elevator, resulting in lower costs. After a simple on-site deployment and setting, the positioning device 10 can be applied to all models of elevators on the market, without the need for multiple developments due to different elevator protocols. The various devices in the above positioning device can also be deployed separately, the image acquisition device is installed in the elevator car 12, and the processing devices in the positioning device including the image processing device 102, the image recognition device 103, the storage device 104 and the wireless communication device 105 are installed at the robot body or other suitable locations, which can be set according to user needs.

[0035] Exemplarily, the above-mentioned extraction of the display panel 14 image and the floor button 15 image from the image information may include: extracting the feature graphics of preset specifications in the image information by using the Hough transform through the image processing device 102 in the positioning device 10. Then the image information is cropped according to the extracted feature graphics, so as to determine the display panel 14 image and the button image in the image information. Among them, the above-mentioned display panel image displays the floor number of the above-mentioned elevator; the above-mentioned button image displays the on and off state of the background light of the button corresponding to the selected floor; the above-mentioned preset specification feature graphics include at least one of feature lines, feature shapes, and feature points. For example, feature points are set on the display panel 14 inside the elevator, and the image processing device 102 can obtain the required image area according to the above-mentioned set feature points through the Hough transform, and the image can be an AOI (area of ​​interest) image.

[0036] It is understandable that before extracting the feature graphics of preset specifications in the image information, the feature graphics of preset specifications required by the image processing device 102 to process the image need to be stored in the storage device 104 in the positioning device 10. In order to improve the accuracy of image extraction, the feature graphics of preset specifications can be determined using the current elevator scene image. For example, after the positioning device 10 is installed in the elevator, the image acquisition device 101 in the positioning device 10 is used to acquire the current image, and the image processing device 102 identifies the feature lines of the elevator floor display panel 14, thereby obtaining the size of the feature lines, and the size of the feature lines is input into the storage unit, so as to extract the display panel 14 image according to the feature lines later.

[0037] Exemplarily, the above-mentioned extraction of the image of the display panel 14 and the image of the floor button 15 from the image information may include: the image processing device 102 in the positioning device 10 extracts the image of the display panel 14 and the image of the floor button 15 from the image information according to the preset absolute coordinate value. Since the image acquisition unit is fixed on the elevator ceiling 13 and the use environment is not easy to change, the captured image has a high consistency, so the required image can be extracted according to the absolute coordinate value of the required image area preset and stored in the storage unit.

[0038] It is understandable that before extracting the image of the display panel 14 and the image of the floor button 15 from the image information according to the preset absolute coordinate values, the absolute coordinate values ​​required for the image processing device 102 to process the image need to be stored in the storage device 104 in the positioning device 10. Specifically, after the positioning device 10 is installed, the absolute coordinate position of the display screen in the image is input to the storage unit.

[0039] In one embodiment, in order to improve the accuracy of image recognition by the image recognition device 103 and to improve the corresponding processing speed, after extracting the display panel 14 image and the floor button 15 image from the image information, the display panel 14 image and the floor button 15 image may be reprocessed, which may specifically include: the image processing device 102 may crop the image other than the display panel 14 image and the floor button 15 image; the image processing device 102 may also perform a quadrilateral transformation on the display panel 14 image and the floor button 15 image to obtain a front view of the display panel 14 image and the floor button 15 image; the image processing device 102 may also convert the display panel 14 image and the floor button 15 image into grayscale images respectively; the image processing device 102 may also reduce the size of the display panel 14 image and the floor button 15 image to speed up the processing speed of the image processing device 102; the image processing device 102 may also adopt an image morphological processing method to corrode and dilate the display panel 14 image and the floor button 15 image, thereby removing interference in the image and improving the accuracy of the later result recognition.

[0040] In one embodiment, if the image of the display panel 14 and / or the image of the floor button 15 in the currently acquired image information is blocked, resulting in the inability to accurately determine the floor result corresponding to the image, the image information can be re-acquired for judgment; the blockage may be caused by too many people in the elevator, or by factors such as stains on the display panel 14 and the floor button 15, so an alarm device can be integrated to sound an alarm to remind the elevator personnel to move, avoid the image information being blocked, or remind relevant staff to check.

[0041] Step S102 : performing image recognition on the display panel 14 image and the button image respectively, and determining recognition results of the display panel 14 image and the button image.

[0042] In this embodiment, the image recognition device 103 in the positioning device 10 performs image recognition on the display panel 14 image and the button image in a preset manner according to the floor images corresponding to each floor stored in the storage device 104, and determines the recognition results of the display panel 14 image and the button image, wherein the recognition result of the above-mentioned display panel 14 image is the floor currently displayed by the display panel 14; the recognition result of the above-mentioned button image is the display change result of the elevator button background light; the above-mentioned preset method includes image pixel level comparison, machine learning, deep learning and other methods; the above-mentioned each floor image includes the image displayed on the display panel 14 of each floor, the image of the switch and light status of the elevator car door 16, and the image of the background light on and off of each floor button 15.

[0043] It is understandable that before the image recognition of the display panel 14 image and the button image is performed respectively, the images of each floor required for the image recognition device 103 to recognize the image and the label values ​​set by the staff for each floor need to be stored in the storage device 104 in the positioning device 10, and the format of the pre-stored images of each floor is consistent with the format of the image to be recognized. Specifically, in order to improve the accuracy of image extraction, the current elevator can be used to determine the images of each floor. For example, after the positioning device 10 is installed, the image acquisition device 101 sequentially acquires the images displayed on the display panel 14 of each floor, and performs image processing through the image processing device 102. For the processed display panel 14 images of each floor, the floor labels are set in turn. If the current display panel 14 image is the first floor, the floor label is set to the first floor. Correspondingly, the staff stores the images of the background light on and off of the buttons 15 of each floor, and sets the floor labels in turn.

[0044] In one embodiment, Figure 3 As shown, the above step S102 includes:

[0045] Step S301: Calculate the grayscale difference of each pixel point between the image of the display panel 14 and the preset images of each floor panel.

[0046] Step S302 : taking the floor number corresponding to the floor panel image and having the grayscale difference of a preset number of pixels within a preset range as the recognition result of the image of the display panel 14 .

[0047] In this embodiment, the image recognition device 103 compares the display panel 14 image with the preset floor panel images. If the grayscale value deviation between the pixel points of the display panel 14 image and the preset floor panel image for comparison is within the threshold range, it can be considered that the grayscale of the pixel points compared by the above two images is consistent, and then the number of pixel points whose grayscale value deviation of the above two images is within the threshold range is accumulated. If the number is greater than or equal to the minimum number of identical pixel points when the images are consistent, it is considered that the images are consistent, that is, the display panel 14 image and the preset floor panel image for comparison are on the same floor, wherein the minimum number of identical pixel points when the images are consistent is the above preset number. When it is determined that the two images are the same, the floor number corresponding to the preset floor panel image for comparison is used as the recognition result of the current display panel 14 image.

[0048] Step S303: Calculate the grayscale difference of each pixel point between the button image and the preset images of each floor button 15.

[0049] Step S304: taking the floor labeling information corresponding to the floor button 15 image whose grayscale difference of a preset number of pixels is within a preset range as the recognition result of the button image.

[0050] In this embodiment, the image recognition device 103 compares the button image with the preset images of each floor button 15. If the grayscale value deviation between the button image and the preset floor button 15 image for comparison is within the threshold range, it can be considered that the grayscale of the pixels compared by the above two images is consistent, and then the number of pixels whose grayscale value deviations of the above two images are within the threshold range is accumulated. If the number is greater than or equal to the minimum number of identical pixels when the images are consistent, it is considered that the images are consistent, that is, the button image and the preset floor button 15 image for comparison are the same floor. When it is determined that the two images are the same, the floor labeling information corresponding to the preset floor button 15 image for comparison is used as the recognition result of the current button image.

[0051] In one embodiment, the above step S102 includes: the image recognition device 103 inputs the display panel 14 image and the button image into the network model preset therein for image recognition, thereby determining the recognition results of the display panel 14 image and the button image. The above network model can be trained by the images of each floor in the elevator, thereby ensuring the reliability of the output result of the image recognition device 103.

[0052] Exemplarily, the image recognition according to the network model may include: the image recognition device 103 reads parameters pre-stored in the storage unit for digital information training, and then uses the network model to digitally recognize the image on the display panel 14. Accordingly, the machine learning parameters called by the image recognition device 103 and the machine learning parameters after enhanced training are used to train the network model, and the images of the corresponding scenes pre-stored in the storage unit may be further called for targeted enhanced training.

[0053] Exemplarily, the image recognition according to the network model may also include: the image recognition device 103 retrieves the floor images of each floor pre-stored in the storage unit, and then uses logistic regression or convolutional neural network to recognize the display panel 14 image and the button image. Accordingly, the image recognition device 103 calls the floor images and corresponding floor labels pre-stored in the storage device 104 to train the network model using logistic regression or convolutional neural network.

[0054] In one embodiment, due to the difference in ambient light, image recognition using pixel-level comparison may not produce accurate results, and image recognition using machine learning may also produce erroneous results. Therefore, in conjunction with the manufacturer of the elevator, a real-time tracking and recognition function for the background light of the elevator floor button 15 can be added to assist in floor recognition, thereby improving the accuracy of floor recognition.

[0055] Step S103: Determine the floor positioning result according to the recognition result of the display panel 14 image and the button image.

[0056] In this embodiment, the current elevator floor can be determined based on the recognition result of the display panel 14 image, and further based on the recognition result of the button image, it is determined whether the background light of the button corresponding to the elevator floor determined by the current display panel 14 image is off. If it is, it means that the current floor positioning result is the elevator floor determined by the current display panel 14 image. It can be understood that Figure 4 and Figure 5 As shown in the figure, different display schemes of the floor display panel 14 adopted by different elevator manufacturers are shown. The display panel 14 can be Figure 4 The broken code dot matrix shown can also be Figure 5The digital fonts displayed on the LCD screens shown are different. If the image recognition is based only on the image of the display panel 14, the reliability of the result will be low. Therefore, the recognition result of the button image can be further used to more accurately determine the floor positioning result.

[0057] In one embodiment, after step S103, the following includes: the image processing device 102 in the positioning device 10 extracts features from the image information, determines the elevator scene image in the image information, and performs image recognition on the elevator scene image through the image recognition device 103 in the positioning device 10, thereby determining the recognition result of the elevator scene image; if it is determined that the recognition result of the elevator scene image is that a robot exists inside the elevator, the positioning device needs to send the currently determined floor positioning result to the robot inside the elevator.

[0058] Step S104: Obtain the positioning request sent by the robot, and send the corresponding floor positioning result to the robot according to the positioning request.

[0059] In this embodiment, the wireless communication device 105 in the positioning device 10 obtains a positioning request sent by the robot 11 through the wireless communication device 105 set on its body, and the positioning device 10 sends the floor positioning result to the robot 11 through the wireless communication device 105 according to the positioning request, so that the robot 11 performs corresponding operations according to the floor positioning result.

[0060] In one embodiment, after step S104, the following includes: the image processing device 102 in the positioning device 10 extracts features from the image information, determines the image of the elevator car door 16 in the image information, and performs image recognition on the image of the elevator car door 16 through the image recognition device 103 in the positioning device 10, thereby determining the recognition result of the image of the elevator car door 16, and the positioning device further determines the state of the elevator car door according to the recognition result of the above-mentioned image of the elevator car door. The positioning device 10 sends the determined state of the elevator car door to the robot according to the request sent by the robot 11, so that the robot 11 performs corresponding operations according to the floor positioning result and the state of the elevator car door 16. Among them, when it is detected that the elevator car door 16 is fully opened and the corresponding light state also meets the preset light state, it is determined that the recognition result of the above-mentioned image of the elevator car door 16 is the state of the car door opening.

[0061] Exemplarily, if the robot 11 is currently outside or inside the elevator, the robot 11 sends a request to the positioning device 10, and the positioning device 10 sends the identified floor positioning result and the recognition result of the elevator car door 16 image to the robot 11 through the wireless communication device 105 according to the request sent by the robot 11, so that the robot 11 can determine whether it can enter or exit the elevator based on the floor positioning result and the recognition result of the elevator car door 16 image.

[0062] In one embodiment, after obtaining the positioning request sent by the robot in step S104, it includes: when the positioning device 10 detects that the floor positioning result meets the target floor in the positioning request and the elevator door 16 is in the open state, the positioning device 10 sends a reminder signal to the robot 11 through the wireless communication device 105, so that the robot 11 performs corresponding operations according to the reminder signal.

[0063] Exemplarily, if the robot 11 is currently outside or inside the elevator, the robot 11 sends a request to the positioning device 10, and the positioning device 10 makes a judgment based on the request sent by the robot 11. When the identified floor positioning result meets the target floor in the request sent by the robot 11 and the elevator car door 16 is in the open state, the positioning device 10 sends a reminder signal to the robot 11 through its internal wireless communication device 105, so that the robot 11 enters the elevator according to the reminder signal. Among them, the above-mentioned floor positioning result is that the recognition result of the image of the display panel 14 is the target floor, and the recognition result of the button image is that the button corresponding to the target floor is in the off state. The above-mentioned positioning device 10 can obtain the current floor state of the elevator more safely and reliably by simultaneously monitoring and identifying the current floor information displayed on the elevator floor display panel 14, the background light of the target floor of the robot 11 on the elevator button panel, and the state of the elevator car door 16, and send the current elevator floor state to the robot 11 waiting in the waiting area or in the elevator car 12 through the wireless communication unit.

[0064] In the embodiment of the present application, image information is obtained by an image acquisition device built into the elevator to avoid the problem of excessive deviation of image information obtained at different times, and the display panel image and the floor button image are extracted from the above image information to exclude the interference image in the image information to improve the accuracy of image recognition. By performing image recognition on the above display panel image and the above button image respectively, a double detection of the current floor is achieved. After determining the recognition results of the above display panel image and the above button image, the floor positioning result is determined according to the recognition results of the above display panel image and the above button image, thereby further improving the accuracy of the elevator floor positioning result. After obtaining the positioning request sent by the robot, the corresponding floor positioning result is sent to the robot according to the above positioning request, so that the robot performs corresponding operations according to the elevator floor positioning result with higher accuracy.

[0065] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0066] Corresponding to the elevator floor positioning method described above, Figure 6FIG. 1 is a schematic diagram of the structure of an elevator floor positioning device according to an embodiment of the present application. Figure 6 As shown, the above-mentioned elevator floor positioning device may include:

[0067] The extraction module 601 is used to obtain image information through the built-in image acquisition device of the elevator, and extract the display panel image and the floor button image from the image information.

[0068] The recognition module 602 is used to perform image recognition on the display panel image and the button image respectively, and determine the recognition results of the display panel image and the button image.

[0069] A result determination module 603, used to determine the floor positioning result according to the recognition results of the display panel image and the button image;

[0070] The acquisition module 604 is used to acquire the positioning request sent by the robot, and send the corresponding floor positioning result to the robot according to the positioning request.

[0071] In one embodiment, the extraction module 601 may include:

[0072] The first extraction unit is used to extract characteristic patterns of preset specifications from the image information.

[0073] The cropping unit is used to crop the image information according to the characteristic pattern to determine the display panel image and the button image; the display panel image displays the floor number of the elevator, and the button image displays the on and off status of the background light of the button corresponding to the selected floor.

[0074] In one embodiment, the extraction module 601 may include:

[0075] The second extraction unit is used to extract the display panel image and the floor button image from the image information according to the preset absolute coordinate values.

[0076] In one embodiment, the identification module 602 may include:

[0077] The first calculation unit is used to calculate the grayscale difference of each pixel point between the display panel image and the preset floor panel images.

[0078] The first recognition unit is used to use the floor number corresponding to the floor panel image whose grayscale difference of a preset number of pixels is within a preset range as the recognition result of the display panel image.

[0079] The second calculation unit is used to calculate the grayscale difference of each pixel point between the button image and the preset button images of each floor.

[0080] The second recognition unit is used to use the floor labeling information corresponding to the floor button image whose grayscale difference values ​​of a preset number of pixels are within a preset range as the recognition result of the button image.

[0081] In one embodiment, the identification module 602 may include:

[0082] The training unit is used to input the display panel image and the button image into a preset network model for image recognition, and determine the recognition results of the display panel image and the button image; the above network model is trained by the images of each floor in the elevator.

[0083] In one embodiment, the elevator floor positioning device may further include:

[0084] The first feature extraction module is used to extract features from the image information and determine the elevator scene image in the image information.

[0085] The result sending module is used to send the floor positioning result to the robot when a robot is identified to exist inside the elevator according to the elevator scene image.

[0086] In one embodiment, the elevator floor positioning device may further include:

[0087] The second feature extraction module is used to extract features from the image information and determine the elevator door image in the image information.

[0088] The first image recognition module is used to perform image recognition on the elevator door image and determine the recognition result of the elevator door image.

[0089] The state determination module is used to determine the state of the elevator door according to the recognition result of the elevator door image.

[0090] The first execution module is used to send the elevator door status to the robot according to the positioning request, so that the robot performs corresponding operations according to the floor positioning result and the elevator door status.

[0091] In one embodiment, the elevator floor positioning device may further include:

[0092] The second execution module is used to send a reminder signal when it is detected that the floor positioning result meets the target floor in the positioning request and the elevator door state is the door open state, so that the robot performs a corresponding operation according to the reminder signal.

[0093] In the embodiment of the present application, image information is obtained by an image acquisition device built into the elevator to avoid the problem of excessive deviation of image information obtained at different times, and the display panel image and the floor button image are extracted from the above image information to exclude the interference image in the image information to improve the accuracy of image recognition. By performing image recognition on the above display panel image and the above button image respectively, a double detection of the current floor is achieved. After determining the recognition results of the above display panel image and the above button image, the floor positioning result is determined according to the recognition results of the above display panel image and the above button image, thereby further improving the accuracy of the elevator floor positioning result. After obtaining the positioning request sent by the robot, the corresponding floor positioning result is sent to the robot according to the above positioning request, so that the robot performs corresponding operations according to the elevator floor positioning result with higher accuracy.

[0094] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned system embodiments and method embodiments, and will not be repeated here.

[0095] Figure 7 This is a schematic diagram of the structure of a terminal device provided in an embodiment of the present application. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0096] like Figure 7 As shown, the terminal device 7 of this embodiment includes: at least one processor 700 ( Figure 7 Only one is shown in the figure), a memory 701 connected to the processor 700, and a computer program 702 stored in the memory 701 and executable on the at least one processor 700, such as an elevator floor positioning program. When the processor 700 executes the computer program 702, the steps in the above-mentioned various elevator floor positioning method embodiments are implemented, such as Figure 1 Alternatively, when the processor 700 executes the computer program 702, the functions of the modules in the above-mentioned device embodiments are realized, for example, Figure 6 Functions of modules 601 to 604 are shown.

[0097] Exemplarily, the computer program 702 may be divided into one or more modules, which are stored in the memory 701 and executed by the processor 700 to complete the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 702 in the terminal device 7. For example, the computer program 702 may be divided into an extraction module 601, an identification module 602, a result determination module 603, and an acquisition module 604, and the specific functions of each module are as follows:

[0098] An extraction module 601 is used to obtain image information through an image acquisition device built into the elevator, and extract a display panel image and a floor button image from the image information;

[0099] The recognition module 602 is used to perform image recognition on the display panel image and the button image respectively, and determine the recognition results of the display panel image and the button image;

[0100] A result determination module 603, used to determine the floor positioning result according to the recognition results of the display panel image and the button image;

[0101] The acquisition module 604 is used to acquire the positioning request sent by the robot, and send the corresponding floor positioning result to the robot according to the positioning request.

[0102] The terminal device 7 may include, but is not limited to, a processor 700 and a memory 701. Those skilled in the art will appreciate that Figure 7 It is only an example of the terminal device 7 and does not constitute a limitation on the terminal device 7. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, buses, etc.

[0103] The processor 700 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0104] In some embodiments, the memory 701 may be an internal storage unit of the terminal device 7, such as a hard disk or memory of the terminal device 7. In other embodiments, the memory 701 may also be an external storage device of the terminal device 7, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 7. Further, the memory 701 may also include both an internal storage unit and an external storage device of the terminal device 7. The memory 701 is used to store an operating system, an application program, a boot loader, data, and other programs, such as the program code of the computer program. The memory 701 may also be used to temporarily store data that has been output or is to be output.

[0105] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the above-mentioned device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0106] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0107] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0108] In the embodiments provided in the present application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic. For example, the division of the above modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0109] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0110] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0111] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The above-mentioned computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the above-mentioned computer program includes computer program code, and the above-mentioned computer program code can be in source code form, object code form, executable file or some intermediate form. The above-mentioned computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0112] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. An elevator floor positioning method, characterized in that: include: Acquire image information through an image acquisition device built into the elevator, and extract a display panel image and a floor button image from the image information; Performing image recognition on the display panel image and the button image respectively, and determining recognition results of the display panel image and the button image; Determining the floor positioning result according to the recognition results of the display panel image and the button image includes: calculating the grayscale difference of each pixel between the display panel image and each preset floor panel image; taking the floor number corresponding to the floor panel image whose grayscale difference of a preset number of pixels is within a preset range as the recognition result of the display panel image; Calculating the grayscale difference of each pixel between the button image and each preset floor button image; taking the floor labeling information corresponding to the floor button image whose grayscale difference of a preset number of pixels is within a preset range as the recognition result of the button image; Alternatively, the display panel image and the button image are respectively input into a preset network model for image recognition to determine recognition results of the display panel image and the button image; the network model is trained by images of each floor in the elevator; Determine the current elevator floor according to the recognition result of the display panel image, and determine whether the background light of the button corresponding to the current elevator floor determined by the current display panel image is off according to the recognition result of the button image. If it is off, it means that the current floor positioning result is the elevator floor determined by the current display panel image. Obtaining a positioning request sent by a robot outside the elevator, and sending a corresponding floor positioning result to the robot according to the positioning request; Extracting features from the image information to determine an elevator door image in the image information; Performing image recognition on the elevator door image to determine a recognition result of the elevator door image; Determining the state of the elevator door according to the recognition result of the elevator door image; wherein, when it is detected that the elevator door is fully opened and the light state of the background light of the button corresponding to the elevator floor in the image information meets the preset light state, it is determined that the recognition result of the elevator door image is the door open state; When it is detected that the floor positioning result meets the target floor in the positioning request and the elevator door state is open, a reminder signal is sent to the robot through the wireless communication device, so that the robot performs corresponding operations according to the reminder signal.

2. The elevator floor positioning method according to claim 1, characterized in that: The step of extracting the display panel image and the floor button image from the image information comprises: Extracting characteristic patterns of preset specifications from the image information; The image information is cropped according to the characteristic pattern to determine the display panel image and the button image; the display panel image displays the floor number of the elevator, and the button image displays the on and off status of the background light of the button corresponding to the selected floor.

3. The elevator floor positioning method according to claim 1, characterized in that: The step of extracting the display panel image and the floor button image from the image information comprises: The display panel image and the floor button image are extracted from the image information according to the preset absolute coordinate values.

4. The elevator floor positioning method according to any one of claims 1 to 3, characterized in that: After determining the floor positioning result according to the recognition results of the display panel image and the button image, the method includes: Extracting features from the image information to determine an elevator scene image in the image information; When a robot is identified to exist inside the elevator according to the elevator scene image, the floor positioning result is sent to the robot.

5. An elevator floor positioning device, characterized in that: include: An extraction module, used to obtain image information through an image acquisition device built into the elevator, and extract a display panel image and a floor button image from the image information; a recognition module, configured to perform image recognition on the display panel image and the button image respectively, and determine recognition results of the display panel image and the button image; The recognition module includes: a first calculation unit, which is used to calculate the grayscale difference of each pixel between the display panel image and each preset floor panel image; a first recognition unit, which is used to take the floor number corresponding to the floor panel image whose grayscale difference of a preset number of pixels is within a preset range as the recognition result of the display panel image; a second calculation unit, which is used to calculate the grayscale difference of each pixel between the key image and each preset floor key image; a second recognition unit, which is used to take the floor labeling information corresponding to the floor key image whose grayscale difference of a preset number of pixels is within a preset range as the recognition result of the key image; or, the recognition module includes: a training unit, which is used to input the display panel image and the key image into a preset network model for image recognition, and determine the recognition results of the display panel image and the key image; the network model is trained by the images of each floor in the elevator; the recognition module is also used to determine the current elevator floor according to the recognition result of the display panel image, and determine whether the background light of the key corresponding to the current elevator floor determined by the current display panel image is in an off state according to the recognition result of the key image, and if it is in an off state, it means that the current floor positioning result is the elevator floor determined by the current display panel image; A result determination module, used to determine a floor positioning result according to the recognition results of the display panel image and the button image; An acquisition module, used to acquire a positioning request sent by a robot outside the elevator, and send a corresponding floor positioning result to the robot according to the positioning request; A second feature extraction module is used to extract features from the image information and determine the elevator door image in the image information; A first image recognition module, used to perform image recognition on the elevator door image and determine the recognition result of the elevator door image; A state determination module, configured to determine the state of the elevator door according to the recognition result of the elevator door image; wherein, when it is detected that the elevator door is fully opened and the light state of the background light of the button corresponding to the elevator floor in the image information meets the preset light state, the recognition result of the elevator door image is determined to be the door open state; The second execution module is used to send a reminder signal to the robot through a wireless communication device when it is detected that the floor positioning result meets the target floor in the positioning request and the elevator door state is open, so that the robot performs a corresponding operation according to the reminder signal.

6. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of an elevator floor positioning method as described in any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of an elevator floor positioning method as described in any one of claims 1 to 4 are implemented.

Images