Methods, devices, electronic equipment, and readable storage media for handling indoor environments.

Abstract

This disclosure provides a method, apparatus, electronic device, and readable storage medium for handling movement in an indoor environment, relating to the fields of data processing technology and image processing technology, and particularly to artificial intelligence technologies such as computer vision, intelligent voice technology, and intelligent transportation technology. The specific implementation involves: acquiring a positioning command provided by a user within the indoor environment, the positioning command including gesture commands and voice commands; obtaining, based on the positioning command, the positioning location information and positioning description information of the location indicated by the positioning command in the indoor environment; and associating the positioning location information and the positioning description information for automatic movement within the indoor environment.

Description

Technical Field

[0001] This disclosure relates to the fields of data processing technology and image processing technology, and in particular to artificial intelligence technology fields such as computer vision, intelligent voice technology, and intelligent transportation technology. Background Technology

[0002] With the booming development of the robotics industry, more and more robots are being used in indoor environments such as warehouses, factories, hospitals, shopping malls, hotels, and offices to provide services such as goods distribution and delivery, and wayfinding, which greatly improves production efficiency and service efficiency, reduces labor costs, and provides convenience for users.

[0003] Typically, autonomous navigation of robots in indoor environments requires path planning based on a map of the indoor environment, which is manually marked at specific locations. Summary of the Invention

[0004] This disclosure provides a method, apparatus, electronic device, and readable storage medium for processing indoor environments.

[0005] According to one aspect of this disclosure, a method for handling indoor environments is provided, comprising:

[0006] The system acquires location commands provided by the user within an indoor environment, including gesture commands and voice commands.

[0007] According to the positioning command, obtain the positioning location information and the positioning description information of the positioning location indicated by the positioning command in the indoor environment;

[0008] The location information and the location description information are associated to enable automatic movement within the indoor environment.

[0009] According to another aspect of this disclosure, a travel processing device for an indoor environment is provided, comprising:

[0010] The acquisition unit is used to acquire the positioning instructions provided by the user in the indoor environment, the positioning instructions including gesture instructions and voice instructions;

[0011] The positioning unit is configured to obtain, according to the positioning instruction, the positioning location information and the positioning description information of the positioning location indicated by the positioning instruction in the indoor environment;

[0012] The association unit is used to associate the location information and the location description information to enable automatic movement within the indoor environment.

[0013] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0014] At least one processor; and

[0015] A memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the methods described above and any possible implementations.

[0017] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing the computer to perform the methods described above and any possible implementation thereof.

[0018] According to another aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the aspects and any possible implementations described above.

[0019] As can be seen from the above technical solution, the embodiments of this disclosure do not require human intervention, are simple to operate, and have a high accuracy rate, thereby improving the efficiency and reliability of indoor environment positioning.

[0020] In addition, the technical solutions provided in this disclosure can effectively improve the user experience.

[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. The drawings are used to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0023] Figure 1A This is a schematic diagram based on the first embodiment of the present disclosure;

[0024] Figure 1B yes Figure 1A A schematic diagram of the gesture commands provided by the user in the corresponding embodiment;

[0025] Figure 1C yes Figure 1AAnother schematic diagram of the gesture commands provided by the user in the corresponding embodiment;

[0026] Figure 2 This is a schematic diagram according to the second embodiment of the present disclosure;

[0027] Figure 3 This is a block diagram of an electronic device used to implement the indoor environment movement processing method of the embodiments of this disclosure. Detailed Implementation

[0028] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0029] Obviously, the described embodiments are only some, not all, of the embodiments disclosed herein. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0030] It should be noted that the terminal devices involved in the embodiments of this disclosure may include, but are not limited to, smart devices such as mobile phones, personal digital assistants (PDAs), wireless handheld devices, and tablet computers; the display devices may include, but are not limited to, personal computers, televisions, and other devices with display functions.

[0031] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0032] Existing indoor navigation systems using mobile intelligent terminals such as robots require prior scanning of the indoor environment using sensing devices such as LiDAR and visual cameras to create a mapping map. Then, a human operator views the map using desktop devices such as computers, tablets, and mobile phones, marking specific locations to refine the indoor environment mapping.

[0033] If the location points need to be updated, such as by adding or modifying them, manual adjustments are required again.

[0034] Furthermore, the mobile smart terminal can only travel to a given location point based on a survey map.

[0035] Existing indoor navigation systems based on surveying maps and mobile smart terminals suffer from poor flexibility and ease of use.

[0036] Therefore, there is an urgent need to provide a more flexible and user-friendly method for indoor navigation of mobile smart terminals based on surveying maps.

[0037] Figure 1A This is a schematic diagram based on the first embodiment of the present disclosure, as shown below. Figure 1A As shown.

[0038] 101. Obtain the location instructions provided by the user in the indoor environment, the location instructions including gesture instructions and voice instructions.

[0039] 102. Based on the positioning command, obtain the positioning location information and the positioning description information of the positioning location indicated by the positioning command in the indoor environment.

[0040] 103. The location information and the location description information are associated for automatic movement within the indoor environment.

[0041] This establishes a correlation between the location information of any point in the indoor environment and its corresponding location description information, enabling the provision of more convenient automated movement solutions within the indoor environment based on the correlation processing performed.

[0042] For example, based on the associated processing performed, update operations such as adding, modifying, and deleting location points can be performed on the initial mapping map of the indoor environment to build an updated mapping map, thereby quickly updating the mapping map.

[0043] Alternatively, for example, without having to add specific locations to the survey map in advance, delivery robots, vehicles, and other mobile intelligent terminals can be directly instructed to automatically move to the specific location in the indoor environment based on the association processing performed, thereby quickly realizing automatic movement control in the indoor environment.

[0044] It should be noted that some or all of the execution entities of 101 to 103 can be applications located on local terminals (e.g., delivery robots, vehicles, and other mobile intelligent terminals), or they can be functional units such as plug-ins or software development kits (SDKs) set in applications located on local terminals, or they can be processing engines located on network-side servers, or they can be distributed systems located on the network side, such as processing engines or distributed systems in mobile intelligent terminal processing platforms in indoor environments on the network side. This embodiment does not impose any particular limitations on these.

[0045] It is understood that the application may be a native program installed on the local terminal, or it may be a web application of a browser on the local terminal. This embodiment does not limit this.

[0046] In this way, by acquiring positioning commands, including gesture commands and voice commands, provided by the user in the indoor environment, and then obtaining the positioning location information and positioning description information of the location indicated by the positioning command in the indoor environment, the positioning location information and positioning description information can be associated for automatic movement in the indoor environment. Because a command method combining gesture commands and voice commands is used, the positioning location information indicated by the gesture command in the indoor environment can be automatically associated with the positioning description information of the location indicated by the voice command. Based on the association processing, a convenient automatic movement scheme in the indoor environment can be provided. For example, updating operations such as adding, modifying, and deleting positioning locations can be performed on the initial mapping map of the indoor environment to construct an updated mapping map, directly instructing delivery robots, vehicles, and other mobile intelligent terminals to move automatically in the indoor environment without human intervention. The operation is simple and has a high accuracy rate, thereby improving the efficiency and reliability of indoor positioning.

[0047] The technical solution provided in this disclosure utilizes gesture recognition and voice recognition interaction technologies to quickly add location points to surveying and mapping maps, thereby improving the efficiency and reliability of surveying and mapping map construction.

[0048] The technical solution provided in this disclosure utilizes gesture recognition and voice recognition interaction technologies to quickly instruct delivery robots, vehicles, and other mobile intelligent terminals to automatically travel to a specific location in an indoor environment. This enhances the flexibility and ease of use of mobile intelligent terminals that navigate based on survey maps, making the operation more natural and user-friendly.

[0049] In this disclosure, a common delivery robot can be used as a mobile intelligent terminal to collect the positioning commands provided by the user in the indoor environment for execution of steps 101 to 103.

[0050] Optionally, in one possible implementation of this embodiment, before step 101, the position and orientation of the mobile smart terminal can be further adjusted according to the user's body position and facial orientation. For example, the image acquisition device of the mobile smart terminal can be adjusted to face the user directly to ensure that the mobile smart terminal can accurately acquire the positioning instructions provided by the user.

[0051] Therefore, after adjusting the position and orientation of the mobile smart terminal, the mobile smart terminal can collect the gesture commands and voice commands provided by the user in the indoor environment for execution 101.

[0052] Optionally, in one possible implementation of this embodiment, in step 102, the location information of the location indicated by the gesture command in the indoor environment can be obtained based on the gesture command, and the location description information of the location can be obtained based on the voice command.

[0053] The so-called location information refers to the location description information used to identify the relative position of the location in the indoor environment, such as 50m east of the entrance.

[0054] The so-called location description information refers to the descriptive information used to describe the basic attributes of the location. For example, the name attribute of the location indicated by the voice command "This is the front desk", the functional attribute of the location indicated by the voice command "This is the restroom", etc., or it can also be used to describe the travel attributes of the location, such as the travel destination attribute indicated by the voice command "Wait for me here".

[0055] In a specific implementation, the gesture command can be processed by gesture recognition to determine the direction of the gesture pointed by the user's finger. Based on the determined direction of the gesture pointed by the user's finger, the location information of the location indicated by the gesture command in the indoor environment can be obtained.

[0056] Specifically, based on the gesture command, the direction information of the user's finger pointing to the gesture command can be obtained. Then, based on the direction information of the user's finger and the height information of the user's finger from the ground, the relative distance between the location and the user can be obtained. Then, based on the user's location information in the indoor environment and the relative distance between the location and the user, the location information of the location can be obtained.

[0057] The direction information pointed to by the user's finger can refer to the angle between the direction of the gesture pointed to by the user's finger and a specific reference direction (e.g., horizontal or vertical direction).

[0058] The height information of the user's finger from the ground can refer to the distance between the tip of the user's finger and the ground.

[0059] For example, such as Figure 1B As shown, specifically, the distance h from the user's fingertip to the ground and the angle α between the direction of the user's gesture and a specific reference direction (e.g., horizontal or vertical) can be obtained. Then, based on the distance h from the user's fingertip to the ground and the angle α between the direction of the user's gesture and the specific reference direction (e.g., horizontal or vertical), the relative distance D between the location and the user can be obtained, i.e., D = h tanα. Furthermore, the relative position between the location and the user can be determined based on this relative distance. Then, based on the user's location information in the indoor environment and the relative position between the location and the user, the location information of the location in the indoor environment can be obtained. Here, the location information of the location in the indoor environment can be the relative position information between the location and a reference position in the indoor environment.

[0060] In this implementation process, the distance h between the fingertip of the user's finger and the ground can be obtained in a variety of ways.

[0061] For example, such as Figure 1C As shown, specifically, the mobile smart terminal can use laser ranging technology to collect the relative distance d1 between the mobile smart terminal and the fingertip of the user's finger, and the relative distance d2 between the location of the mobile smart terminal and the location of the user. After obtaining the relative distance d1 and d2 between the mobile smart terminal and the user's fingertip, the height h of the user's fingertip from the ground can be obtained based on the pre-configured reference height H of the mobile smart terminal and the obtained relative distances d1 and d2 between the mobile smart terminal and the user's location.

[0062] In this implementation process, the angle α between the direction of the user's finger gesture and a specific reference direction (e.g., horizontal or vertical direction) can be obtained in a variety of ways.

[0063] For example, a mobile smart terminal can collect image information of a user's finger, and then, based on the image information of the user's finger, determine the angle α between the gesture direction pointed by the user's finger and a specific reference direction (e.g., horizontal or vertical direction).

[0064] In another specific implementation, the voice command can be subjected to speech recognition processing to obtain the speech recognition result. After obtaining the speech recognition result, attribute keywords in the speech recognition result can be further identified using keyword recognition technologies such as named entity recognition technology, and used as location description information for the location.

[0065] By utilizing gesture recognition and voice recognition technologies, the location information and description of a newly added location can be quickly determined, facilitating convenient and rapid association between the two. This allows for the rapid addition of locations to the mapping map, improving the efficiency and reliability of map construction. Simultaneously, it can quickly instruct delivery robots, vehicles, and other mobile intelligent terminals to automatically navigate to specific locations within indoor environments, enhancing the flexibility and ease of use of these map-based navigation devices and making operation more natural and user-friendly.

[0066] Optionally, in one possible implementation of this embodiment, in step 103, the location information and the location description information can be statically associated on the initial mapping map of the indoor environment to construct an updated mapping map of the indoor environment.

[0067] In this way, users can use a combination of gesture and voice commands to indicate the location of a new location on the initial mapping map of the indoor environment. Then, based on the location information of the location, the location description information of the location can be correlated in the corresponding area on the initial mapping map, thereby improving the efficiency and reliability of mapping map updates.

[0068] In a specific implementation process, prior to step 103, an initial mapping map of the indoor environment can be further obtained.

[0069] For example, an initial mapping map of the indoor environment can be constructed. Specifically, in the indoor environment, sensing devices can be used to detect the surrounding environment in order to construct the initial mapping map of the indoor environment.

[0070] Alternatively, for example, an initial mapping map of the indoor environment constructed by another specific terminal device can be obtained. Specifically, this specific terminal device can use sensing devices to detect the surrounding environment within the indoor space to construct the initial mapping map. Furthermore, simple, specific location markers such as doors and entrances can be manually marked on the initial mapping map constructed by the specific terminal device to refine the initial mapping map of the indoor environment.

[0071] In this way, by acquiring an initial mapping map based on the surrounding environment, and then using it as a reference mapping map to associate newly added location points, the efficiency and reliability of mapping map construction can be effectively improved.

[0072] In another specific implementation process, in step 103, the location can be marked on the initial mapping map based on the location information to obtain the map element corresponding to the location on the initial mapping map. Then, the location description information of the location can be associated with the map element to construct an updated mapping map of the indoor environment.

[0073] In this way, the newly added location is marked with corresponding map elements on the initial mapping map of the indoor environment. Then, the location description information of the location can be associated with the marked map elements to construct an updated mapping map of the indoor environment, which can serve as a navigation basis for delivery robots, vehicles and other mobile intelligent terminals.

[0074] Optionally, in one possible implementation of this embodiment, in step 103, the location information and the location description information can be dynamically associated to generate the current movement instruction within the indoor environment.

[0075] In this way, users can use a combination of gesture and voice commands to indicate the destination location. Then, the location information of the location can be used for movement association processing to generate the movement command, thereby improving the efficiency and reliability of the intelligent terminal's automatic movement.

[0076] In this embodiment, by acquiring positioning commands provided by the user in the indoor environment, including gesture commands and voice commands, and then obtaining the positioning location information and positioning description information of the location indicated by the positioning command in the indoor environment, the positioning location information and positioning description information can be associated for automatic movement in the indoor environment. Because a command method combining gesture commands and voice commands is used, the positioning location information indicated by the gesture command in the indoor environment can be automatically associated with the positioning description information of the location indicated by the voice command. Based on the associated processing, a convenient automatic movement scheme in the indoor environment can be provided. For example, updating operations such as adding, modifying, and deleting positioning locations can be performed on the initial mapping map of the indoor environment to construct an updated mapping map, directly instructing delivery robots, vehicles, and other mobile intelligent terminals to move automatically in the indoor environment without human intervention. The operation is simple and has a high accuracy rate, thereby improving the efficiency and reliability of indoor positioning.

[0077] In addition, the technical solutions provided in this disclosure can effectively improve the user experience.

[0078] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this disclosure is not limited to the described order of actions, because according to this disclosure, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this disclosure.

[0079] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0080] Figure 2 This is a schematic diagram based on the second embodiment of the present disclosure, as shown below. Figure 2 As shown. The indoor environment movement processing device 200 of this embodiment may include an acquisition unit 201, a positioning unit 202, and an association unit 203. The acquisition unit 201 is used to acquire positioning commands provided by a user within the indoor environment, including gesture commands and voice commands; the positioning unit 202 is used to obtain, based on the positioning commands, the positioning location information and the positioning description information of the location indicated by the positioning commands in the indoor environment; the association unit 203 is used to associate the positioning location information and the positioning description information for automatic movement within the indoor environment.

[0081] It should be noted that some or all of the indoor environment mobility processing device in this embodiment may be an application located on a local terminal (i.e., a delivery robot, vehicle, or other mobility intelligent terminal), or it may be a plugin or software development kit (SDK) or other functional unit set in an application located on a local terminal, or it may be a processing engine located on a network-side server, or it may be a distributed system located on the network side, such as a processing engine or distributed system in a network-side indoor environment mobility intelligent terminal processing platform, etc. This embodiment does not impose any particular limitations on this.

[0082] It is understood that the application may be a native program installed on the local terminal, or it may be a web application of a browser on the local terminal. This embodiment does not limit this.

[0083] Optionally, in one possible implementation of this embodiment, the positioning unit 202 may be specifically used to obtain the positioning location information of the positioning location indicated by the gesture command in the indoor environment according to the gesture command; and to obtain the positioning description information of the positioning location according to the voice command.

[0084] In a specific implementation, the positioning unit 202 can be used to obtain the direction information of the user's finger corresponding to the gesture instruction according to the gesture instruction; obtain the relative distance between the positioning location and the user according to the direction information of the user's finger and the height information of the user's finger from the ground; and obtain the positioning location information of the positioning location according to the positioning location information of the user in the indoor environment and the relative distance between the positioning location and the user.

[0085] Optionally, in one possible implementation of this embodiment, the association unit 203 may be used to perform static association processing on the initial mapping map of the indoor environment with the location information and the location description information to construct an updated mapping map of the indoor environment.

[0086] In a specific implementation process, the association unit 203 can also be further used to obtain an initial mapping map of the indoor environment.

[0087] For example, the association unit 203 can also be further used to detect the surrounding environment using sensing devices in the indoor environment in order to construct an initial mapping map of the indoor environment.

[0088] Alternatively, for example, the association unit 203 can also be further used to obtain an initial mapping map of the indoor environment constructed by a specific terminal device.

[0089] In another specific implementation, the association unit 203 can be used to mark the location on the initial mapping map according to the location information to obtain the map element corresponding to the location on the initial mapping map; and to associate the location description information of the location with the map element to construct an updated mapping map of the indoor environment.

[0090] Optionally, in one possible implementation of this embodiment, the association unit 203 may be used to dynamically associate the location information and the location description information to generate the current travel instruction within the indoor environment.

[0091] It should be noted that, Figure 1A The method in the corresponding embodiment can be implemented by the indoor environment movement processing device provided in this embodiment. For a detailed description, please refer to... Figure 1A The relevant content in the corresponding embodiments will not be repeated here.

[0092] In this embodiment, the acquisition unit acquires positioning commands, including gesture commands and voice commands, provided by the user in the indoor environment. Then, the positioning unit obtains the location information and description information of the location indicated by the positioning command in the indoor environment based on the positioning command. This allows the association unit to associate the location information and the description information for automatic movement within the indoor environment. Because a combination of gesture and voice commands is used, the location information indicated by the gesture command in the indoor environment can be automatically associated with the description information of the location indicated by the voice command. Based on this association, a convenient automatic movement scheme within the indoor environment can be provided. For example, updating the initial mapping map of the indoor environment by adding, modifying, or deleting locations can directly instruct delivery robots, vehicles, and other mobile intelligent terminals to move automatically within the indoor environment. This requires no human intervention, is simple to operate, and has a high accuracy rate, thereby improving the efficiency and reliability of indoor positioning.

[0093] In addition, the technical solutions provided in this disclosure can effectively improve the user experience.

[0094] The acquisition, storage, and application of user information involved in the technical solution disclosed herein, such as user location information, user gesture commands, and user voice commands, all comply with relevant laws and regulations and do not violate public order and good morals.

[0095] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0096] Figure 3 A schematic block diagram of an example electronic device 300 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0097] like Figure 3 As shown, the electronic device 300 includes a computing unit 301, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 302 or a computer program loaded from a storage unit 308 into a random access memory (RAM) 303. The RAM 303 may also store various programs and data required for the operation of the electronic device 300. The computing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0098] Multiple components in electronic device 300 are connected to I / O interface 305, including: input unit 306, such as keyboard, mouse, etc.; output unit 307, such as various types of displays, speakers, etc.; storage unit 308, such as disk, optical disk, etc.; and communication unit 309, such as network card, modem, wireless transceiver, etc. Communication unit 309 allows electronic device 300 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0099] The computing unit 301 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 301 performs the various methods and processes described above, such as the method for processing movement in an indoor environment. For example, in some embodiments, the method for processing movement in an indoor environment can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 308. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 300 via ROM 302 and / or communication unit 309. When the computer program is loaded into RAM 303 and executed by the computing unit 301, one or more steps of the method for processing movement in an indoor environment described above can be performed. Alternatively, in other embodiments, computing unit 301 may be configured to perform a method for processing movement in an indoor environment by any other suitable means (e.g., by means of firmware).

[0100] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0101] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0102] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0103] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0104] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), the Internet, and blockchain networks.

[0105] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service ecosystem, addressing the shortcomings of traditional physical hosts and VPS (Virtual Private Server, or simply "VPS") services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0106] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0107] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for handling movement in an indoor environment, comprising: The system acquires location commands provided by the user within an indoor environment, including gesture commands and voice commands. According to the positioning instruction, the positioning location information of the positioning location indicated by the positioning instruction in the indoor environment and the positioning description information of the positioning location are obtained. The positioning description information is descriptive information used to describe the name attribute or functional attribute of the positioning location. The location information and the location description information are associated to enable automatic movement within the indoor environment; The step of obtaining location description information of the location indicated by the location command in the indoor environment according to the location command includes: The voice command is subjected to speech recognition processing to obtain the speech recognition result of the voice command; The attribute keywords in the speech recognition results are identified and used as the location description information of the location; The step of obtaining the location information of the location indicated by the positioning command in the indoor environment according to the positioning command includes: Based on the gesture command, obtain the direction information of the user's finger pointing to the gesture command; Based on the direction information pointed by the user's finger and the height information of the user's finger from the ground, the relative distance between the location and the user is obtained. The height information of the user's finger from the ground is the distance between the fingertip of the user's finger and the ground. Obtaining the distance between the fingertip of the user's finger and the ground includes: collecting the relative distance d1 between the mobile smart terminal and the fingertip of the user's finger, and the relative distance d2 between the location of the mobile smart terminal and the location of the user; and obtaining the distance between the fingertip of the user's finger and the ground based on the reference height of the mobile smart terminal, the relative distance d1, and the relative distance d2. Based on the user's location information in the indoor environment and the relative distance between the location and the user, the location information of the location is obtained. The method further includes: Before acquiring the location command provided by the user in the indoor environment, the position and orientation of the mobile smart terminal are adjusted according to the user's body position and facial orientation, so that the image acquisition device of the mobile smart terminal is facing the user.

2. The method according to claim 1, wherein, The step of associating the location information and the location description information for automatic movement within the indoor environment includes: The location information and the location description information are statically correlated on the initial mapping map of the indoor environment to construct an updated mapping map of the indoor environment; or The location information and the location description information are dynamically correlated to generate the current movement command within the indoor environment.

3. The method according to claim 2, wherein, Before performing static correlation processing on the initial mapping map of the indoor environment to construct an updated mapping map of the indoor environment, the method further includes: In the indoor environment, sensing devices are used to detect the surrounding environment in order to construct an initial mapping map of the indoor environment; or Obtain an initial mapping map of the indoor environment constructed by a specific terminal device.

4. The method according to claim 2 or 3, wherein, The step of statically associating the location information and the location description information on the initial mapping map of the indoor environment to construct an updated mapping map of the indoor environment includes: Based on the location information, the location is marked on the initial mapping map to obtain the map element corresponding to the location on the initial mapping map; The location description information of the location is associated with the map elements to construct an updated mapping map of the indoor environment.

5. A movement processing device for an indoor environment, comprising: The acquisition unit is used to acquire the positioning instructions provided by the user in the indoor environment, the positioning instructions including gesture instructions and voice instructions; The positioning unit is configured to obtain, according to the positioning instruction, the positioning location information of the positioning location indicated by the positioning instruction in the indoor environment and the positioning description information of the positioning location, wherein the positioning description information is descriptive information used to describe the name attribute or functional attribute of the positioning location; The association unit is used to associate the location information and the location description information to enable automatic movement in the indoor environment. Specifically, when the positioning unit obtains the positioning description information of the positioning location indicated by the positioning command in the indoor environment according to the positioning command, it performs the following: The voice command is subjected to speech recognition processing to obtain the speech recognition result of the voice command; The attribute keywords in the speech recognition results are identified and used as the location description information of the location; When the positioning unit obtains the positioning location information of the positioning location indicated by the positioning command in the indoor environment according to the positioning command, it specifically performs the following: Based on the gesture command, obtain the direction information of the user's finger pointing to the gesture command; Based on the direction information pointed by the user's finger and the height information of the user's finger from the ground, the relative distance between the location and the user is obtained. The height information of the user's finger from the ground is the distance between the fingertip of the user's finger and the ground. Obtaining the distance between the fingertip of the user's finger and the ground includes: collecting the relative distance d1 between the mobile smart terminal and the fingertip of the user's finger, and the relative distance d2 between the location of the mobile smart terminal and the location of the user; and obtaining the distance between the fingertip of the user's finger and the ground based on the reference height of the mobile smart terminal, the relative distance d1, and the relative distance d2. Based on the user's location information in the indoor environment and the relative distance between the location and the user, the location information of the location is obtained. The acquisition unit is also used to perform: Before acquiring the location command provided by the user in the indoor environment, the position and orientation of the mobile smart terminal are adjusted according to the user's body position and facial orientation, so that the image acquisition device of the mobile smart terminal is facing the user.

6. The apparatus according to claim 5, wherein, The associated unit is specifically used for The location information and the location description information are statically correlated on the initial mapping map of the indoor environment to construct an updated mapping map of the indoor environment. or The location information and the location description information are dynamically correlated to generate the current movement command within the indoor environment.

7. The apparatus according to claim 6, wherein, The associated unit is also used for In the indoor environment, sensing devices are used to detect the surrounding environment in order to construct an initial mapping map of the indoor environment; or Obtain an initial mapping map of the indoor environment constructed by a specific terminal device.

8. The apparatus according to claim 6 or 7, wherein, The associated unit is specifically used for Based on the location information, the location is marked on the initial mapping map to obtain the map element corresponding to the location on the initial mapping map; as well as The location description information of the location is associated with the map elements to construct an updated mapping map of the indoor environment.

9. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1-4.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-4.

11. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-4.

Images