Object searching method of intelligent glasses, electronic equipment and storage medium

Abstract

The invention relates to the field of intelligent glasses, in particular to an object searching method of intelligent glasses, electronic equipment and a storage medium. The method comprises the following steps: in response to an instruction of searching a target article by a wearer, obtaining pre-stored position information of the target article; and according to the position information, determining a first space where the target object is located and the position of the target object in the first space, and indicating the wearer.

Description

Technical Field

[0001] This disclosure relates to the field of smart glasses, and more specifically, to a method for finding objects in smart glasses, an electronic device, and a storage medium. Background Technology

[0002] In our busy daily lives, almost everyone occasionally encounters the frustration of not being able to find things. This experience can be small but annoying, like urgently needing a pen or remote control—you clearly remember putting it somewhere, but when you actually need it, you can't find it anywhere. Attention deficit is also a major reason for not being able to find things. In modern society, people are faced with all sorts of information and stimuli, making it easy to be distracted. When we need to find something, if our mind isn't on the search, or if we're doing other tasks at the same time, we might overlook details that should be noticed. Sometimes, the item is right in front of us, but we "blind" it because of a lack of focus. Summary of the Invention

[0003] One objective of this disclosure is to provide a method for finding objects with smart glasses, solving the problem of users needing to find an item but being unable to find it in a timely manner.

[0004] According to a first aspect of this disclosure, a method for finding objects with smart glasses is provided, comprising:

[0005] In response to the wearer's instruction to locate the target item, the location information of the target item is retrieved from pre-stored data.

[0006] Based on the location information, the first space where the target item is located and the position of the target item in the first space are determined, and instructions are given to the wearer.

[0007] Optionally, before responding to the wearer's instruction to locate the target item, the method further includes:

[0008] The location information of the target item is stored, wherein the location information includes the identity identifier of the first space where the target item is located, and the location parameters of the target item in the first space.

[0009] Optionally, storing the location information of the target item includes:

[0010] The location where the wearer enters the first space is taken as the reference location, and the identity identifier of the first space is set at the reference location;

[0011] In response to the wearer's storage command, the location information of the target item is stored.

[0012] Optionally, the target item is determined based on eye tracking.

[0013] Optionally, the first location is the position of the smart glasses in the space when the location information of the target item is stored, and the position parameters of the target item in the first space include:

[0014] The distance the first position moves relative to the reference position on the horizontal plane;

[0015] The angles between the first position and the reference position on the horizontal plane;

[0016] The distance between the target item and the first position on the horizontal plane;

[0017] The angle between the target item and the first position on the horizontal plane;

[0018] The distance of the reference position relative to the ground;

[0019] The angle between the target item and the first position in the vertical direction and the distance of the first position relative to the ground; or, the distance of the target item relative to the ground.

[0020] Optionally, the smart glasses are equipped with a laser printing head, and the step of setting the identity identifier of the first space at the reference position includes:

[0021] Print the identification mark of the first space at the reference location.

[0022] Optionally, determining the first space where the target item is located and the position of the target item within the first space based on the location information includes:

[0023] The first space where the target item is located is determined based on the identity identifier;

[0024] Based on the position parameters of the target item in the first space, the relative positional relationship between the target item and the reference position in the horizontal and vertical directions is determined respectively.

[0025] Optionally, determining the relative positional relationship between the target item and the reference position on the horizontal plane includes:

[0026] The relative positional relationship between the first position and the reference position on the horizontal plane is determined based on the distance the first position moves relative to the reference position on the horizontal plane and the angles on the horizontal plane when the first position, the reference position, and the reference position are in the first position.

[0027] The relative positional relationship between the target item and the first position on the horizontal plane is determined based on the distance between the target item and the first position on the horizontal plane and the angle between the target item and the first position on the horizontal plane.

[0028] Based on the relative positional relationship between the first position and the reference position on the horizontal plane and the relative positional relationship between the target item and the first position on the horizontal plane, the relative positional relationship between the target item and the reference position is determined.

[0029] Optionally, determining the relative positional relationship between the target item and the reference position in the vertical direction includes:

[0030] The distance of the target item relative to the ground is determined based on the angle between the target item and the first position in the vertical direction and the distance of the first position relative to the ground.

[0031] The relative positional relationship between the target item and the reference position in the vertical direction is determined based on the distance of the target item relative to the ground and the distance of the reference position relative to the ground.

[0032] According to a second aspect of this disclosure, an electronic device is provided, including a processor and a memory, wherein the memory stores computer instructions that, when executed by the processor, implement the steps of the method described in any of the first aspects.

[0033] According to a third aspect of this disclosure, a storage medium is provided that stores computer instructions thereon, which, when executed by a processor, implement the steps of the method described in any of the first aspects.

[0034] One technical advantage of this disclosure is that it provides a method for finding items using smart glasses. This method responds to the wearer's command to find a target item, retrieves pre-stored location information of the target item, and determines the space containing the target item and its position within that space based on this location information. The method then provides guidance to the wearer. In this way, users can use smart glasses to find items, determine which space and location the item belongs to, solve the problem of difficulty in finding items, and improve the efficiency of item finding.

[0035] Other features and advantages of the embodiments of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0036] The accompanying drawings, which form part of this specification, illustrate embodiments of the present disclosure and, together with the specification, serve to explain the principles of the embodiments of the present disclosure.

[0037] Figure 1This is a flowchart of a smart glasses object-finding method according to one embodiment;

[0038] Figure 2 This is a schematic diagram of the structure of an electronic device according to one embodiment; Detailed Implementation

[0039] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the invention.

[0040] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use.

[0041] Technologies and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such technologies and equipment should be considered part of the specification.

[0042] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0043] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0044] This application discloses a method for finding objects using smart glasses, which can be applied to smart glasses, such as VR, AR, MR, and other types of smart glasses. Figure 1 As shown, it includes steps S11 to S12.

[0045] Step S11: In response to the wearer's instruction to find the target item, obtain the pre-stored location information of the target item.

[0046] In this embodiment, the smart glasses can respond to user commands to identify a target item and obtain its location information. In one example, the command can be a voice command; the smart glasses use voice recognition to determine which item the user needs to find. In another example, the command can be other types of commands. For instance, the user can trigger the item-finding function via a real or virtual button. Afterward, the smart glasses can display images of various stored items, allowing the user to select the item they wish to find.

[0047] In one example of this embodiment, before responding to the wearer's instruction to locate the target item, the method further includes: storing the location information of the target item, wherein the location information includes an identifier of a first space where the target item is located, and location parameters of the target item in the first space.

[0048] In this embodiment, the user can pre-store the location information of the target item. This location information may include an identifier for the target item's location within a first space. In one example, the smart glasses acquire the surrounding environment through an external camera, pre-dividing the wearer's external environment into multiple spaces and assigning an identifier to each space. In one example, the space can be divided according to rooms, with each room considered a space and assigned an identifier. In another example, environments such as larger rooms like warehouses can also be divided according to space size, for example, using every 20 square meters of floor space as a space and assigning an identifier.

[0049] In one example of this embodiment, storing the location information of the target item includes: taking the location where the wearer enters the first space as a reference location and setting an identification mark of the first space at the reference location; and storing the location information of the target item in response to the wearer's storage instruction.

[0050] In this embodiment, when the wearer enters the first space, the smart glasses can record the wearer's position when entering the first space and use this position as a reference position. An identification mark for the space can be set at or near this position. In this embodiment, the mark can be the number of the space, and the identification mark can be set by displaying the virtual number of the space at the reference position.

[0051] In another example, the smart glasses are equipped with a laser printhead to set an identity identifier for the first space at a reference position, including: printing the identity identifier for the first space at the reference position.

[0052] In this example, the smart glasses could also have a laser printer on the outside. When a user enters the first space, the smart glasses can print an identification mark for that space at a reference location. For example, if the first space is a room, the user can print the space's identifier at the entrance, such as on the door. If the first space is divided into areas based on size, corresponding identifiers can also be printed on the ground. In this way, users can be guided by space markings when searching for items in the future, making it easier for them to find their belongings.

[0053] After establishing the spatial identifier at the reference location, the location information of the corresponding target item can be stored based on the user's instruction to store the target item's location information. Similar to the previous embodiment, this instruction can also be a voice command or other types of instructions; no limitation is made here.

[0054] In one example of this embodiment, the target object is determined based on eye tracking.

[0055] In one example, the smart glasses may have one or more cameras on the inside to track the wearer's eyes and identify the direction the user is looking, while using the outer cameras to identify the target object whose location information the user wants to store.

[0056] In this example, by using the above method, users can identify the target item simply by looking at it when storing, improving the user experience and storage efficiency.

[0057] In one example of this embodiment, the first position is the position of the smart glasses in space when storing the position information of the target item. The position parameters of the target item in the first space include: the moving distance of the first position relative to the reference position on the horizontal plane; the angle between the first position and the reference position on the horizontal plane; the distance between the target item and the first position on the horizontal plane; the angle between the target item and the first position on the horizontal plane; the distance between the reference position and the ground; the angle between the target item and the first position in the vertical direction and the distance between the first position and the ground; or the distance between the target item and the ground.

[0058] In this implementation, the first position refers to the location of the smart glasses in space when the user performs storage. The distance the first position moves relative to the reference position on the horizontal plane, and the angle between the first position and the reference position on the horizontal plane, can be obtained using inertial sensors or gyroscopes installed in the smart glasses. The horizontal distance between the target object and the first position can be obtained using ultrasonic sensors, laser rangefinders, or images captured by a camera combined with image recognition algorithms, all installed in the smart glasses. The angle between the target object and the first position on the horizontal plane, the distance of the reference position relative to the ground, the vertical angle between the target object and the first position, the distance of the first position relative to the ground, and even the distance of the target object relative to the ground can all be obtained using images captured by the camera combined with image recognition algorithms.

[0059] In this embodiment, in some cases, such as when an item is placed in a table or cabinet and its height is obtained directly through an image recognition algorithm, the distance of the target item relative to the ground can be determined based on the angle between the target item and the first position in the vertical direction and the distance of the first position relative to the ground. Therefore, the position information may include at least one of these two types of position information or the distance of the target item relative to the ground.

[0060] In one example of this embodiment, the location information of the target item can be stored in the form of an encoding. For example, the encoding starts with the identification code of the first space, followed by the encoding of various location parameters.

[0061] Step S12: Based on the location information, determine the first space where the target item is located and the position of the target item in the first space, and give instructions to the wearer.

[0062] In this embodiment, after obtaining the location information of the target item, the location of the target item can be determined and indicated to the wearer, for example, by indicating the direction of the target item with a virtual arrow, or by highlighting the target item, etc.

[0063] This example provides a method for finding items using smart glasses. In response to the wearer's command to find a target item, the method retrieves pre-stored location information of the target item and, based on this information, determines the space containing the target item and its location within that space, then provides instructions to the wearer. In this way, users can use smart glasses to find items, determine which space and location the item belongs to, solving the problem of difficulty in finding items and improving efficiency.

[0064] In this embodiment, determining the first space where the target item is located and the position of the target item in the first space based on the location information includes: determining the first space where the target item is located based on the identification; and determining the relative positional relationship between the target item and the reference position in the horizontal and vertical directions based on the position parameters of the target item in the first space.

[0065] In this example, the smart glasses can first determine the space in which the target item is located based on its identification information. Then, based on the target item's position parameters within the first space, they can determine its relative position within that space. In this example, a reference position can be used as an anchor point to determine the target item's relative position within the first space.

[0066] In one example of this embodiment, determining the relative positional relationship between the target item and the reference position on the horizontal plane includes: determining the relative positional relationship between the first position and the reference position on the horizontal plane based on the distance the first position moves relative to the reference position on the horizontal plane and the angle between the first position and the reference position on the horizontal plane; determining the relative positional relationship between the target item and the first position on the horizontal plane based on the distance between the target item and the first position on the horizontal plane and the angle between the target item and the first position on the horizontal plane; and determining the relative positional relationship between the target item and the reference position based on the relative positional relationship between the first position and the reference position on the horizontal plane and the relative positional relationship between the target item and the first position on the horizontal plane.

[0067] The position of the target item in the first space can be determined based on the reference position. However, in this embodiment, the smart glasses cannot directly obtain the relative positional relationship of the target item with respect to the reference position. Therefore, the first position where the user stores the target item's position information can be used as a transition. The positional relationship between the target item and the first position and the positional relationship between the first position and the reference position can be used to determine the position of the target item in the first space.

[0068] In this example, on a horizontal plane, the positional relationship between the first position and the reference position can be determined by the distance traveled on the horizontal plane relative to the reference position, detected by an inertial sensor or gyroscope sensor, and the angle between the smart glasses at the first position and the reference position on the horizontal plane. Then, based on the acquired distance and angle between the target object and the first position on the horizontal plane, the positional relationship between the target object and the first position on the horizontal plane is determined. These two positional relationships are used to determine the positional relationship between the target object and the reference position.

[0069] In another example of this embodiment, determining the relative positional relationship between the target item and the reference position in the vertical direction includes: determining the distance of the target item relative to the ground based on the angle between the target item and the first position in the vertical direction and the distance of the first position relative to the ground; and determining the relative positional relationship between the target item and the reference position in the vertical direction based on the distance of the target item relative to the ground and the distance of the reference position relative to the ground.

[0070] In this embodiment, to prevent the smart glasses from failing to acquire an image of the ground during guidance, thus making it impossible to directly determine the height of the target object, the relative position of the target object in the vertical direction can be determined using a reference position. The distance of the target object relative to the ground can be calculated based on the angle between the target object and the first position in the vertical direction and the distance of the first position relative to the ground. Then, the relative height difference between the two, i.e., the relative position in the vertical direction, can be determined using this distance and the distance of the reference position relative to the ground.

[0071] like Figure 2 As shown, this application embodiment also provides an electronic device 200, including a processor 201 and a memory 202. The memory 202 stores computer instructions, which, when executed by the processor 201, implement the steps of any method in the embodiment of the smart glasses object-finding method.

[0072] This application also provides a storage medium storing computer instructions. When the computer instructions are executed by a processor, they implement any one of the above-described smart glasses object-finding embodiments and achieve the same technical effect. To avoid repetition, further details are omitted here.

[0073] The various embodiments in this disclosure are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device and apparatus embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0074] The foregoing has described specific embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0075] Embodiments of this disclosure may be systems, methods, and / or computer program products. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the embodiments of this disclosure.

[0076] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0077] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0078] Computer program instructions used to perform the operations of embodiments of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on a user's computer, partially on a user's computer, as a standalone software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of embodiments of this disclosure.

[0079] Various aspects of embodiments of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0080] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0081] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0082] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be known to those skilled in the art that implementation in hardware, implementation in software, and implementation in a combination of software and hardware are equivalent.

[0083] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for searching for an item using smart glasses, the method comprising: The application is applied to intelligent glasses, comprising: In response to the instruction of the wearer looking for a target object, the position information of the target object is obtained in advance; According to the position information, the first space where the target object is located and the position of the target object in the first space are determined, and the wearer is instructed.

2. The method of claim 1, wherein, Before responding to the instruction of the wearer looking for a target object, the method further comprises: Storing the position information of the target object, wherein the position information comprises the identity of the first space where the target object is located, and the position parameter of the target object in the first space.

3. The method of claim 2, wherein, The storage of the position information of the target object comprises: Taking the position of the wearer entering the first space as a reference position, and setting the identity of the first space at the reference position; In response to the storage instruction of the wearer, the position information of the target object is stored.

4. The method of claim 3, wherein, The first position is the position of the intelligent glasses in the space when the position information of the target object is stored, and the position parameter of the target object in the first space comprises: The moving distance of the first position relative to the reference position on the horizontal plane; The angle between the first position and the reference position on the horizontal plane; The distance between the target object and the first position on the horizontal plane; The angle between the target object and the first position on the horizontal plane; The distance of the reference position relative to the ground; The angle between the target object and the first position in the vertical direction and the distance of the first position relative to the ground; or the distance of the target object relative to the ground.

5. The method of claim 3, wherein, The intelligent glasses are provided with a laser printing head, and the identity of the first space is set at the reference position, comprising: Printing the identity of the first space at the reference position.

6. The method of claim 4, wherein, According to the position information, the first space where the target object is located and the position of the target object in the first space are determined, comprising: According to the identity, the first space where the target object is located is determined; According to the position parameter of the target object in the first space, the relative position relationship between the target object and the reference position in the horizontal and vertical directions is respectively determined.

7. The method of claim 6, wherein, Determining the relative position relationship between the target object and the reference position on the horizontal plane comprises: According to the moving distance of the first position relative to the reference position on the horizontal plane and the first position, the angle between the first position and the reference position on the horizontal plane, the relative position relationship between the first position and the reference position on the horizontal plane is determined; According to the distance between the target object and the first position on the horizontal plane and the angle between the target object and the first position on the horizontal plane, the relative position relationship between the target object and the first position on the horizontal plane is determined; Based on the relative position relationship between the first position and the reference position on the horizontal plane and the relative position relationship between the target object and the first position on the horizontal plane, the relative position relationship between the target object and the reference position is determined.

8. The method of claim 6, wherein, Determining the relative position relationship between the target object and the reference position in the vertical direction comprises: determining the distance of the target object from the ground according to the angle of the target object and the first position in the vertical direction and the distance of the first position from the ground; determining the relative position relationship of the target object and the reference position in the vertical direction according to the distance of the target object from the ground and the distance of the reference position from the ground.

9. An electronic device, comprising: A computer program product, comprising a computer readable storage medium having computer instructions stored thereon, the computer instructions being executable by a processor to perform the steps of any one of claims 1-8.

10. A storage medium, characterized by A computer program product, comprising a computer readable storage medium having computer instructions stored thereon, the computer instructions being executable by a processor to perform the steps of any one of claims 1-8.

Images