Article searching method and device, equipment, storage medium and program product

Through wearable devices, the real-time location and carrier location of the item are obtained, and the navigation signals are generated, which solves the problem of middle-aged and elderly people forgetting the location of the item, and realizes rapid search and guidance of items, avoiding the shortcomings of memos.

CN118967792APending Publication Date: 2024-11-15SHANGHAI LONGCHEER TECH CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411116244.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-14
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

Due to memory loss, middle-aged and elderly people often forget the storage location of important items. The existing memos are not updated in time and the information is not comprehensive, so the location of items cannot be quickly found.

Method used

Provide an item search method, continuously collecting environmental image information through wearable devices, obtaining real-time location information and carrier location information of target items, and generating navigation signals when receiving object search instructions, guiding users to find items, avoiding traditional memos insufficient.

Benefits of technology

It realizes rapid determination and guidance of item locations, avoids the problems of untimely update of memos and incomplete information, ensures rapid discovery of items, and is suitable for real-time analysis and updates that do not affect daily activities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118967792A_ABST
    Figure CN118967792A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides an article searching method and device, equipment, a storage medium and a program product, and relates to the technical field of artificial intelligence. The method comprises the following steps: in response to a marking instruction, continuously collecting environment image information; acquiring real-time position information corresponding to the at least one target article based on the environment image information; the real-time position information comprises object position information of the target object and carrier position information of the target object; correspondingly storing the target article, the corresponding article position information and the carrier position information; and when an object searching instruction for the target object is received, generating a navigation signal based on the object position information and the carrier position information. The method is used for quickly determining and guiding the placement position of the article.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to an object search method, device, equipment, storage medium and program product. Background Art

[0002] As we all know, the trend of population aging is highly correlated with the level of economic development. With the development of the domestic economy, the proportion of middle-aged and elderly people will continue to increase in the future. However, due to their declining memory, middle-aged and elderly people often forget the storage location of some important items.

[0003] Usually, middle-aged and elderly people use memos to record the location information of important items. However, memos may not be updated in a timely manner or the information may not be complete. In addition, the memo (electronic or paper) itself also needs a place to be stored, and it may not be possible to find the memo quickly when needed.

[0004] How to quickly determine the location of an object is an urgent problem to be solved. Summary of the invention

[0005] The embodiments of the present application provide an object search method, device, equipment, storage medium and program product to achieve a fast and automated object search effect.

[0006] In a first aspect, an embodiment of the present application provides an object search method, which is applied to a wearable device, comprising:

[0007] In response to the marking instruction, continuously collecting environmental image information;

[0008] Based on the environmental image information, obtaining real-time location information corresponding to at least one target object; the real-time location information includes object location information of the target object and carrier location information of the target object;

[0009] Storing the target object in correspondence with the corresponding object location information and carrier location information;

[0010] When a search instruction for the target object is received, a navigation signal is generated based on the object location information and the carrier location information.

[0011] In a possible implementation manner, before acquiring the real-time location information corresponding to at least one target object based on the environmental image information, the method further includes:

[0012] Get item feature information;

[0013] The acquiring, based on the environmental image information, real-time location information corresponding to at least one target object includes:

[0014] Based on the environmental image information, an object that meets the object feature information is used as the target object, and real-time position information corresponding to the target object is obtained.

[0015] In a possible implementation, the item feature information includes first item feature information and second item feature information;

[0016] The obtaining of item feature information comprises:

[0017] In response to the marking instruction, obtaining first item characteristic information carried by the marking instruction;

[0018] And, in response to the collection instruction, the second item feature information is obtained by a collection device set on the wearable device.

[0019] In a possible implementation, the method further includes:

[0020] Performing three-dimensional modeling on the target space environment to obtain a three-dimensional space model of the target space environment;

[0021] In response to the description instruction, at least one item tag and at least one space tag are created in the three-dimensional space model; the item tag is used to identify the original item in the three-dimensional space model, and the space tag is used to identify different space areas in the three-dimensional space model;

[0022] The acquiring, based on the environmental image information, real-time location information corresponding to at least one target object includes:

[0023] The real-time position information corresponding to the target object is obtained by combining the object label, the space label and the environmental image information in the three-dimensional space model.

[0024] In a possible implementation, the acquiring the real-time location information corresponding to the target object by combining the object label, the space label and the environment image information in the three-dimensional space model includes:

[0025] Matching the environment image information with the three-dimensional space model to obtain object labels and space labels contained in the environment image information;

[0026] When the target object is included in the environmental image information, the object location information is determined based on a space tag corresponding to a space region where the target object is located and an object tag of an original object adjacent to the target object.

[0027] In a possible implementation, the method further includes:

[0028] When the target object is not included in the environmental image information, obtaining historical environmental image information collected within a preset time period in the past;

[0029] When the environmental image information and the historical environmental image information indicate that the target object is stored in the original object, the carrier position information is determined based on the object tag corresponding to the original object.

[0030] In a possible implementation, when the environmental image information and the historical environmental image information indicate that the target item is stored in the original item, using the item tag of the corresponding original item as the carrier location information includes:

[0031] When the historical environmental image information includes the target object, the environmental image information does not include the target object, and the spatial tag included in the historical environmental image information and the spatial tag included in the environmental image information meet the preset requirements, the object tag of the original object adjacent to the target object in the historical environmental image information is used as the carrier position information.

[0032] In a possible implementation manner, the real-time location information also includes real-time time information and real-time image information;

[0033] After continuously collecting environmental image information in response to the marking instruction, the method further includes:

[0034] Obtaining acquisition time information corresponding to the environmental image information;

[0035] When the target object is included in the environment image information, determining the object location information based on a space tag corresponding to a space region where the target object is located and an object tag of an original object adjacent to the target object further includes:

[0036] When the target object is included in the environmental image information, the environmental image information is used as the real-time image information, and the acquisition time information corresponding to the environmental image information is used as the real-time time information.

[0037] In a possible implementation manner, storing the target object in correspondence with the corresponding object location information and carrier location information further includes:

[0038] storing the target object in correspondence with the corresponding object location information, carrier location information, real-time time information and real-time image information;

[0039] When receiving a search instruction for the target object, generating a navigation signal based on the object location information and the carrier location information includes:

[0040] When a search instruction for the target object is received, the navigation signal is generated based on the object location information, carrier location information, real-time time information and real-time image information corresponding to the target object.

[0041] In a possible implementation manner, storing the target object in correspondence with the corresponding object location information, carrier location information, real-time time information, and real-time image information further includes:

[0042] Delete the real-time location information corresponding to the target object stored in the past.

[0043] In a possible implementation, the method further includes:

[0044] When the environmental image information fails to match the three-dimensional space model and the environmental image information includes the target object, using the preset position information as the real-time position information of the target object;

[0045] The storing the target object in correspondence with the corresponding object location information and carrier location information includes:

[0046] storing the target object and the preset location information in correspondence with each other;

[0047] The generating of the navigation signal based on the object position information and the carrier position information comprises:

[0048] The navigation signal is generated based on the preset position information.

[0049] In a second aspect, an embodiment of the present application provides an item searching device, including:

[0050] A collection module, used for continuously collecting environmental image information in response to a marking instruction;

[0051] An acquisition module, configured to acquire real-time location information corresponding to at least one target object based on the environmental image information; the real-time location information includes object location information of the target object and carrier location information of the target object;

[0052] A storage module, used for storing the target object and the corresponding object location information and carrier location information in correspondence;

[0053] The navigation module is used to generate a navigation signal based on the object location information and the carrier location information when receiving a search instruction for the target object.

[0054] In a third aspect, an embodiment of the present application provides a computer device, including: a memory, a processor;

[0055] The memory stores computer-executable instructions;

[0056] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0057] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementations of the first aspect.

[0058] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect.

[0059] The object search method, apparatus, equipment, storage medium and program product provided in the embodiments of the present application are applied to wearable devices. Users can analyze the actual locations of important objects in their environment in real time through the wearable devices without affecting their daily activities, and update the placement information of important objects in real time. Moreover, when the wearable device obtains the object location information of the important objects, it can also further obtain the carrier location information corresponding to the important objects, thereby avoiding the situation where the user places the important objects inside other original objects and the original objects move, causing the actual location of the important objects to be inaccurate. When receiving a search instruction for the important objects, the wearable device can generate a navigation signal based on the object location information and the carrier location information to guide the user to find the important objects, thereby avoiding the problems of traditional paper memos that are not updated in a timely manner, have incomplete information, and cannot be found quickly. The wearable device can follow the user throughout the process, record the latest object placement location, and realize rapid determination and guidance of the object placement location. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0061] Figure 1 A schematic diagram of a scenario of the object search method provided in this application;

[0062] Figure 2 A flowchart of the item search method provided by this application;

[0063] Figure 3 A schematic diagram of the structure of the object search device provided by this application;

[0064] Figure 4 A schematic diagram of the structure of the computer device provided for this application.

[0065] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0066] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0067] The object search method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, the wearable device 102 communicates with the server 104 via a network.

[0068] For example, the object search method is applied to the wearable device 102. After receiving the marking instruction, the wearable device 102 continuously collects environmental image information; then, based on the environmental image information, obtains the real-time location information corresponding to at least one target object; the real-time location information includes the object location information of the target object and the carrier location information of the target object; and the target object and the corresponding object location information and carrier location information are stored in the data storage system of the server 104; finally, when the wearable device 102 receives the object search instruction for the target object, it generates a navigation signal based on the object location information and the carrier location information. Among them, the wearable device 102 can be smart glasses, smart head-mounted devices, smart vests, smart neck rings, etc. The server 104 can be implemented as an independent server or a server cluster composed of multiple servers. The wearable device 102 and the server 104 can be directly or indirectly connected through wired or wireless communication, such as through a network connection.

[0069] For another example, the object search method is applied to the server 104. After receiving the marking instruction, the wearable device 102 continuously collects environmental image information and sends the collected environmental image information to the server 104. Then, the server 104 obtains the real-time location information corresponding to at least one target object based on the environmental image information; the real-time location information includes the object location information of the target object and the carrier location information of the target object; and the target object and the corresponding object location information and carrier location information are stored in the data storage system; finally, when the server 104 receives the object search instruction for the target object, it generates a navigation signal based on the object location information and the carrier location information. It can be understood that the data storage system can be an independent storage device, or the data storage system is located on the server 104, or the data storage system is located on another terminal.

[0070] It should be noted that the network communication between the wearable device 102 and the server 104 is applicable to different network standards, for example, it can be applicable to global system of mobile communication (GSM), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time division synchronous code division multiple access (TD-SCDMA), long term evolution (LTE) system and future 5G network standards. Optionally, the above communication system can be a system in the scenario of ultra-reliable and low latency communications (URLLC) transmission in a 5G communication system.

[0071] Therefore, optionally, the above-mentioned base station can be a base station (Base Transceiver Station, referred to as BTS) and / or a base station controller in GSM or CDMA, or a base station (NodeB, referred to as NB) and / or a radio network controller (Radio Network Controller, referred to as RNC) in WCDMA, or an evolved base station (EvolutionalNode B, referred to as eNB or eNodeB) in LTE, or a relay station or access point, or a base station (gNB) in a future 5G network, etc., and the present application is not limited here.

[0072] In one embodiment, a method for searching for an item is provided. This embodiment uses the method applied to a wearable device as an example for illustration. It is understandable that the method can also be applied to a server, or to a system including a wearable device and a server, and is implemented through the interaction between the wearable device and the server. Figure 2 As shown, the item search method includes:

[0073] Step 202: In response to the marking instruction, continuously collect environmental image information.

[0074] The marking instruction refers to the instruction that triggers the acquisition device on the wearable device to acquire images and videos.

[0075] Among them, the marking instruction can be issued by the user of the wearable device by pressing a pre-set button on the wearable device, which button can be a power start button or a specific function button. Alternatively, the user of the wearable device can establish a communication connection between the terminal and the wearable device, and issue a marking instruction through the human-computer interaction interface of the terminal. The human-computer interaction interface can specifically display a designated platform interface related to the wearable device. The user issues a marking instruction by clicking a specific area on the designated platform interface. The specific area can, for example, represent a power start virtual button for powering the wearable device or a function virtual button for triggering the wearable device to perform image or video capture.

[0076] The wearable device in this embodiment is provided with a collection device, which may include a camera for image and video collection, and may also include a recording device for voice collection.

[0077] The environmental image information may include at least one environmental image. When the environmental image refers to a specific indoor environment, the environmental image may include the indoor spatial layout, the structure and hierarchy of the room (such as the positions of walls, doors and windows), and the furniture arrangement (sofas, tables and chairs, bookshelves, etc.).

[0078] Step 204: Based on the environmental image information, obtain real-time location information corresponding to at least one target object; the real-time location information includes object location information of the target object and carrier location information of the target object.

[0079] The target item may refer to an item that is pre-marked by a user and whose storage location needs to be recorded.

[0080] The real-time location information is used to indicate the actual placement position of the target object. The real-time location information includes the object location information and the carrier location information corresponding to the target object. The object location information is used to indicate the spatial location information of the target object and the adjacent position relationship information with other original objects in a specific indoor environment. Furthermore, when the target object is placed inside another original object, the carrier location information is used to indicate the spatial location information corresponding to the original object that carries the target object and the adjacent position relationship information between the original object that carries the target object and the other original objects.

[0081] Step 206: Store the target object in correspondence with the corresponding object location information and carrier location information.

[0082] For example, the wearable device can store the target item and the corresponding item location information and carrier location information locally, or send them to a server and store them in a data storage system.

[0083] Step 208: When receiving a search instruction for a target object, a navigation signal is generated based on the object location information and the carrier location information.

[0084] A find-and-seek command is a command to find the physical location of a specific item.

[0085] Among them, the search command can be issued by the user of the wearable device by pressing a pre-set voice collection button on the wearable device and giving a specific password through the collection device set on the wearable device. The specific password can be, for example, "find item A". Alternatively, the user of the wearable device can issue a search command through the human-computer interaction interface of the terminal. The human-computer interaction interface can specifically display a designated platform interface related to the wearable device. The user can issue a marking command by clicking a pre-integrated virtual button on the designated platform interface. The virtual button can be pre-connected to a specific target item.

[0086] The navigation signal refers to a signal that informs the user of the wearable device of the actual location of a specific object to guide the user to find the specific object.

[0087] The navigation signal may be in the form of voice, picture, or video, for example. When the navigation signal is in the form of voice, the wearable device may play the corresponding voice through a player set on the wearable device; when the navigation signal is in the form of picture or video, the wearable device may display the corresponding picture or video through the human-computer interaction interface of the communication-connected terminal.

[0088] The object search method provided in the embodiment of the present application is applied to wearable devices. Users can analyze the actual locations of important objects in the environment in real time through the wearable devices without affecting their daily activities, and update the placement information of important objects in real time. Moreover, when the wearable device obtains the object location information of the important objects, it can further obtain the carrier location information corresponding to the important objects, thereby avoiding the situation where the user puts the important objects into other original objects and the original objects move, causing the actual location of the important objects to be inaccurate. When receiving a search instruction for the important objects, the wearable device can generate a navigation signal based on the object location information and the carrier location information to guide the user to find the important objects, thereby avoiding the problems of traditional paper memos that are not updated in a timely manner, have incomplete information, and cannot be found quickly. The wearable device can follow the user throughout the process, record the latest object placement location, and realize rapid determination and guidance of the object placement location.

[0089] In some optional embodiments, before step 204, the following steps are included:

[0090] Get item feature information;

[0091] Step 204 includes:

[0092] Based on the environmental image information, the objects that meet the object feature information are taken as target objects, and the real-time location information corresponding to the target objects is obtained.

[0093] Item feature information is used to uniquely identify a specific item. Item feature information may include image feature information, text feature information, and other forms.

[0094] Specifically, the item feature information includes first item feature information and second item feature information.

[0095] As an example, in the step of the wearable device acquiring the characteristic information of the object, the user can, before issuing the marking instruction, input the image characteristic information, text characteristic information, etc. of the target object in advance through the human-computer interaction interface of the terminal connected to the wearable device as the first object characteristic information. Alternatively, the user can press the information collection button pre-set on the wearable device to issue a collection instruction to trigger the collection device to collect real-time images, and use the collected real-time images as the image characteristic information; or press the voice collection button pre-set on the wearable device to issue a collection instruction to trigger the collection device to collect audio, convert the collected audio into text, and use the converted text as the text characteristic information, the audio being, for example, "black round-frame glasses"; wherein the image characteristic information and text characteristic information collected by the collection device are used as the second object characteristic information.

[0096] Furthermore, in step 204, the wearable device may pre-process the environmental image information to perform normalization, denoising and other processing, and then use common models such as convolutional neural networks to extract feature vectors contained in the environmental image information. When the object feature information includes image feature information and text feature information, the wearable device may use word embedding technology (such as Word2Vec, GloVe) or other models (such as BERT model) to convert the text feature information into a vector representation, and use common models such as convolutional neural networks to convert the image feature information into a vector representation. Then, the feature vector contained in the environmental image information is matched with the vector representation of the text feature information and the vector representation of the image feature information, and various similarity metrics (such as cosine similarity, Euclidean distance) are used to evaluate the similarity. When the similarity reaches a preset threshold, the corresponding area in the environmental image information is used as the target object.

[0097] In this embodiment, the user can upload the relevant information of important items to the wearable device through various methods such as oral narration, taking photos, terminal input, etc., which facilitates the user to record important items.

[0098] In some optional embodiments, the item search method further includes:

[0099] Perform three-dimensional modeling on the target space environment to obtain a three-dimensional space model of the target space environment;

[0100] In response to the description instruction, at least one item tag and at least one space tag are created in the three-dimensional space model; the item tag is used to identify the original item in the three-dimensional space model, and the space tag is used to identify different space areas in the three-dimensional space model;

[0101] Step 204 includes:

[0102] The real-time location information of the target object is obtained by combining the object labels, space labels and environmental image information in the three-dimensional space model.

[0103] In this embodiment, the wearable device can, for example, pre-collect relevant data of various spaces and original objects in the target space environment, and perform point cloud processing on the relevant data and then use modeling software to perform three-dimensional modeling, and finally obtain a three-dimensional space model of the target space environment. It should be noted that this application does not limit the process of three-dimensional modeling based on a specific space.

[0104] The description instruction refers to an instruction for labeling in the generated three-dimensional space model. The description instruction can be issued by the user through the human-computer interaction interface of the terminal and after selecting the voxels to be marked, or it can be automatically generated by the wearable device after obtaining the three-dimensional space model. The wearable device can, for example, pre-acquire a common object-space feature information correspondence table from the data storage system of the server, and compare the three-dimensional space model with the feature information correspondence table. As an example, the feature information correspondence table may include the correspondence between original objects such as refrigerators, pots, and faucets and the kitchen space, and the correspondence between original objects such as wardrobes and beds and the bedroom space. The wearable device compares the feature information of the original objects such as refrigerators, pots, faucets, wardrobes, and beds contained in the three-dimensional space model with the feature information correspondence table, and automatically marks it, and then marks the space where each original object is located according to the above correspondence.

[0105] Both object tags and space tags are used to identify at least one voxel in a three-dimensional space model. It should be noted that the same voxel can carry both object tags and space tags. For example, when multiple voxels in a three-dimensional space model are marked with the object tag "refrigerator", these voxels can still be marked with the space tag "kitchen".

[0106] Furthermore, the step of obtaining the real-time location information corresponding to the target object by combining the object label, the space label and the environment image information in the three-dimensional space model includes:

[0107] Matching the environment image information with the three-dimensional space model to obtain the object labels and space labels contained in the environment image information;

[0108] When the target object is included in the environmental image information, the object location information is determined based on the space label corresponding to the spatial area where the target object is located and the object labels of the original objects adjacent to the target object.

[0109] The process of matching environmental image information with a three-dimensional space model involves a registration process of pixels in a two-dimensional image with voxels in a three-dimensional model. This application does not limit this registration process and will not be elaborated on in detail.

[0110] When the target object is included in the environmental image information, the wearable device can match the voxels around the target object in the three-dimensional space model based on the pixels around the target object, and determine the spatial area where the target object is located according to the spatial labels corresponding to these voxels, and determine the original objects adjacent to the target object according to the object labels corresponding to these voxels, and use the spatial area where the target object is located and the original objects adjacent to the target object as the object location information. As an example, when the voxels around the target object "glasses" correspond to the spatial label "bedroom", and the voxels around the target object "glasses" correspond to the object label "bedside table", the object location information can be "near the bedside table in the bedroom".

[0111] In some optional embodiments, the item search method further includes:

[0112] When the environmental image information does not contain the target object, the historical environmental image information collected within a preset time period is obtained;

[0113] When the environmental image information and the historical environmental image information indicate that the target item is stored in the original item, the carrier position information is determined based on the item tag of the original item.

[0114] There are two situations in which the target object is not included in the environmental image information. One is that the target object is not in the spatial area corresponding to the environmental image information, and the other is that the target object is in the spatial area corresponding to the environmental image information and is placed inside the original object contained in the spatial area. At this time, the historical environmental image information collected within a preset time period in the past can be obtained. If the historical environmental image information and the environmental image information indicate that the target object is stored in the original object, the wearable device can use the object tag corresponding to the original object storing the target object as the carrier location information.

[0115] Specifically, when the historical environmental image information contains the target object and the environmental image information does not contain the target object, and the spatial tag contained in the historical environmental image information and the spatial tag contained in the environmental image information meet the preset requirements, the object tag of the original object adjacent to the target object in the historical environmental image information is used as the carrier location information.

[0116] Among them, the preset requirement may refer to the similarity between the spatial tag contained in the historical environmental image information and the spatial tag contained in the environmental image information reaching the preset similarity threshold. Through this setting, it can be determined that the viewing angle of the acquisition device has not changed, and the spatial area range that the acquisition device can collect has not changed. At this time, the target item disappears, which can be understood as the target item being stored in other original items within the current spatial area. The wearable device can determine the original item storing the target item based on the pixel point closest to the target item in the historical environmental image information and the item label of the corresponding voxel, and use the original item storing the target item as the carrier location information. As an example, when the voxels around the target item "glasses" in the historical environmental image correspond to the original item "bedside table", the carrier location information can be "inside the bedside table".

[0117] It should be noted that when the target object has carrier location information, the object location information corresponding to the original object storing the target object can be directly stored as the location information of the target object.

[0118] Furthermore, the wearable device can establish a connection between the original item "bedside table" and the target item "glasses". When the subsequently collected environmental image information indicates that the original item "bedside table" has moved, the wearable device can obtain the item location information of the original item "bedside table" as the item location information of the target item "glasses".

[0119] It should be noted that when a target object has both object location information and carrier location information, and the object location information and the carrier location information are inconsistent, the real-time location information of the target object can be generated according to the generation time corresponding to the object location information and the carrier location information, based on the generation time with a newer generation time, where the generation time can be understood as the collection time of the environmental image information corresponding to the object location information and the carrier location information.

[0120] In this embodiment, based on image processing, it is possible to screen out situations where important items are placed inside other original items, so that the location of the important items can be tracked by monitoring the original items that store the important items, thereby improving the accuracy of locating the important items.

[0121] In some optional embodiments, the real-time location information also includes real-time time information and real-time image information;

[0122] After step 202, the method further includes:

[0123] Obtaining acquisition time information corresponding to the environmental image information;

[0124] When the environment image information includes the target object, the step of determining the object location information based on the space label corresponding to the space region where the target object is located and the object label of the original object adjacent to the target object also includes:

[0125] When the environmental image information contains the target object, the environmental image information is used as the real-time image information, and the acquisition time information corresponding to the environmental image information is used as the real-time time information.

[0126] Correspondingly, step 206 includes: storing the target object in correspondence with the corresponding object location information, carrier location information, real-time time information and real-time image information.

[0127] Step 208 includes: upon receiving a search instruction for a target object, generating a navigation signal based on object location information, carrier location information, real-time time information, and real-time image information corresponding to the target object.

[0128] Furthermore, when storing the object location information, carrier location information, real-time time information and real-time image information of the target object, the wearable device can first determine whether the historical real-time location information corresponding to the target object already exists at the stored location. If so, the historical real-time location information is deleted to update the real-time location information of the target object.

[0129] In this embodiment, the real-time location information not only includes the spatial area where the target item is located, the original items adjacent to the target item, and the original items storing the target item, but also includes the specific image of the environmental image information, and the real-time time information when the acquisition device acquires the environmental image information. Through this setting, when the wearable device subsequently generates a navigation signal, it can not only provide accurate item placement information, but also display the picture of the user placing the target item and the specific time of placing the target item to the user, thereby preventing the user from forgetting the specific location of the target item and facilitating the user to quickly find the target item.

[0130] In some optional embodiments, the item search method further includes:

[0131] When the environmental image information fails to match the three-dimensional space model and the environmental image information contains the target object, the preset position information is used as the real-time position information of the target object;

[0132] Step 206 includes: storing the target object and the preset location information in correspondence;

[0133] Step 208 includes: generating a navigation signal based on the preset position information.

[0134] When the environmental image information fails to match the three-dimensional space model and the environmental image information contains the target object, it can be considered that the target object has been moved outside the target space environment. At this time, the wearable device cannot locate the specific position of the target object, and the preset position information is directly used as the real-time position information of the target object. The preset position information can be, for example, "outside the target space environment."

[0135] As an example, the target space environment is the user's home. After the user takes the target item "glasses" out of the house, if the environmental image information outside the home collected by the wearable device fails to match the three-dimensional space model, and the target item "glasses" appears in the environmental image information, the wearable device uses the preset location information "outside the home" as the real-time location information. When receiving a search instruction for the target item "glasses", a navigation signal "glasses are outside the home" is generated based on the preset location information "outside the home".

[0136] Alternatively, the wearable device may use the preset location information, environmental image information, and real-time time information of the environmental image information as the real-time location information of the target item. As an example, the wearable device uses the preset location information "outside home" and the real-time time information of the environmental image information "A year B month C time" as the real-time location information. When a search instruction for the target item "glasses" is received, a navigation signal "the glasses are outside home at A year B month C time" is generated based on the preset location information "outside home", the environmental image information and the real-time time information "A year B month C time", and the environmental image information is displayed.

[0137] The above-mentioned item searching method, through a wearable device that can be easily carried, enables the user to perform real-time detection and analysis of the daily environment without being affected in daily activities, so as to find the actual location of important items in the environment and store the actual location of the important items. In addition, the wearable device will also store the actual location of the original item of the important item, which will be directly regarded as the actual location of the important item, thereby avoiding the situation where the user puts the important item into other original items and the actual location of the important item is inaccurate because the important item cannot be detected by the collection device; at the same time, when receiving a search instruction for the important item, the collected environmental image information and the time information when the collection is performed can be displayed to the user together, which is convenient for the user to remember and realize the quick search of the important item.

[0138] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0139] Based on the same inventive concept, the embodiment of the present application also provides an item search device for implementing the item search method involved above. The implementation solution provided by the item search device to solve the problem is similar to the implementation solution recorded in the above item search method, so the specific limitations in one or more device embodiments provided below can refer to the limitations of the item search method above, and will not be repeated here.

[0140] In one embodiment, Figure 3 As shown, an object search device 300 is provided, comprising:

[0141] The acquisition module 302 is used to continuously acquire environmental image information in response to the marking instruction;

[0142] The acquisition module 304 is used to acquire the real-time location information corresponding to at least one target object based on the environmental image information; the real-time location information includes the object location information of the target object and the carrier location information of the target object;

[0143] The storage module 306 is used to store the target object and the corresponding object location information and carrier location information in correspondence;

[0144] The navigation module 308 is used to generate a navigation signal based on the object location information and the carrier location information when receiving a search instruction for a target object.

[0145] In some optional embodiments, the acquisition module 302 is further configured to:

[0146] Get item feature information;

[0147] Based on the environmental image information, the objects that meet the object feature information are taken as target objects, and the real-time location information corresponding to the target objects is obtained.

[0148] In some optional embodiments, the item feature information includes first item feature information and second item feature information;

[0149] The acquisition module 302 is further configured to:

[0150] In response to the marking instruction, obtaining first item characteristic information carried by the marking instruction;

[0151] And, in response to the collection instruction, the second item feature information is obtained through a collection device set on the wearable device.

[0152] In some optional embodiments, the acquisition module 304 is further configured to:

[0153] Perform three-dimensional modeling on the target space environment to obtain a three-dimensional space model of the target space environment;

[0154] In response to the description instruction, at least one item tag and at least one space tag are created in the three-dimensional space model; the item tag is used to identify the original item in the three-dimensional space model, and the space tag is used to identify different space areas in the three-dimensional space model;

[0155] The real-time location information of the target object is obtained by combining the object labels, space labels and environmental image information in the three-dimensional space model.

[0156] In some optional embodiments, the acquisition module 304 is further configured to:

[0157] Matching the environment image information with the three-dimensional space model to obtain the object labels and space labels contained in the environment image information;

[0158] When the target object is included in the environmental image information, the object location information is determined based on the space label corresponding to the spatial area where the target object is located and the object labels of the original objects adjacent to the target object.

[0159] In some optional embodiments, the acquisition module 304 is further configured to:

[0160] When the environmental image information does not contain the target object, the historical environmental image information collected within a preset time period is obtained;

[0161] When the environmental image information and the historical environmental image information indicate that the target object is stored in the original object, the carrier position information is determined based on the object tag corresponding to the original object.

[0162] In some optional embodiments, the acquisition module 304 is further configured to:

[0163] When the historical environmental image information contains the target object, the environmental image information does not contain the target object, and the spatial tag contained in the historical environmental image information and the spatial tag contained in the environmental image information meet the preset requirements, the object tag of the original object adjacent to the target object in the historical environmental image information is used as the carrier location information.

[0164] In some optional embodiments, the real-time location information also includes real-time time information and real-time image information;

[0165] The acquisition module 302 is further configured to:

[0166] Obtaining acquisition time information corresponding to the environmental image information;

[0167] The acquisition module 304 is further configured to: when the environment image information contains the target object, use the environment image information as real-time image information, and use the acquisition time information corresponding to the environment image information as real-time time information.

[0168] In some optional embodiments, the storage module 306 is further configured to:

[0169] storing the target object in correspondence with the corresponding object location information, carrier location information, real-time time information and real-time image information;

[0170] The navigation module 308 is also configured to:

[0171] When a search instruction for a target object is received, a navigation signal is generated based on the object location information, carrier location information, real-time time information and real-time image information corresponding to the target object.

[0172] In some optional embodiments, the storage module 306 is further configured to:

[0173] Delete the real-time location information corresponding to the target object that was stored in the past.

[0174] In some optional embodiments, the acquisition module 304 is further configured to:

[0175] When the environmental image information fails to match the three-dimensional space model and the environmental image information contains the target object, the preset position information is used as the real-time position information of the target object;

[0176] The storage module 306 is further configured to:

[0177] Storing the target object and the preset location information in correspondence;

[0178] The navigation module 308 is also configured to:

[0179] A navigation signal is generated based on the preset position information.

[0180] Each module in the above device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0181] The (device subject) provided in this embodiment can execute the object search method provided in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be described in detail here.

[0182] Figure 4 This is a schematic diagram of the structure of the computer device provided in this application. Figure 4 As shown, the electronic device 400 provided in this embodiment includes: at least one processor 401 and a memory 402. Optionally, the device 40 further includes a communication component 403. The processor 401, the memory 402 and the communication component 403 are connected via a bus 404.

[0183] In a specific implementation process, at least one processor 401 executes the computer-executable instructions stored in the memory 402, so that at least one processor 401 executes the above method.

[0184] The specific implementation process of the processor 401 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0185] In the above embodiments, it should be understood that the processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the invention can be directly implemented as a hardware processor, or can be implemented by a combination of hardware and software modules in the processor.

[0186] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage.

[0187] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0188] The present application also provides a computer program product, including a computer program, which implements the above-mentioned object search method when executed by a processor.

[0189] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above-mentioned object search method is implemented.

[0190] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0191] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (Application Specific Integrated Circuits, referred to as: ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0192] The division of units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0193] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0194] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0195] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0196] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0197] Finally, it should be noted that those skilled in the art will readily conceive of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention, which follow the general principles of the present invention and include common knowledge or customary technical means in the art not disclosed by the present invention, are not limited to the precise structure described above and shown in the drawings, and may be modified and changed in various ways without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A method for searching an item. It is characterized in that Applied to wearable devices, including: In response to the marking instruction, continuously collecting environmental image information; Based on the environmental image information, obtaining real-time location information corresponding to at least one target object; the real-time location information includes object location information of the target object and carrier location information of the target object; Storing the target object in correspondence with the corresponding object location information and carrier location information; When a search instruction for the target object is received, a navigation signal is generated based on the object location information and the carrier location information.

2. The method according to claim 1, It is characterized in that Before acquiring the real-time location information corresponding to at least one target object based on the environmental image information, the method further includes: Get item feature information; The acquiring, based on the environmental image information, real-time location information corresponding to at least one target object includes: Based on the environmental image information, an object that meets the object feature information is used as the target object, and real-time position information corresponding to the target object is obtained.

3. The method according to claim 2, It is characterized in that The item feature information includes first item feature information and second item feature information; The obtaining of item feature information comprises: In response to the marking instruction, obtaining first item characteristic information carried by the marking instruction; And, in response to the collection instruction, the second item feature information is obtained by a collection device set on the wearable device.

4. The method according to claim 1, It is characterized in that The method further comprises: Performing three-dimensional modeling on the target space environment to obtain a three-dimensional space model of the target space environment; In response to the description instruction, at least one item tag and at least one space tag are created in the three-dimensional space model; the item tag is used to identify the original item in the three-dimensional space model, and the space tag is used to identify different space areas in the three-dimensional space model; The acquiring, based on the environmental image information, real-time location information corresponding to at least one target object includes: The real-time position information corresponding to the target object is obtained by combining the object label, the space label and the environmental image information in the three-dimensional space model.

5. The method according to claim 4, It is characterized in that The step of combining the object label, the space label and the environment image information in the three-dimensional space model to obtain the real-time location information corresponding to the target object includes: Matching the environment image information with the three-dimensional space model to obtain object labels and space labels contained in the environment image information; When the target object is included in the environmental image information, the object location information is determined based on a space tag corresponding to a space region where the target object is located and an object tag of an original object adjacent to the target object.

6. The method according to claim 5, It is characterized in that The method further comprises: When the target object is not included in the environmental image information, obtaining historical environmental image information collected within a preset time period in the past; When the environmental image information and the historical environmental image information indicate that the target object is stored in the original object, the carrier position information is determined based on the object tag corresponding to the original object.

7. The method according to claim 6, It is characterized in that When the environment image information and the historical environment image information indicate that the target object is stored in the original object, using the corresponding object tag of the original object as the carrier position information includes: When the historical environmental image information includes the target object, the environmental image information does not include the target object, and the spatial tag included in the historical environmental image information and the spatial tag included in the environmental image information meet the preset requirements, the object tag of the original object adjacent to the target object in the historical environmental image information is used as the carrier position information.

8. The method according to claim 5, It is characterized in that The real-time location information also includes real-time time information and real-time image information; After continuously collecting environmental image information in response to the marking instruction, the method further includes: Obtaining acquisition time information corresponding to the environmental image information; When the target object is included in the environment image information, determining the object location information based on a space tag corresponding to a space region where the target object is located and an object tag of an original object adjacent to the target object further includes: When the target object is included in the environmental image information, the environmental image information is used as the real-time image information, and the acquisition time information corresponding to the environmental image information is used as the real-time time information.

9. The method according to claim 8, It is characterized in that The storing the target object in correspondence with the corresponding object location information and carrier location information also includes: storing the target object in correspondence with the corresponding object location information, carrier location information, real-time time information and real-time image information; When receiving a search instruction for the target object, generating a navigation signal based on the object location information and the carrier location information includes: When a search instruction for the target object is received, the navigation signal is generated based on the object location information, carrier location information, real-time time information and real-time image information corresponding to the target object.

10. The method according to claim 9, It is characterized in that The storing the target object in correspondence with the corresponding object location information, carrier location information, real-time time information and real-time image information also includes: Delete the real-time location information corresponding to the target object stored in the past.

11. The method according to claim 5, It is characterized in that The method further comprises: When the environmental image information fails to match the three-dimensional space model and the environmental image information includes the target object, using the preset position information as the real-time position information of the target object; The storing the target object in correspondence with the corresponding object location information and carrier location information includes: storing the target object and the preset location information in correspondence with each other; The generating of the navigation signal based on the object position information and the carrier position information comprises: The navigation signal is generated based on the preset position information.

12. An object search device, It is characterized in that include: A collection module, used for continuously collecting environmental image information in response to a marking instruction; An acquisition module, configured to acquire real-time location information corresponding to at least one target object based on the environmental image information; the real-time location information includes object location information of the target object and carrier location information of the target object; A storage module, used for storing the target object and the corresponding object location information and carrier location information in correspondence; The navigation module is used to generate a navigation signal based on the object location information and the carrier location information when receiving a search instruction for the target object.

13. A computer device, It is characterized in that include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor executes the object search method according to any one of claims 1 to 11.

14. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the object search method according to any one of claims 1 to 11.

15. A computer program product, It is characterized in that The invention comprises a computer program, which, when executed by a processor, implements the object search method according to any one of claims 1 to 11.

Citation Information

Cited By

  • Method and apparatus for searching for item, and device, storage medium and program product

    WO2026036918A1