Method and apparatus for searching for item, and device, storage medium and program product

Abstract

The embodiments of the present application relate to the technical field of artificial intelligence. Provided are a method and apparatus for searching for an item, and a device, a storage medium and a program product. The method comprises: in response to a marking instruction, continuously collecting environmental image information; on the basis of the environmental image information, acquiring real-time position information corresponding to at least one target item, wherein the real-time position information comprises item position information of the target item, and carrier position information of the target item; storing the target item in correspondence with the corresponding item position information and carrier position information; and upon receiving a search instruction for the target item, on the basis of the item position information and the carrier position information, generating a navigation signal. The method is used for achieving rapid determination of and guidance to a position where an item is placed.

Description

Article searching method, device, equipment, storage medium and program product

[0001] The present application claims priority to the Chinese patent application No. 202411116244.6, filed on August 14, 2024, and entitled "Article searching method, device, equipment, storage medium and program product", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the field of artificial intelligence, more specifically, to an article searching method, device, equipment, storage medium and program product. BACKGROUND

[0003] It is well known that the population aging trend is highly related to the level of economic development. With the development of domestic economy, the proportion of the middle-aged and elderly population will continue to increase. Because of memory loss, the middle-aged and elderly population often forget the storage location of some important articles.

[0004] Usually, the middle-aged and elderly population will record the location information of important articles by means of a memo. However, the memo may not be updated in time and the information may not be comprehensive. Moreover, the memo (electronic or paper) itself also needs a location to be stored and may not be quickly found when needed.

[0005] How to quickly determine the location of an article is a problem to be solved. SUMMARY

[0006] The embodiments of the present application provide an article searching method, device, equipment, storage medium and program product to achieve the effect of quick and automatic searching of articles.

[0007] In a first aspect, the embodiments of the present application provide an article searching method applied to a wearable device, comprising:

[0008] In response to a marking instruction, continuously collecting environment image information;

[0009] Based on the environment image information, obtaining real-time location information corresponding to at least one target article; the real-time location information includes article location information of the target article and carrier location information of the target article;

[0010] Storing the target article and the corresponding article location information and carrier location information;

[0011] When receiving a searching instruction for the target article, generating a navigation signal based on the article location information and the carrier location information.

[0012] In a possible implementation, before the acquiring the real-time location information corresponding to the at least one target item based on the environment image information, the method further includes:

[0013] acquiring item feature information;

[0014] The acquiring the real-time location information corresponding to the at least one target item based on the environment image information includes:

[0015] based on the environment image information, taking an item meeting the item feature information as the target item, and acquiring the real-time location information corresponding to the target item.

[0016] In a possible implementation, the item feature information includes first item feature information and second item feature information.

[0017] The acquiring the item feature information includes:

[0018] in response to the marking instruction, acquiring the first item feature information carried by the marking instruction;

[0019] and, in response to a collection instruction, acquiring the second item feature information by using a collection device arranged on the wearable device.

[0020] In a possible implementation, the method further includes:

[0021] performing three-dimensional modeling on a target space environment to acquire a three-dimensional space model of the target space environment;

[0022] in response to a description instruction, creating at least one item label and at least one space label in the three-dimensional space model; the item label is used to identify an original item in the three-dimensional space model, and the space label is used to identify different space regions in the three-dimensional space model;

[0023] The acquiring the real-time location information corresponding to the at least one target item based on the environment image information includes:

[0024] combining the item label, the space label in the three-dimensional space model, and the environment image information to acquire the real-time location information corresponding to the target item.

[0025] In a possible implementation, the combining the item label, the space label in the three-dimensional space model, and the environment image information to acquire the real-time location information corresponding to the target item includes:

[0026] matching the environment image information with the three-dimensional space model to acquire the item label and the space label contained in the environment image information;

[0027] When the target item is contained in the environment image information, the item position information is determined based on a space label corresponding to a space region where the target item is located and an item label of a neighboring original item of the target item.

[0028] In a possible implementation, the method further includes:

[0029] When the target item is not contained in the environment image information, historical environment image information collected in a preset time period is acquired.

[0030] When the environment image information and the historical environment image information indicate that the target item is stored in the original item, the carrier position information is determined based on an item label corresponding to the original item.

[0031] In a possible implementation, when the environment image information and the historical environment image information indicate that the target item is stored in the original item, the item label of the corresponding original item is taken as the carrier position information.

[0032] When the historical environment image information contains the target item, the environment image information does not contain the target item, and a space label contained in the historical environment image information meets a preset requirement with a space label contained in the environment image information, an item label of a neighboring original item of the target item in the historical environment image information is taken as the carrier position information.

[0033] In a possible implementation, the real-time position information further includes real-time time information and real-time image information.

[0034] After the environment image information is continuously acquired in response to the marking instruction, the method further includes:

[0035] Acquisition time information corresponding to the environment image information is acquired.

[0036] When the target item is contained in the environment image information, the item position information is determined based on a space label corresponding to a space region where the target item is located and an item label of a neighboring original item of the target item.

[0037] When the target item is contained in the environment image information, the environment image information is taken as the real-time image information, and acquisition time information corresponding to the environment image information is taken as the real-time time information.

[0038] In a possible implementation, the target item and the corresponding item position information and carrier position information are stored correspondingly, and the method further includes:

[0039] store the target item and corresponding item position information, carrier position information, real-time time information and real-time image information correspondingly;

[0040] The navigation signal is generated based on the item position information and the carrier position information when a search instruction for the target item is received.

[0041] The navigation signal is generated based on the item position information and the carrier position information when a search instruction for the target item is received.

[0042] In a possible implementation, the storing of the target item and corresponding item position information, carrier position information, real-time time information and real-time image information correspondingly further includes:

[0043] The real-time position information corresponding to the target item stored in the past is deleted.

[0044] In a possible implementation, the method further includes:

[0045] When the environment image information fails to match the three-dimensional space model and the environment image information contains the target item, preset position information is used as real-time position information of the target item.

[0046] The storing of the target item and corresponding item position information, carrier position information correspondingly includes:

[0047] The target item and the preset position information are stored correspondingly.

[0048] The generating of the navigation signal based on the item position information and the carrier position information includes:

[0049] The navigation signal is generated based on the preset position information.

[0050] In a second aspect, an item searching device is provided, which includes:

[0051] The acquisition module is configured to continuously acquire environment image information in response to a marking instruction.

[0052] The acquisition module is configured to continuously acquire environment image information in response to a marking instruction.

[0053] The storage module is configured to store the target item and corresponding item position information, carrier position information correspondingly.

[0054] A navigation module is configured to generate a navigation signal based on the item location information and the carrier location information when a search instruction for the target item is received.

[0055] In a third aspect, an embodiment of the present application provides a computer device, comprising: a memory, a processor;

[0056] The memory stores computer execution instructions.

[0057] The processor executes the computer execution instructions stored in the memory, so that the processor executes the first aspect and / or various possible implementation manners of the first aspect.

[0058] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, wherein the computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by a processor to implement the first aspect and / or various possible implementation manners of the first aspect.

[0059] In a fifth aspect, an embodiment of the present application provides a computer program product, comprising a computer program, and the computer program is executed by a processor to implement the first aspect and / or various possible implementation manners of the first aspect.

[0060] The item finding method, device, equipment, storage medium and program product provided by the embodiments of the present application are applied to a wearable device, and a user can analyze the actual position of an important item in an environment in real time through the wearable device without being affected by daily activities, and update the placement position information of the important item in real time. In addition, when the wearable device acquires the item location information of the important item, the wearable device can further acquire the carrier location information corresponding to the important item, so as to avoid the case that after the important item is placed in other original items, the movement of the original items causes the actual position of the important item to be inaccurate. When a search instruction for the important item is received, the wearable device can generate a navigation signal based on the item location information and the carrier location information to guide the user to find the important item, thereby avoiding the problems of traditional paper memo, such as untimely update, incomplete information and inability to find quickly. The wearable device can follow the user throughout the process, record the latest item placement position, and realize quick determination and guidance of the item placement position. BRIEF DESCRIPTION OF DRAWINGS

[0061] The accompanying drawings, which are incorporated herein and form part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the application.

[0062] FIG. 1 is a scene schematic diagram of an item finding method provided by the present application;

[0063] FIG. 2 is a flowchart of an article searching method according to an embodiment of the present application;

[0064] FIG. 3 is a structural diagram of an article searching device according to an embodiment of the present application;

[0065] FIG. 4 is a structural diagram of a computer device according to an embodiment of the present application.

[0066] The above-described embodiments of the present application have been shown and described, and the following detailed description will be given. These drawings and detailed description are not intended to limit the scope of the present application in any way, but to explain the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0067] The exemplary embodiments will be described in detail herein with reference to the attached drawings. In the following description, the same numbers are used to indicate the same or similar components. The embodiments described in the following exemplary embodiments are not meant to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the present application as detailed in the appended claims.

[0068] The article searching method according to an embodiment of the present application can be applied to an application environment as shown in FIG. 1. In the application environment, a wearable device 102 communicates with a server 104 through a network.

[0069] For example, the article searching method is applied to the wearable device 102. After receiving a marking instruction, the wearable device 102 continuously acquires environmental image information. Then, based on the environmental image information, real-time location information corresponding to at least one target article is obtained. The real-time location information includes article location information of the target article and carrier location information of the target article. The target article and the corresponding article location information and carrier location information are stored in a data storage system of the server 104. Finally, when the wearable device 102 receives a searching instruction for the target article, a navigation signal is generated based on the article location information and the carrier location information. The wearable device 102 can be smart glasses, a smart head-mounted device, a smart vest, a smart neck ring, etc. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers. The wearable device 102 and the server 104 can be directly or indirectly connected through wired or wireless communication, for example, through network connection.

[0070] For example, the item searching method is applied to the server 104, and the wearable device 102 continuously collects the environmental image information after receiving the marking instruction, and sends the collected environmental image information to the server 104. Then, the server 104 obtains the real-time position information corresponding to the at least one target item based on the environmental image information. The real-time position information includes the item position information of the target item and the carrier position information of the target item. The target item, the corresponding item position information and the carrier position information are stored in the data storage system. Finally, the server 104 generates a navigation signal based on the item position information and the carrier position information when receiving a searching instruction for the target item. It can be understood that the data storage system can be a separate storage device, or the data storage system is located on the server 104, or the data storage system is located on another terminal.

[0071] It should be noted that the network communication between the wearable device 102 and the server 104 is suitable for different network systems, for example, it can be suitable for Global System of Mobile communication (GSM), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division-Synchronous Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE) system and future 5G network system. Optionally, the above communication system can be a system in the scene of Ultra-Reliable and Low Latency Communications (URLLC) transmission in the 5G communication system.

[0072] Therefore, optionally, the above base station can be a base transceiver station (BTS) and / or a base station controller in GSM or CDMA, or a node B (NB) and / or a radio network controller (RNC) in WCDMA, or an evolutional node B (eNB or eNodeB) in LTE, or a relay station or an access point, or a base station (gNB) in future 5G network, etc., which is not limited herein.

[0073] In an embodiment, an article searching method is provided, which is exemplified by being applied to a wearable device. It can be understood that the method can also be applied to a server, and can also be applied to a system including a wearable device and a server, and can be implemented through the interaction of the wearable device and the server. As shown in FIG. 2, the article searching method includes:

[0074] In step 202, environmental image information is continuously collected in response to a marking instruction.

[0075] The marking instruction refers to an instruction for triggering the image and video collection of the collection device on the wearable device.

[0076] The marking instruction can be issued by pressing a pre-set button on the wearable device by the user of the wearable device. The button can be a power-on button or a specific function button. Alternatively, the user of the wearable device can establish a communication connection between a terminal and the wearable device, and issue the marking instruction through the human-computer interaction interface of the terminal. The human-computer interaction interface can specifically display a designated platform interface related to the wearable device. The user can issue the marking instruction by clicking a specific area on the designated platform interface. The specific area can be represented by a power-on virtual button for powering the wearable device or a function virtual button for triggering the image and video collection of the wearable device.

[0077] The wearable device in the embodiment is provided with a collection device. The collection device can include a camera, a camera, etc. for image and video collection, and can also include a recording device, etc. for voice collection.

[0078] The environmental image information can include at least one environmental image. When the environmental image refers to a specific indoor environment, the environmental image can include the spatial layout of the indoor environment, the structure and hierarchy of the room (such as the position of the wall, door and window), and the furniture arrangement (such as sofa, table, chair, bookshelf, etc.).

[0079] In step 204, real-time location information corresponding to at least one target article is obtained based on the environmental image information. The real-time location information includes article location information of the target article and carrier location information of the target article.

[0080] The target article refers to an article that is pre-marked by the user and needs to record the storage location.

[0081] The real-time position information is used to indicate an actual placement position of the target item, and includes item position information corresponding to the target item and carrier position information. The item position information is used to indicate spatial position information of the target item and adjacent position relationship information of the target item with other original items in a specific indoor environment. Further, when the target item is placed inside another original item, the carrier position information is used to indicate spatial position information of the original item carrying the target item and adjacent position relationship information of the original item carrying the target item with other original items.

[0082] In step 206, the target item is stored in correspondence with the corresponding item position information and carrier position information.

[0083] The wearable device may, for example, store the target item in correspondence with the corresponding item position information and carrier position information locally or send the target item to a server for storage in a data storage system.

[0084] In step 208, a navigation signal is generated based on the item position information and the carrier position information when a search instruction for the target item is received.

[0085] The search instruction refers to an instruction to find an actual placement position of a specific item.

[0086] The search instruction may, for example, be issued by a user of the wearable device by pressing a pre-set voice collection button on the wearable device and giving a specific password through a collection device provided on the wearable device. The specific password may, for example, be "find item A". Alternatively, the user of the wearable device may issue the search instruction through a human-computer interaction interface of a terminal. The human-computer interaction interface may, for example, display a designated platform interface of the wearable device, and the user may issue the marking instruction by clicking a pre-integrated virtual button on the designated platform interface. The virtual button may be pre-connected to a specific target item.

[0087] The navigation signal refers to a signal that informs a user of the wearable device of an actual placement position of a specific item to guide the user to find the specific item.

[0088] The navigation signal may, for example, be in the form of voice, pictures, or video. When the navigation signal is in the form of voice, the wearable device may play the corresponding voice through a player provided on the wearable device. When the navigation signal is in the form of pictures or video, the wearable device may display the corresponding pictures or video through a human-computer interaction interface of a terminal connected in communication.

[0089] The article searching method provided by the embodiment of the application is applied to a wearable device, and a user can analyze the actual position of an important article in an environment in real time through the wearable device without being affected by daily activities, and update the placement position information of the important article in real time. In addition, when the wearable device acquires the article position information of the important article, the wearable device can further acquire the carrier position information corresponding to the important article, so that when the user puts the important article into another original article, the movement of the original article does not cause the actual position of the important article to be inaccurate. When a searching instruction for the important article is received, the wearable device can generate a navigation signal based on the article position information and the carrier position information to guide the user to find the important article, so that the problems of a traditional paper memo that is not updated in time, information is not comprehensive, and the important article cannot be found quickly are avoided. The wearable device can follow the user throughout the process, record the latest article placement position, and realize quick determination and guidance of the article placement position.

[0090] In some optional embodiments, before step 204, the following steps are included:

[0091] acquiring article feature information;

[0092] Step 204 includes:

[0093] based on the environment image information, taking an article that meets the article feature information as a target article, and acquiring real-time position information corresponding to the target article.

[0094] The article feature information is used to uniquely indicate a specific article. The article feature information can include image feature information, text feature information, and the like.

[0095] Specifically, the article feature information includes first article feature information and second article feature information.

[0096] As an example, in the step of acquiring the article feature information by the wearable device, before issuing a marking instruction, the user can input image feature information, text feature information, and the like of the target article as the first article feature information through a human-computer interaction interface of a terminal connected to the wearable device, or the user can press a pre-set information collection button on the wearable device to issue a collection instruction to trigger a collection device to collect real-time images, and the collected real-time images are taken as image feature information. Alternatively, the user can press a pre-set voice collection button on the wearable device to issue a collection instruction to trigger the collection device to collect audio, and the collected audio is converted into text, and the converted text is taken as text feature information. The audio is, for example, "black round frame glasses". The image feature information and the text feature information collected by the collection device are taken as the second article feature information.

[0097] Further, in step 204, the wearable device can preprocess the environmental image information for normalization, denoising, etc., and then extract the feature vectors contained in the environmental image information using common models such as convolutional neural networks. When the item feature information contains image feature information and text feature information, the wearable device can use word embedding techniques (such as Word2Vec, GloVe) or other models (such as the BERT model) to convert the text feature information into vector representation, use common models such as convolutional neural networks to convert the image feature information into vector representation, then match the feature vectors contained in the environmental image information with the vector representation of the text feature information and the vector representation of the image feature information, and use various similarity measures (such as cosine similarity, Euclidean distance) to evaluate the similarity. When the similarity reaches a preset threshold, the corresponding region in the environmental image information is taken as the target item.

[0098] In this embodiment, the user can upload the relevant information of the important item to the wearable device through various ways such as oral description, photographing, terminal input, etc., which facilitates the user to record the important item.

[0099] In some optional embodiments, the item finding method further comprises:

[0100] Three-dimensional modeling is performed on the target space environment to obtain a three-dimensional space model of the target space environment;

[0101] In response to the description instruction, at least one item label and at least one space label are created in the three-dimensional space model; the item label is used to identify the original item in the three-dimensional space model, and the space label is used to identify different space regions in the three-dimensional space model;

[0102] Step 204 comprises:

[0103] The real-time location information corresponding to the target item is obtained in combination with the item label, the space label, and the environmental image information in the three-dimensional space model.

[0104] In this embodiment, the wearable device can pre-collect relevant data of each space and original item in the target space environment, and then perform three-dimensional modeling on the relevant data after point cloud processing using modeling software, to finally obtain a three-dimensional space model of the target space environment. It should be noted that the process of three-dimensional modeling based on a specific space is not limited in the present application.

[0105] The description instruction refers to the instruction of labeling in the generated three-dimensional space model. The description instruction can be issued by the user through the man-machine interaction interface of the terminal after selecting the voxel to be labeled, or can be automatically generated by the wearable device after obtaining the three-dimensional space model. The wearable device can pre-acquire a feature information corresponding table of common objects-space from the data storage system of the server, and compare the three-dimensional space model with the feature information corresponding table. As an example, the feature information corresponding table can include the corresponding relationship between the refrigerator, the pot, the faucet and other original objects and the kitchen space, the corresponding relationship between the wardrobe, the bed and other original objects and the bedroom space. The wearable device compares the feature information of the refrigerator, the pot, the faucet, the wardrobe, the bed and other original objects in the three-dimensional space model with the feature information corresponding table, and automatically labels. Then, according to the above corresponding relationship, the space where each original object is located is labeled.

[0106] The object label and the space label are used to identify at least one voxel in the three-dimensional space model. It should be noted that the same voxel can carry both the object label and the space label. As an example, when a plurality of voxels in the three-dimensional space model are labeled with the object label "refrigerator", these voxels can still be labeled with the space label "kitchen".

[0107] Further, in combination with the object label, the space label and the environment image information in the three-dimensional space model, the step of obtaining the real-time position information corresponding to the target object comprises:

[0108] Matching the environment image information with the three-dimensional space model to obtain the object label and the space label contained in the environment image information;

[0109] When the target object is contained in the environment image information, determining the object position information based on the space label corresponding to the space region where the target object is located and the object label of the adjacent original object of the target object.

[0110] The process of matching the environment image information with the three-dimensional space model involves the registration process of the pixel points in the two-dimensional image and the voxels in the three-dimensional model. The present application does not limit the registration process, which is not specifically described here.

[0111] When the target item is included in the environment image information, the wearable device can determine a space region where the target item is located based on the pixels around the target item, the voxels in the three-dimensional space model that match the pixels around the target item, and the space labels corresponding to the voxels, and determine the original item adjacent to the target item based on the item labels corresponding to the voxels, and take the space region where the target item is located and the original item adjacent to the target item as the item location information. As an example, when the voxels around the target item "eyeglasses" correspond to the space label "bedroom" and the voxels around the target item "eyeglasses" correspond to the item label "bedside table", the item location information can be "near the bedside table in the bedroom".

[0112] In some optional embodiments, the item finding method further includes:

[0113] When the environment image information does not include the target item, historical environment image information collected within a preset time period is obtained.

[0114] When the environment image information and the historical environment image information represent that the target item is stored in the original item, carrier location information is determined based on the item label of the original item.

[0115] There are two cases when the environment image information does not include the target item. One is that the target item is not in the space region corresponding to the environment image information, and the other is that the target item is in the space region corresponding to the environment image information and is placed inside the original item contained in the space region. In this case, historical environment image information collected within a preset time period can be obtained. If the historical environment image information and the environment image information represent that the target item is stored in the original item, the wearable device can take the item label corresponding to the original item storing the target item as the carrier location information.

[0116] Specifically, when the historical environment image information includes the target item and the environment image information does not include the target item, and the space label included in the historical environment image information meets the preset requirement with the space label included in the environment image information, the item label of the original item adjacent to the target item in the historical environment image information is taken as the carrier location information.

[0117] The preset requirement can be that the similarity between the spatial label included in the historical environment image information and the spatial label included in the environment image information reaches a preset similarity threshold. Through this setting, it can be determined that the view angle of the collection device has not changed, and the range of the space region that the collection device can collect has not changed. At this time, the target object disappears, which can be understood as the target object being stored in other original objects in the current space region range. The wearable device can determine the original object storing the target object based on the pixel point closest to the target object in the historical environment image information and the object label of the corresponding voxel, and take the original object storing the target object as the carrier position information. As an example, when the voxel around the target object "glasses" in the historical environment image corresponds to the original object "bedside table", the carrier position information can be "inside the bedside table".

[0118] It should be noted that when the target object has carrier position information, the object position information corresponding to the original object storing the target object can be directly stored as the position information of the target object.

[0119] Further, the wearable device can establish a connection between the original object "bedside table" and the target object "glasses". When the subsequently collected environment image information indicates that the original object "bedside table" has moved, the wearable device can obtain the object position information of the original object "bedside table" as the object position information of the target object "glasses".

[0120] It should be noted that when a target object has both object position information and carrier position information, and the object position information and the carrier position information are inconsistent, the real-time position information of the target object can be generated according to the party with the newer generation time, where the generation time can be understood as the collection time of the environment image information corresponding to the object position information and the carrier position information.

[0121] In this embodiment, important objects placed in other original objects can be screened out based on image processing, so that the position tracking of important objects is achieved by monitoring the original objects storing the important objects, and the positioning accuracy of important objects is improved.

[0122] In some optional embodiments, the real-time position information further includes real-time time information and real-time image information.

[0123] After step 202, the method further includes:

[0124] obtaining collection time information corresponding to the environment image information;

[0125] When the target item is included in the environment image information, the step of determining the item position information based on the space label corresponding to the space region where the target item is located and the item label of the original item adjacent to the target item further includes:

[0126] When the target item is included in the environment image information, the environment image information is taken as the real-time image information, and the collection time information corresponding to the environment image information is taken as the real-time time information.

[0127] Correspondingly, the step 206 includes: storing the target item in correspondence with the corresponding item position information, carrier position information, real-time time information and real-time image information.

[0128] The step 208 includes: when the search instruction for the target item is received, generating a navigation signal based on the item position information, carrier position information, real-time time information and real-time image information corresponding to the target item.

[0129] Further, when the wearable device stores the item position information, carrier position information, real-time time information and real-time image information of the target item, it can first determine whether the stored position already has historical real-time position information corresponding to the target item, and if so, delete the historical real-time position information to update the real-time position information of the target item.

[0130] In the present embodiment, the real-time position information not only includes the space region where the target item is located, the original item adjacent to the target item, and the original item storing the target item, but also includes the specific image of the environment image information and the real-time time information when the collection device collects the environment image information. Through this kind of setting, when the wearable device generates a navigation signal subsequently, it can not only provide accurate item placement position information, but also display the picture when the user places the target item and the specific time when the target item is placed to the user, so as to avoid the user forgetting the specific position of the target item and facilitate the user to quickly find the target item.

[0131] In some optional embodiments, the item searching method further includes:

[0132] When the environment image information fails to match the three-dimensional space model and the environment image information includes the target item, the preset position information is taken as the real-time position information of the target item.

[0133] The step 206 includes: storing the target item in correspondence with the preset position information.

[0134] The step 208 includes: generating a navigation signal based on the preset position information.

[0135] When the environmental image information fails to match the three-dimensional space model and the environmental image information contains the target item, it can be considered that the target item is moved to a place outside the target space environment. At this time, the wearable device cannot locate the specific position of the target item, and the preset position information is directly used as the real-time position information of the target item. The preset position information can be, for example, “outside the target space environment”.

[0136] As an example, the target space environment is the home of a user, and the user takes the target item “eyeglasses” out of the home. When the wearable device collects environmental image information outside the home and the environmental image information fails to match the three-dimensional space model and the target item “eyeglasses” appears in the environmental image information, the wearable device uses the preset position information “outside the home” as the real-time position information. When a find instruction for the target item “eyeglasses” is received, a navigation signal “eyeglasses are in a place outside the home” is generated based on the preset position information “outside the home”.

[0137] Alternatively, the wearable device can use the preset position information, the environmental image information, and the real-time time information of the environmental image information as the real-time position information of the target item. As an example, the wearable device uses the preset position information “outside the home”, the real-time time information “A year B month C time” of the environmental image information as the real-time position information. When a find instruction for the target item “eyeglasses” is received, a navigation signal “eyeglasses are in a place outside the home at A year B month C time” is generated based on the preset position information “outside the home”, the environmental image information, and the real-time time information “A year B month C time”, and the environmental image information is displayed.

[0138] The above-mentioned item finding method enables a user to perform real-time detection and analysis on an environment in which the user is located without being affected by daily activities, so as to find the actual position of an important item in the environment and store the actual position. The wearable device directly regards the actual position of an original item as the actual position of the important item, thereby avoiding inaccurate actual positions of the important item caused by the fact that the important item cannot be detected by a collection device after the important item is placed in other original items. The wearable device can also display the collected environmental image information and time information to the user when a find instruction for the important item is received, thereby facilitating the user to remember and quickly find the important item.

[0139] It should be understood that although the steps in the flowcharts involved in the above embodiments are shown in sequence according to the arrows, the steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, the execution of the steps is not strictly limited in sequence, and the steps can be executed in other orders. Moreover, at least some of the steps in the flowcharts involved in the above embodiments can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of the steps or stages is not necessarily sequential, but can be alternately executed with other steps or steps or stages in other steps.

[0140] Based on the same inventive concept, the embodiments of the present application also provide an article searching device for implementing the above-mentioned article searching method. The problem-solving implementation scheme provided by the article searching device is similar to the implementation scheme described in the above-mentioned article searching method, so the specific limitations in one or more device embodiments provided below can refer to the limitations of the article searching method in the above text, which will not be repeated here.

[0141] In one embodiment, as shown in FIG. 3, an article searching device 300 is provided, comprising:

[0142] The acquisition module 302 is configured to continuously acquire environmental image information in response to a marking instruction;

[0143] The acquisition module 304 is configured to acquire real-time position information corresponding to at least one target article based on the environmental image information; the real-time position information includes article position information of the target article and carrier position information of the target article;

[0144] The storage module 306 is configured to store the target article and the corresponding article position information and carrier position information;

[0145] The navigation module 308 is configured to generate a navigation signal based on the article position information and the carrier position information when a searching instruction for the target article is received.

[0146] In some optional embodiments, the acquisition module 302 is further configured to:

[0147] acquire article feature information;

[0148] based on the environmental image information, an article meeting the article feature information is taken as a target article, and real-time position information corresponding to the target article is acquired.

[0149] In some optional embodiments, the article feature information includes first article feature information and second article feature information;

[0150] The collection module 302 is further configured to:

[0151] In response to the marking instruction, acquire the first item characteristic information carried by the marking instruction;

[0152] In response to the collection instruction, acquire the second item characteristic information through the collection device arranged on the wearable device.

[0153] In some optional embodiments, the acquisition module 304 is further configured to:

[0154] Perform three-dimensional modeling on the target space environment to acquire a three-dimensional space model of the target space environment;

[0155] In response to the description instruction, create at least one item label and at least one space label in the three-dimensional space model; the item label is used to identify the original item in the three-dimensional space model, and the space label is used to identify different space regions in the three-dimensional space model;

[0156] Combine the item label, the space label, and the environment image information in the three-dimensional space model to acquire real-time position information corresponding to the target item.

[0157] In some optional embodiments, the acquisition module 304 is further configured to:

[0158] Match the environment image information with the three-dimensional space model to acquire the item label and the space label contained in the environment image information;

[0159] When the target item is contained in the environment image information, determine the item position information based on the space label corresponding to the space region where the target item is located and the item label of the original item adjacent to the target item.

[0160] In some optional embodiments, the acquisition module 304 is further configured to:

[0161] When the target item is not contained in the environment image information, acquire historical environment image information collected within a preset time period in the past;

[0162] When the environment image information and the historical environment image information indicate that the target item is stored in the original item, determine the carrier position information based on the item label corresponding to the original item.

[0163] In some optional embodiments, the acquisition module 304 is further configured to:

[0164] When the historical environment image information contains the target item, the environment image information does not contain the target item, and the space label contained in the historical environment image information meets a preset requirement with the space label contained in the environment image information, the item label of the original item adjacent to the target item in the historical environment image information is taken as the carrier position information.

[0165] In some optional embodiments, the real-time location information further comprises real-time time information and real-time image information.

[0166] The acquisition module 302 is further configured to:

[0167] acquire the acquisition time information corresponding to the environment image information;

[0168] The acquisition module 304 is further configured to: when the target item is contained in the environment image information, take the environment image information as the real-time image information, and take the acquisition time information corresponding to the environment image information as the real-time time information.

[0169] In some optional embodiments, the storage module 306 is further configured to:

[0170] store the target item and the corresponding item location information, carrier location information, real-time time information and real-time image information correspondingly;

[0171] The navigation module 308 is further configured to:

[0172] when receiving a search instruction for the target item, generate a navigation signal based on the item location information, carrier location information, real-time time information and real-time image information corresponding to the target item.

[0173] In some optional embodiments, the storage module 306 is further configured to:

[0174] delete the real-time location information corresponding to the target item stored in the past.

[0175] In some optional embodiments, the acquisition module 304 is further configured to:

[0176] when the environment image information fails to match the three-dimensional space model and the environment image information contains the target item, take the preset location information as the real-time location information of the target item;

[0177] The storage module 306 is further configured to:

[0178] store the target item and the preset location information correspondingly;

[0179] The navigation module 308 is further configured to:

[0180] generate a navigation signal based on the preset location information.

[0181] Each of the modules in the above apparatus can be implemented by software, hardware, and combinations thereof, in whole or in part. The above modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a memory in a computer device in software form, so as to be invoked and executed by a processor to perform the operations corresponding to the above modules.

[0182] The device subject provided by the embodiment can perform the article searching method provided by the method embodiments, and has similar implementation principles and technical effects, which will not be described here again.

[0183] FIG. 4 is a structural schematic diagram of a computer device provided by the present application. As shown in FIG. 4, the electronic device 400 provided by the embodiment includes at least one processor 401 and a memory 402. Optionally, the device 40 further includes a communication component 403. The processor 401, the memory 402, and the communication component 403 are connected through a bus 404.

[0184] In the specific implementation process, the at least one processor 401 executes the computer execution instructions stored in the memory 402, so that the at least one processor 401 performs the above method.

[0185] The specific implementation process of the processor 401 can refer to the method embodiments, which have similar implementation principles and technical effects, and will not be described here again.

[0186] In the above embodiments, it should be understood that the processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the application can be directly embodied as hardware processor execution, or executed by a combination of hardware and software modules in the processor.

[0187] The memory can include a random access memory (RAM), and can also include a non-volatile memory (NVM), for example, at least one disk memory.

[0188] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, the bus in the drawings of the present application does not limit to only one bus or one type of bus.

[0189] The present application also provides a computer program product, comprising a computer program, which, when executed by a processor, implements the above-mentioned article searching method.

[0190] The present application also provides a computer readable storage medium, which stores computer execution instructions, and when a processor executes the computer execution instructions, the above-mentioned article searching method is implemented.

[0191] The above-mentioned readable storage medium can be realized by any type of volatile or non-volatile storage device or their combination, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0192] An exemplary readable storage medium is coupled to the processor, so that the processor can read information from the readable storage medium, and can write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in the device.

[0193] The division of units is only a logical functional division, and in actual implementation, there can be another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0194] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e. may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0195] In addition, each functional unit in various embodiments of the application can be integrated into one processing unit, or each unit can exist physically, or two or more units can be integrated into one unit.

[0196] If the function is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the application essentially or the part of the prior art that contributes to the technical solutions or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the various embodiment methods of the application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0197] Those skilled in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware. The aforementioned program can be stored in a computer readable storage medium. The program executes the steps including the above-mentioned method embodiments when executed; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, and various media that can store program codes.

[0198] Finally, it should be noted that those skilled in the art, after considering the specification and practicing the application disclosed herein, will easily think of other embodiments of the application. The application is intended to cover any variations, uses or adaptations of the application that follow the general principles of the application and include common knowledge or conventional techniques in the art that are not disclosed by the application, and is not limited to the precise structure described above and shown in the drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the application is only limited by the appended claims.

Claims

1. An article search method characterized by comprising: The method is applied to a wearable device, and comprises the following steps: In response to a marking instruction, continuously collecting environmental image information; Based on the environmental image information, obtaining real-time position information corresponding to at least one target item; the real-time position information comprises item position information of the target item and carrier position information of the target item; Storing the target item, the corresponding item position information and the carrier position information; When a searching instruction for the target item is received, generating a navigation signal based on the item position information and the carrier position information.

2. The method of claim 1, wherein, Before the step of obtaining the real-time position information corresponding to the target item based on the environmental image information, the method further comprises the following steps: Obtaining item feature information; The step of obtaining the real-time position information corresponding to the target item based on the environmental image information comprises the following steps: Based on the environmental image information, taking an item meeting the item feature information as the target item, and obtaining real-time position information corresponding to the target item.

3. The method of claim 2, wherein, The item feature information comprises first item feature information and second item feature information; The step of obtaining the item feature information comprises the following steps: In response to the marking instruction, obtaining the first item feature information carried by the marking instruction; And in response to a collection instruction, obtaining the second item feature information by a collection device arranged on the wearable device.

4. The method of claim 1, wherein, The method further comprises the following steps: Three-dimensional modeling is performed on a target space environment to obtain a three-dimensional space model of the target space environment; In response to a description instruction, creating at least one item label and at least one space label in the three-dimensional space model; the item label is used to identify an original item in the three-dimensional space model, and the space label is used to identify different space regions in the three-dimensional space model; The step of obtaining the real-time position information corresponding to the target item based on the environmental image information comprises the following steps: Combining the item label, the space label in the three-dimensional space model and the environmental image information, obtaining the real-time position information corresponding to the target item.

5. The method of claim 4, wherein, The step of combining the item label, the space label in the three-dimensional space model and the environmental image information, obtaining the real-time position information corresponding to the target item comprises the following steps: Matching the environmental image information with the three-dimensional space model to obtain the item label and the space label contained in the environmental image information; When the target item is contained in the environmental image information, determining the item position information based on the space label corresponding to the space region where the target item is located and the item label of the original item adjacent to the target item.

6. The method of claim 5, wherein, The method further comprises the following steps: When the target item is not contained in the environmental image information, obtaining historical environmental image information collected in a past preset time period; When the environmental image information and the historical environmental image information indicate that the target item is stored in the original item, determining the carrier position information based on the item label corresponding to the original item.

7. The method of claim 6, wherein, The environment image information and the historical environment image information represent that the target object is stored in the original object, the object label of the corresponding original object is taken as the carrier position information. When the historical environment image information contains the target object, the environment image information does not contain the target object, and the space label contained in the historical environment image information meets the preset requirement with the space label contained in the environment image information, the object label of the original object adjacent to the target object in the historical environment image information is taken as the carrier position information.

8. The method of claim 5, wherein, The real-time position information further includes real-time time information and real-time image information; After continuously collecting the environment image information in response to the marking instruction, the method further includes: acquiring the acquisition time information corresponding to the environment image information; When the target object is contained in the environment image information, the object position information is determined based on the space label corresponding to the space region where the target object is located and the object label of the original object adjacent to the target object. When the target object is contained in the environment image information, the environment image information is taken as the real-time image information, and the acquisition time information corresponding to the environment image information is taken as the real-time time information.

9. The method of claim 8, wherein, The target object and the corresponding object position information and carrier position information are stored correspondingly, and the method further includes: The target object and the corresponding object position information, carrier position information, real-time time information and real-time image information are stored correspondingly. When the search instruction for the target object is received, the navigation signal is generated based on the object position information and the carrier position information, and the method further includes: When the search instruction for the target object is received, the navigation signal is generated based on the object position information, carrier position information, real-time time information and real-time image information corresponding to the target object.

10. The method of claim 9, wherein, The target object and the corresponding object position information, carrier position information, real-time time information and real-time image information are stored correspondingly, and the method further includes: The real-time position information corresponding to the target object stored in the past is deleted.

11. The method of claim 5, wherein, The method further includes: When the environment image information fails to match the three-dimensional space model and the environment image information contains the target object, the preset position information is taken as the real-time position information of the target object. The target object and the corresponding object position information and carrier position information are stored correspondingly, and the method further includes: The target object and the preset position information are stored correspondingly. The navigation signal is generated based on the object position information and the carrier position information, and the method further includes: The navigation signal is generated based on the preset position information.

12. An article search device characterized by comprising: The method includes: The acquisition module is configured to continuously collect environment image information in response to a marking instruction. The acquisition module is configured to acquire real-time position information corresponding to at least one target object based on the environment image information; the real-time position information includes object position information of the target object and carrier position information of the target object. The storage module is configured to store the target object and the corresponding object position information and carrier position information correspondingly. A navigation module configured to generate a navigation signal based on the item location information and the carrier location information when a search instruction for the target item is received.

13. A computer device, comprising: Comprising: a memory, a processor; the memory stores computer-executed instructions; the processor executes the computer-executed instructions stored in the memory, so that the processor executes the item searching method according to any one of claims 1-11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executed instructions, and the computer-executed instructions are executed by the processor to implement the item searching method according to any one of claims 1-11.

15. A computer program product, characterised in that, A computer program is executed by the processor to implement the item searching method according to any one of claims 1-11.

Images