Indoor area retrieval method and device, electronic equipment and storage medium

Abstract

The invention provides an indoor area retrieval method and device, electronic equipment and a storage medium, and relates to the field of artificial intelligence such as large models, deep learning, computer vision and cloud computing. The method comprises the steps that retrieval requirement description information is obtained, the retrieval requirement description information is description information about a to-be-retrieved target area, and the target area is an indoor area; the obtained candidate images are utilized to form a first image set, the number of the candidate images is larger than 1, the candidate images are shot images covering an indoor area of a target building, and the target building is a building serving as a retrieval object of current retrieval; and in response to a candidate image which is identified from the first image set and matched with the retrieval demand description information, taking the matched candidate image as a target image, and generating a retrieval result corresponding to the target area according to the target image and returning the retrieval result.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to indoor area retrieval methods, devices, electronic devices and storage media in the fields of large models, deep learning, computer vision and cloud computing. Background Art

[0002] Outdoor area search can often be achieved through pre-marked geographic coordinates. For example, if you want to find a park, you can directly enter the park name in the electronic map to search. However, indoor areas are usually more complex and there is no way to pre-mark geographic coordinates. Therefore, the current indoor area search is mainly achieved through communication between people, which is inefficient and difficult to guarantee accuracy. Summary of the invention

[0003] The present disclosure provides an indoor area search method, device, electronic device and storage medium.

[0004] An indoor area retrieval method, comprising:

[0005] Acquire retrieval requirement description information, wherein the retrieval requirement description information is description information about a target area to be retrieved, and the target area is an indoor area;

[0006] The first image set is formed by using the acquired candidate images, the number of the candidate images is greater than 1, and the candidate images are images captured and cover the indoor area of ​​the target building, and the target building is the building that is the search object of this search;

[0007] In response to identifying a candidate image matching the retrieval requirement description information from the first image set, the matching candidate image is used as a target image, and a retrieval result corresponding to the target area is generated and returned based on the target image.

[0008] An indoor area retrieval method, comprising:

[0009] Acquire retrieval requirement description information input by a user, wherein the retrieval requirement description information is description information about a target area to be retrieved, and the target area is an indoor area;

[0010] Sending the search requirement description information to the cloud service;

[0011] In response to obtaining the retrieval result returned by the cloud service, the retrieval result is displayed to the user, or the retrieval result is displayed to the user after a predetermined conversion, the retrieval result is a retrieval result corresponding to the target area generated by the cloud service according to the target image, the target image is a candidate image identified by the cloud service from a first image set that matches the retrieval requirement description information, the first image set is composed of candidate images, and the candidate images in the first image set are images captured covering the indoor area of ​​the target building, and the target building is a building that is the retrieval object of this retrieval.

[0012] An indoor area retrieval device comprises: a first acquisition module, a second acquisition module and a retrieval processing module;

[0013] The first acquisition module is used to acquire retrieval requirement description information, wherein the retrieval requirement description information is description information about a target area to be retrieved, and the target area is an indoor area;

[0014] The second acquisition module is used to form a first image set using the acquired candidate images, the number of the candidate images is greater than 1, and the candidate images are images captured and cover the indoor area of ​​the target building, and the target building is a building that is a search object of this search;

[0015] The retrieval processing module is used to respond to identifying a candidate image matching the retrieval requirement description information from the first image set, taking the matching candidate image as a target image, and generating a retrieval result corresponding to the target area based on the target image and returning it.

[0016] An indoor area search device includes: a third acquisition module, an information sending module and a result processing module;

[0017] The third acquisition module is used to acquire the search requirement description information input by the user, wherein the search requirement description information is description information about the target area to be searched, and the target area is an indoor area;

[0018] The information sending module is used to send the search requirement description information to the cloud service;

[0019] The result processing module is used to display the search result to the user in response to obtaining the search result returned by the cloud service, or to display the search result to the user after performing a predetermined conversion on the search result, wherein the search result is a search result corresponding to the target area generated by the cloud service according to the target image, and the target image is a candidate image matched with the search requirement description information identified by the cloud service from the first image set, the first image set is composed of candidate images, and the candidate images in the first image set are images captured covering the indoor area of ​​the target building, and the target building is a building that is the search object of this search.

[0020] An electronic device, comprising:

[0021] at least one processor; and

[0022] a memory communicatively connected to the at least one processor; wherein,

[0023] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method described above.

[0024] A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the method as described above.

[0025] A computer program product comprises a computer program / instruction, wherein the computer program / instruction implements the method described above when executed by a processor.

[0026] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0028] Figure 1 This is a flow chart of the first embodiment of the indoor area retrieval method disclosed in the present invention;

[0029] Figure 2 This is a flow chart of a second embodiment of the indoor area retrieval method disclosed in the present invention;

[0030] Figure 3 Schematic diagram of the composition structure of the first embodiment 300 of the indoor area search device of the present disclosure;

[0031] Figure 4It is a schematic diagram of the structure of the second embodiment 400 of the indoor area search device of the present disclosure;

[0032] Figure 5 A schematic block diagram of an electronic device 500 that can be used to implement an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0033] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0034] In addition, it should be understood that the term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0035] Figure 1 FIG. 1 is a flow chart of the first embodiment of the indoor area search method disclosed in the present invention. Figure 1 As shown, the following specific implementation methods are included.

[0036] In step 101, retrieval requirement description information is obtained, where the retrieval requirement description information is description information about a target area to be retrieved, and the target area is an indoor area.

[0037] In step 102, the first image set is formed using the acquired candidate images, the number of candidate images is greater than 1, and the candidate images are images captured covering the indoor area of ​​the target building, and the target building is the building that is the retrieval object of this retrieval.

[0038] In step 103, in response to identifying a candidate image matching the retrieval requirement description information from the first image set, the matching candidate image is used as a target image, and a retrieval result corresponding to the target area is generated and returned based on the target image.

[0039] By adopting the scheme described in the above method embodiment, based on the retrieval requirement description information and candidate images, through matching and recognition operations, the retrieval results (i.e., search results) of the target area to be retrieved (i.e., to be searched) can be automatically generated, and the target area is an indoor area, thereby simplifying manual operations, improving processing efficiency, and improving the accuracy of processing results.

[0040] In practical applications, Figure 1 The execution subject of the illustrated embodiment may be a cloud service, that is, a background service. The cloud service may obtain the retrieval requirement description information from the smart terminal and may obtain the candidate images captured by the camera.

[0041] For example, if a user is in a shopping mall (target building) and wants to find a specific restaurant, or wants to find restaurants in the shopping mall in general, he can enter the search requirement description information into the smart terminal, such as "Help me find ** restaurant". For another example, if a user is in a supermarket (target building) and wants to find the fresh food section, he can enter the search requirement description information into the smart terminal, such as "Help me find the location of the fresh food section".

[0042] In addition, at least one camera may be provided on each floor of the target building. The specific number may be determined according to actual needs. Each camera may collect candidate images. Generally speaking, the candidate images collected by the cameras on each floor need to be able to cover all indoor areas of the target building.

[0043] In some embodiments of the present disclosure, the candidate images constituting the first image set may be the candidate images acquired most recently, wherein the candidate images may be acquired again after every predetermined period of time.

[0044] That is to say, each camera can re-capture an image at a predetermined interval and send it to the backend service. The specific value of the predetermined interval can also be determined according to actual needs. Taking the target building as a shopping mall as an example, considering that the store layout in the shopping mall may change, candidate images can be collected periodically, and the latest candidate images can be used to form the first image set to improve the accuracy of subsequent processing results.

[0045] For the first image set, candidate images matching the search requirement description information can be identified, and the identified matching candidate images can be used as target images. In actual applications, target images matching the search requirement description information may not be identified, such as when a user wants to search for a fast food restaurant of a certain brand, but there is no fast food restaurant of that brand in the target building. In this case, there is no restriction on how to handle it. For example, a prompt message of "no relevant results found" can be directly returned to the smart terminal so that the smart terminal can display it to the user.

[0046] In some embodiments of the present disclosure, in response to determining that the retrieval requirement description information is in text form, keywords can be extracted from the retrieval requirement description, and a target image can be recognized based on the keywords. In response to determining that the retrieval requirement description information is in voice form, the retrieval requirement description information can be converted from voice to text, and keywords can be extracted from the conversion result, and then the target image can be recognized based on the keywords.

[0047] In other words, it can support users to input the search requirement description information in text form, and it can also support users to input the search requirement description information in voice form, which is very flexible and convenient, and can meet the different usage needs of users.

[0048] If it is in text form, keywords can be directly extracted from the retrieval requirement description information. If it is in voice form, the retrieval requirement description information can first be converted from voice to text to obtain the required conversion result, and then keywords can be extracted from the conversion result.

[0049] The extracted keywords may include the name and features of the target area, etc. For example, if the search requirement description information is "help me find the location of the fresh food area", then the extracted keywords may be "fresh food area" (the name of the target area). For another example, if the search requirement description information is "help me find a hot pot restaurant in this mall", then the extracted keywords may be "hot pot" and "restaurant".

[0050] In addition, there is no restriction on how to extract keywords. For example, the extraction can be performed based on rules, or the extraction can be performed using a pre-trained extraction model.

[0051] After the keywords are acquired, the target image can be identified according to the keywords. In some embodiments of the present disclosure, the following processing can be performed for each candidate image: in response to identifying that the candidate image includes the target area according to the keywords, the candidate image is added to the second image set, and in response to determining that the second image set is not empty, the candidate image in the second image set is determined as the target image.

[0052] For example, suppose that the first image set includes 30 candidate images, which are referred to as candidate images 1 to 30 for ease of description. For candidate image 1, if it is identified that the target area is included in candidate image 1 according to the keyword, then candidate image 1 can be added to the second image set. If it is identified that the target area is not included in candidate image 1 according to the keyword, then candidate image 1 will not be added to the second image set. Similarly, for candidate image 2, if it is identified that the target area is included in candidate image 2 according to the keyword, then candidate image 2 can also be added to the second image set. If it is identified that the target area is not included in candidate image 2 according to the keyword, then candidate image 2 will not be added to the second image set. The processing methods for other candidate images are similar and will not be repeated. Assuming that the second image set is not empty after all candidate images have been processed, the candidate images in the second image set can be determined as target images.

[0053] By adopting the above processing method, all target images can be identified from the first image set, thereby improving the comprehensiveness and accuracy of the recognition result.

[0054] Alternatively, in some embodiments of the present disclosure, when target image recognition is performed based on keywords, a statistical parameter may be set with an initial value of 0, and each candidate image may be traversed in sequence according to a predetermined order, and the following processing is performed for each traversed candidate image: in response to recognizing that the candidate image includes a target area based on the keyword, the candidate image is added to the third image set, and the value of the statistical parameter is increased by 1; in response to determining that the latest value of the statistical parameter is equal to N, the traversal is terminated, and the candidate image in the third image set is determined as the target image, where N is a positive integer; in response to determining that the latest value of the statistical parameter is equal to The value is less than N, and it is determined that there are candidate images that have not been traversed, continue to traverse the next candidate image, in response to determining that the latest value of the statistical parameter is less than N, and it is determined that all candidate images have been traversed, the candidate image in the third image set is determined as the target image; in response to identifying that the candidate image does not include the target area according to the keyword, and it is determined that there are candidate images that have not been traversed, continue to traverse the next candidate image, in response to identifying that the candidate image does not include the target area according to the keyword, and it is determined that all candidate images have been traversed, the candidate image in the third image set is determined as the target image.

[0055] The specific value of N can be determined according to actual needs, and N can represent an upper limit set by the user. In addition, the specific order of the predetermined order can also be determined according to actual needs.

[0056] Assume that the first image set includes 30 candidate images, which are referred to as candidate image 1 to candidate image 30 for ease of description, and assume that they are traversed in the order of candidate image 1 to candidate image 30. Taking candidate image 15 as an example, if it is identified that the target area is included in candidate image 15 based on the keyword, then candidate image 15 can be added to the third image set, and the value of the statistical parameter can be increased by 1. If it is determined that the value of the updated statistical parameter is equal to N, candidate images 16 to 30 may not be traversed, but the candidate image in the third image set may be directly determined as the target image. If it is determined that the value of the updated statistical parameter is less than N, then candidate image 16 may continue to be traversed. Alternatively, if it is identified that the target area is not included in candidate image 15 based on the keyword, candidate image 16 may continue to be traversed. If after all 30 candidate images are traversed, the value of the statistical parameter is less than or equal to N, then the candidate image in the third image set may also be determined as the target image.

[0057] Using the above processing method, once it is determined that the number of identified candidate images including the target area reaches N, the traversal can be terminated and the candidate image in the current third image set can be directly determined as the target image, thereby eliminating the need to identify all candidate images, thereby improving processing efficiency and avoiding returning too much information, affecting the user's information browsing efficiency, etc.

[0058] For any candidate image, the large model can be used to identify whether it includes the target area. In some embodiments of the present disclosure, a prompt word of the large model can be first constructed based on the keyword, and then the candidate image and the prompt word can be used as input of the large model. In response to obtaining the first output result of the large model, it can be determined that the candidate image includes the target area, and in response to obtaining the second output result of the large model, it can be determined that the candidate image does not include the target area.

[0059] There is no restriction on how to construct the prompt words. For example, assuming that the extracted keywords include "hot pot" and "restaurant", the constructed prompt words can be "You are ***, please determine whether the picture includes a hot pot restaurant, ****", etc. Accordingly, the large model can generate a first output result indicating "yes" or a second output result indicating "no" based on the input candidate image and prompt words.

[0060] The big model refers to the Large Language Model (LLM), which is a deep learning model trained using a large amount of text data. It can generate natural language text or understand the meaning of language text, etc. It is an important path to artificial intelligence.

[0061] Considering that the large model can understand images and thus objects in geographic space, the solution described in the present disclosure uses the large model to identify whether the candidate image includes the target area, thereby improving the accuracy of the recognition result. In addition, the solution described in the present disclosure does not limit the type of the large model, as long as it can complete the corresponding function.

[0062] After the target image is determined, a search result corresponding to the target area can be generated according to the target image, and the search result can be returned to the smart terminal.

[0063] In some embodiments of the present disclosure, the first image set may also include: a camera identifier (ID) of a camera corresponding to each candidate image, and the camera corresponding to any candidate image is the camera used to capture the candidate image. Accordingly, a method for returning a search result corresponding to a target area generated according to a target image may include: determining the location information corresponding to the target image as the target location information by querying the correspondence between each pre-saved camera identifier and the corresponding location information, and returning the target location information as a search result, wherein any location information includes: the floor where the corresponding camera is located and the coordinate information of the corresponding camera on the floor where it is located.

[0064] Each camera can send its own camera ID at the same time when sending candidate images to the backend service. In addition, the cloud service can also pre-store the location information corresponding to different camera IDs. For any camera, the corresponding location information may include the floor where the camera is located, such as the 2nd floor, and the coordinate information of the camera on the floor where it is located. The coordinate information may refer to the specific horizontal and vertical coordinates in the building.

[0065] For example, assuming that the target image only includes candidate image 15, the location information corresponding to the camera corresponding to candidate image 15 can be determined by querying the corresponding relationship, and the location information can be returned to the smart terminal as a retrieval result, so that the smart terminal can display the retrieval result or the retrieval result after a predetermined conversion to the user, so that the user can know the location of the target area, etc.

[0066] In addition, in actual applications, cloud services can be further subdivided into access services, voice conversion services, extraction services, retrieval services, and database (Mongo) services based on distributed file storage, etc. Among them, the access service can obtain retrieval requirement description information from the smart terminal and obtain candidate images from the camera, and can return the retrieval results to the smart terminal. In addition, other services can be called, such as calling the voice conversion service to convert the voice-based retrieval requirement description information into text form, calling the extraction service to extract keywords from the text-based retrieval requirement description information, calling the retrieval service to identify the target image from the first image set based on the keywords and return the retrieval results generated based on the target image, calling the Mongo service to query the correspondence between each camera identifier and the corresponding location information, etc. Among them, the access service, voice conversion service, extraction service, and retrieval service can all be implemented using serverless technology.

[0067] Furthermore, the search requirement description information obtained by the cloud service may not come from the user, but from smart devices such as delivery robots. In this case, the search requirement description information is usually in text form and may not be sent through smart terminals but through Hypertext Transfer Protocol (HTTP) to the cloud service. After generating the search results, the cloud service can return the search results to the smart device.

[0068] The above mainly describes the solution of the present disclosure from the cloud service side, and the following further describes the solution of the present disclosure from the smart terminal side.

[0069] Figure 2 FIG. 1 is a flow chart of the second embodiment of the indoor area search method described in the present disclosure. Figure 2 As shown, the following specific implementation methods are included.

[0070] In step 201, the search requirement description information input by the user is obtained, where the search requirement description information is description information about a target area to be searched, and the target area is an indoor area.

[0071] In step 202, the search requirement description information is sent to the cloud service.

[0072] In step 203, in response to obtaining the retrieval result returned by the cloud service, the retrieval result is displayed to the user, or the retrieval result is displayed to the user after a predetermined conversion. The retrieval result is a retrieval result corresponding to the target area generated by the cloud service according to the target image. The target image is a candidate image that matches the retrieval requirement description information and is identified by the cloud service from the first image set. The first image set is composed of candidate images, and the candidate images in the first image set are images captured covering the indoor area of ​​the target building. The target building is a building that is the retrieval object of this retrieval.

[0073] By adopting the scheme described in the above method embodiment, the retrieval results of the target area to be searched can be automatically generated based on the retrieval requirement description information and candidate images through matching and recognition operations, and the target area is an indoor area, thereby simplifying manual operations, improving processing efficiency, and improving the accuracy of processing results.

[0074] In practical applications, Figure 2 The execution subject of the illustrated embodiment may be a smart terminal, which may obtain the search requirement description information input by the user and send it to the cloud service.

[0075] The smart terminal can also obtain the search results generated and returned by the cloud service. In response to identifying a candidate image matching the search requirement description information from the first image set, the cloud service can use the matching candidate image as the target image and generate a search result corresponding to the target area based on the target image and return it.

[0076] In some embodiments of the present disclosure, the first image set may also include: camera identifiers of cameras corresponding to respective candidate images, the cameras corresponding to any candidate image are cameras used to capture the candidate images, and at least one camera is provided on each floor of the target building. Accordingly, the retrieval results may include: the cloud service determines the location information corresponding to the target image by querying the correspondence between the pre-saved camera identifiers and the corresponding location information, wherein any location information includes: the floor where the corresponding camera is located and the coordinate information of the corresponding camera on the floor where it is located.

[0077] Furthermore, the intelligent terminal can directly display the search results to the user, or can display the search results to the user after a predetermined conversion. For example, assuming that the target image only includes candidate image 15, the location information corresponding to the camera corresponding to candidate image 15 can be directly displayed to the user, or the location information corresponding to the camera corresponding to candidate image 15 can be first converted into a form that is easier for the user to understand, and then the conversion result can be displayed to the user. For example, if the location information corresponding to the camera corresponding to candidate image 15 is 2nd floor + coordinate information, the coordinate information can be converted into a form that is easier for the user to understand, such as southwest corner, north middle, etc., so as to facilitate user use.

[0078] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present disclosure is not limited by the described order of actions, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present disclosure. In addition, for parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0079] The above is an introduction to the method embodiment. The following is a further explanation of the scheme disclosed in the present invention through an apparatus embodiment.

[0080] Figure 3 FIG. 3 is a schematic diagram of the composition structure of the first embodiment 300 of the indoor area search device described in the present disclosure. Figure 3 As shown, it includes: a first acquisition module 301, a second acquisition module 302 and a retrieval processing module 303.

[0081] The first acquisition module 301 is used to acquire retrieval requirement description information, where the retrieval requirement description information is description information about a target area to be retrieved, and the target area is an indoor area.

[0082] The second acquisition module 302 is used to use the acquired candidate images to form a first image set, the number of candidate images is greater than 1, and the candidate images are images captured covering the indoor area of ​​the target building, and the target building is the building that serves as the retrieval object of this retrieval.

[0083] The retrieval processing module 303 is used to respond to identifying a candidate image matching the retrieval requirement description information from the first image set, taking the matching candidate image as a target image, and generating a retrieval result corresponding to the target area based on the target image and returning it.

[0084] In some embodiments of the present disclosure, the candidate images constituting the first image set may be the candidate images acquired most recently, wherein the second acquisition module 302 may acquire the candidate images again after a predetermined period of time.

[0085] For the first image set, the retrieval processing module 303 can identify candidate images that match the retrieval requirement description information, and can use the identified matching candidate images as target images. In some embodiments of the present disclosure, in response to determining that the retrieval requirement description information is in text form, the retrieval processing module 303 can extract keywords from the retrieval requirement description, and can identify the target image based on the keywords. In response to determining that the retrieval requirement description information is in voice form, the retrieval requirement description information can be converted from voice to text, and keywords can be extracted from the conversion result, and then the target image can be identified based on the keywords.

[0086] In some embodiments of the present disclosure, the retrieval processing module 303 may perform the following processing for each candidate image: in response to identifying that the candidate image includes a target area based on keywords, the candidate image is added to the second image set; in response to determining that the second image set is not empty, the candidate image in the second image set is determined as the target image.

[0087] Alternatively, in some embodiments of the present disclosure, the retrieval processing module 303 may set a statistical parameter with an initial value of 0, and may sequentially traverse each candidate image in a predetermined order, and perform the following processing for each candidate image traversed: in response to identifying that the candidate image includes a target area according to the keyword, the candidate image is added to the third image set, and the value of the statistical parameter is increased by 1; in response to determining that the latest value of the statistical parameter is equal to N, the traversal is terminated, and the candidate image in the third image set is determined as the target image, N is a positive integer; in response to determining that the latest value of the statistical parameter is less than N, and it is determined that there are candidate images that have not been traversed, continue to traverse the next candidate image, in response to determining that the latest value of the statistical parameter is less than N, and it is determined that all candidate images have been traversed, the candidate image in the third image set is determined as the target image; in response to identifying that the candidate image does not include the target area according to the keyword, and it is determined that there are candidate images that have not been traversed, continue to traverse the next candidate image, in response to identifying that the candidate image does not include the target area according to the keyword, and it is determined that all candidate images have been traversed, the candidate image in the third image set is determined as the target image.

[0088] Among them, for any candidate image, the retrieval processing module 303 can use the large model to identify whether the target area is included therein. In some embodiments of the present disclosure, the retrieval processing module 303 can first construct a prompt word of the large model based on the keyword, and then use the candidate image and the prompt word as input of the large model. In response to obtaining the first output result of the large model, it can be determined that the candidate image includes the target area, and in response to obtaining the second output result of the large model, it can be determined that the candidate image does not include the target area.

[0089] In some embodiments of the present disclosure, the first image set may further include: camera identifiers of cameras corresponding to each candidate image, and the camera corresponding to any candidate image is the camera used to capture the candidate image. Accordingly, the retrieval processing module 303 generates a retrieval result corresponding to the target area according to the target image and returns the retrieval result in a manner that may include: determining the location information corresponding to the target image as the target location information by querying the correspondence between each pre-saved camera identifier and the corresponding location information, and returning the target location information as the retrieval result, wherein any location information includes: the floor where the corresponding camera is located and the coordinate information of the corresponding camera on the floor where it is located.

[0090] Figure 4 FIG. 4 is a schematic diagram of the structure of the second embodiment 400 of the indoor area search device described in the present disclosure. Figure 4 As shown, it includes: a third acquisition module 401, an information sending module 402 and a result processing module 403.

[0091] The third acquisition module 401 is used to acquire the search requirement description information input by the user, where the search requirement description information is description information about the target area to be searched, and the target area is an indoor area.

[0092] The information sending module 402 is used to send the search requirement description information to the cloud service.

[0093] The result processing module 403 is used to display the search results to the user in response to obtaining the search results returned by the cloud service, or to display the search results to the user after performing a predetermined conversion. The search results are search results corresponding to the target area generated by the cloud service according to the target image. The target image is a candidate image that matches the search requirement description information and is identified by the cloud service from the first image set. The first image set is composed of candidate images, and the candidate images in the first image set are images captured and cover the indoor area of ​​the target building. The target building is a building that is the search object of this search.

[0094] In some embodiments of the present disclosure, the first image set may also include: camera identifiers of cameras corresponding to respective candidate images, the cameras corresponding to any candidate image are cameras used to capture the candidate images, and at least one camera is provided on each floor of the target building. Accordingly, the retrieval results may include: the cloud service determines the location information corresponding to the target image by querying the correspondence between the pre-saved camera identifiers and the corresponding location information, wherein any location information includes: the floor where the corresponding camera is located and the coordinate information of the corresponding camera on the floor where it is located.

[0095] The specific working processes of the above-mentioned device embodiments can refer to the relevant descriptions in the above-mentioned method embodiments and will not be repeated here.

[0096] The solution disclosed in the present disclosure can be applied to the field of artificial intelligence, especially to the fields of large models, deep learning, computer vision and cloud computing. Artificial intelligence is a discipline that studies how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.). It has both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing. Artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, as well as machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.

[0097] In addition, the search requirement description information and images in the embodiments of the present disclosure are not for a specific user and do not reflect the personal information of a specific user. In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0098] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0099] Figure 5 A schematic block diagram of an electronic device 500 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0100] like Figure 5 As shown, the device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 to a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0101] A number of components in the device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0102] The computing unit 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI, Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSP, Digital Signal Processing), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 501 performs the various methods and processes described above, such as the methods described in the present disclosure. For example, in some embodiments, the methods described in the present disclosure may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the methods described in the present disclosure may be executed. Alternatively, in other embodiments, the computing unit 501 may be configured to execute the method described in the present disclosure in any other appropriate manner (for example, by means of firmware).

[0103] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard parts (ASSPs), system on chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs, which can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a dedicated or general programmable processor, which can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0104] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0105] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory (EPROM, Electronically Programmable Read-Only Memory), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM, Compact Disc Read-Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0106] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tube (CRT) or a liquid crystal display (LCD) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0107] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0108] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0109] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0110] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for indoor area retrieval, comprising: Acquire retrieval requirement description information, wherein the retrieval requirement description information is description information about a target area to be retrieved, and the target area is an indoor area; The first image set is formed by using the acquired candidate images, the number of the candidate images is greater than 1, and the candidate images are images captured and cover the indoor area of ​​the target building, and the target building is the building that is the search object of this search; In response to identifying a candidate image matching the retrieval requirement description information from the first image set, the matching candidate image is used as a target image, and a retrieval result corresponding to the target area is generated and returned based on the target image.

2. The method according to claim 1, wherein: The candidate images include: the candidate images acquired most recently; wherein the candidate images are acquired again after every predetermined time period.

3. The method according to claim 1, wherein: In response to identifying a candidate image matching the retrieval requirement description information from the first image set, taking the matching candidate image as a target image comprises: In response to determining that the retrieval requirement description information is in text form, extracting keywords from the retrieval requirement description, and identifying the target image according to the keywords; In response to determining that the retrieval requirement description information is in voice form, the retrieval requirement description information is converted from voice to text, keywords are extracted from the conversion result, and the target image is recognized based on the keywords.

4. The method according to claim 3, wherein: The identifying of the target image according to the keyword includes: For each candidate image, the following processing is performed respectively: in response to identifying that the candidate image includes the target area according to the keyword, adding the candidate image to the second image set; In response to determining that the second image set is not empty, a candidate image in the second image set is determined as the target image.

5. The method according to claim 3, wherein: The identifying of the target image according to the keyword includes: Set the statistical parameters, with the initial value of 0, and traverse each candidate image in a predetermined order. For each candidate image traversed, perform the following processing: In response to identifying that the candidate image includes the target area according to the keyword, adding the candidate image to the third image set and adding 1 to the value of the statistical parameter; in response to determining that the latest value of the statistical parameter is equal to N, ending the traversal and determining the candidate image in the third image set as the target image, where N is a positive integer; in response to determining that the latest value of the statistical parameter is less than N and that there is a candidate image that has not been traversed, continuing to traverse the next candidate image; in response to determining that the latest value of the statistical parameter is less than N and that all candidate images have been traversed, determining the candidate image in the third image set as the target image; In response to identifying that the target area is not included in the candidate image according to the keyword, and determining that there are candidate images that have not been traversed, continue to traverse the next candidate image; in response to identifying that the target area is not included in the candidate image according to the keyword, and determining that all candidate images have been traversed, determine the candidate image in the third image set as the target image.

6. The method according to claim 4 or 5, wherein: The step of identifying that the target area is included in the candidate image according to the keyword comprises: constructing a prompt word of a large model according to the keyword, taking the candidate image and the prompt word as inputs of the large model, and determining that the target area is included in the candidate image in response to obtaining a first output result of the large model; The step of identifying that the target area is not included in the candidate image based on the keyword includes: constructing the prompt word based on the keyword, taking the candidate image and the prompt word as input of the large model, and determining that the target area is not included in the candidate image in response to obtaining a second output result of the large model.

7. The method according to any one of claims 1 to 5, wherein: The first image set also includes: camera identifiers of cameras corresponding to the candidate images, the cameras corresponding to any candidate image are cameras used to capture the candidate images, and at least one camera is provided on each floor of the target building; The method of returning the retrieval result corresponding to the target area generated according to the target image includes: determining the location information corresponding to the target image as the target location information by querying the correspondence between each pre-saved camera identification and the corresponding location information, and returning the target location information as the retrieval result, wherein any location information includes: the floor where the corresponding camera is located and the coordinate information of the corresponding camera on the floor where it is located.

8. An indoor area retrieval method, comprising: Acquire retrieval requirement description information input by a user, wherein the retrieval requirement description information is description information about a target area to be retrieved, and the target area is an indoor area; Sending the search requirement description information to the cloud service; In response to obtaining the retrieval result returned by the cloud service, the retrieval result is displayed to the user, or the retrieval result is displayed to the user after a predetermined conversion, the retrieval result is a retrieval result corresponding to the target area generated by the cloud service according to the target image, the target image is a candidate image identified by the cloud service from a first image set that matches the retrieval requirement description information, the first image set is composed of candidate images, and the candidate images in the first image set are images captured covering the indoor area of ​​the target building, and the target building is a building that is the retrieval object of this retrieval.

9. The method according to claim 8, wherein: The first image set also includes: camera identifiers of cameras corresponding to the candidate images, the cameras corresponding to any candidate image are cameras used to capture the candidate images, and at least one camera is provided on each floor of the target building; The retrieval result includes: the cloud service determines the location information corresponding to the target image by querying the correspondence between each pre-saved camera identification and the corresponding location information, wherein any location information includes: the floor where the corresponding camera is located and the coordinate information of the corresponding camera on the floor.

10. An indoor area search device, comprising: A first acquisition module, a second acquisition module and a retrieval processing module; The first acquisition module is used to acquire retrieval requirement description information, wherein the retrieval requirement description information is description information about a target area to be retrieved, and the target area is an indoor area; The second acquisition module is used to form a first image set using the acquired candidate images, the number of the candidate images is greater than 1, and the candidate images are images captured and cover the indoor area of ​​the target building, and the target building is a building that is a search object of this search; The retrieval processing module is used to respond to identifying a candidate image matching the retrieval requirement description information from the first image set, taking the matching candidate image as a target image, and generating a retrieval result corresponding to the target area based on the target image and returning it.

11. The device according to claim 10, wherein: The candidate images include: the candidate image acquired most recently; The second acquisition module is further configured to reacquire the candidate image every time a predetermined time period has passed.

12. The device according to claim 10, wherein: In response to determining that the retrieval requirement description information is in text form, the retrieval processing module extracts keywords from the retrieval requirement description and identifies the target image based on the keywords. In response to determining that the retrieval requirement description information is in voice form, the retrieval requirement description information is converted from voice to text, and keywords are extracted from the conversion result, and the target image is identified based on the keywords.

13. The device according to claim 12, wherein: The retrieval processing module performs the following processing for each candidate image: in response to identifying that the candidate image includes the target area based on the keyword, the candidate image is added to a second image set; in response to determining that the second image set is not empty, the candidate image in the second image set is determined as the target image.

14. The device according to claim 12, wherein: The retrieval processing module sets a statistical parameter with an initial value of 0, and traverses each candidate image in turn according to a predetermined order, and performs the following processing for each candidate image traversed each time: in response to identifying that the candidate image includes the target area according to the keyword, the candidate image is added to the third image set, and the value of the statistical parameter is increased by 1; in response to determining that the latest value of the statistical parameter is equal to N, the traversal is terminated, and the candidate image in the third image set is determined as the target image, N is a positive integer; in response to determining that the latest value of the statistical parameter is less than N and that there is a candidate image that has not been traversed, the traversal continues to the next candidate image; in response to determining that the latest value of the statistical parameter is less than N and that all candidate images have been traversed, the candidate image in the third image set is determined as the target image; In response to identifying that the target area is not included in the candidate image according to the keyword and determining that there are candidate images that have not been traversed, continue to traverse the next candidate image; in response to identifying that the target area is not included in the candidate image according to the keyword and determining that all candidate images have been traversed, determine the candidate image in the third image set as the target image.

15. The device according to claim 13 or 14, wherein: The retrieval processing module constructs a prompt word of the large model based on the keywords, takes the candidate image and the prompt word as input of the large model, and determines that the candidate image includes the target area in response to obtaining a first output result of the large model, and determines that the candidate image does not include the target area in response to obtaining a second output result of the large model.

16. The device according to any one of claims 10 to 14, wherein: The first image set also includes: camera identifiers of cameras corresponding to the candidate images, the cameras corresponding to any candidate image are cameras used to capture the candidate images, and at least one camera is provided on each floor of the target building; The retrieval processing module determines the location information corresponding to the target image as the target location information by querying the correspondence between each pre-saved camera identification and the corresponding location information, and returns the target location information as the retrieval result, wherein any location information includes: the floor where the corresponding camera is located and the coordinate information of the corresponding camera on the floor where it is located.

17. An indoor area search device, comprising: A third acquisition module, an information sending module and a result processing module; The third acquisition module is used to acquire the search requirement description information input by the user, wherein the search requirement description information is description information about the target area to be searched, and the target area is an indoor area; The information sending module is used to send the search requirement description information to the cloud service; The result processing module is used to display the search result to the user in response to obtaining the search result returned by the cloud service, or to display the search result to the user after performing a predetermined conversion on the search result, wherein the search result is a search result corresponding to the target area generated by the cloud service according to the target image, and the target image is a candidate image matched with the search requirement description information identified by the cloud service from the first image set, the first image set is composed of candidate images, and the candidate images in the first image set are images captured covering the indoor area of ​​the target building, and the target building is a building that is the search object of this search.

18. The device according to claim 17, wherein: The first image set also includes: camera identifiers of cameras corresponding to the candidate images, the cameras corresponding to any candidate image are cameras used to capture the candidate images, and at least one camera is provided on each floor of the target building; The retrieval result includes: the cloud service determines the location information corresponding to the target image by querying the correspondence between each pre-saved camera identification and the corresponding location information, wherein any location information includes: the floor where the corresponding camera is located and the coordinate information of the corresponding camera on the floor.

19. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

20. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to make a computer execute the method according to any one of claims 1 to 9.

21. A computer program product, comprising a computer program / instruction, wherein when the computer program / instruction is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images