Augmented reality target display method and device, equipment and storage medium

Abstract

The application discloses a target display method and device of augmented reality, equipment and storage medium, relates to the field of AR technology, and provides a self-defined search result mode. When the first search result displayed in the AR display interface does not meet the requirement, the display operation can be updated to self-define the information to be displayed, the self-defined search is performed, the problem that only fixed and single search requirements can be met in the related art is solved, the problem of complicated operation caused by multiple switching of APPs is avoided, the search efficiency is improved, meanwhile, the related information of the target object is directly superimposed and displayed in the AR display interface, so that the information display is more intuitive and efficient. The method can be applied to the fields of AR search and maps.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more particularly to the field of augmented reality (AR) technology, providing an augmented reality target display method, apparatus, device, and storage medium. Background Technology

[0002] Currently, in real-world scenarios, to learn about information about a physical object, one needs to input the object's name or photo into a terminal device to retrieve the relevant information from the internet. For example, when selecting books in a bookstore, to find out a book's rating or price, one needs to input the book's photo or name into the relevant application (APP) to view the information and aid in deciding whether to purchase it.

[0003] Among related technologies, there are ways to search based on augmented reality. For example, in shopping apps, AR mode can be used to recommend relevant products based on image recognition and price preferences; in educational apps, AR technology can be used to take a picture to search for questions, that is, to compare the image recognition results with its own question bank and then directly display the search results.

[0004] However, current AR search methods can usually only recommend information for specific needs. For example, in shopping apps, the search intent is to search for products, so products related to the input can be recommended; in educational apps, the search intent is to search for answers to questions, so the explanation process and answers to questions related to the input can be recommended.

[0005] Clearly, AR search methods based on related technologies can only meet fixed and relatively simple search needs. When other search intentions need to be met, it may be necessary to switch apps multiple times to satisfy different search needs, which is cumbersome and inefficient. Summary of the Invention

[0006] This application provides an augmented reality target display method, apparatus, device, and storage medium, which provides the function of custom search results, improves the accuracy of search results for target objects, and improves object search efficiency.

[0007] On the one hand, an augmented reality target display method is provided, the method comprising:

[0008] The augmented reality (AR) display interface is displayed, and the first search result associated with the target object is displayed in the target display area corresponding to the target object in the AR display interface;

[0009] In response to an update display instruction triggered for the target object, a second search result associated with the target object is displayed within the target display area. The second search result is different from the first search result and includes information corresponding to the information category specified by the update display instruction.

[0010] On the one hand, an augmented reality target display device is provided, the device comprising:

[0011] The first display unit is used to display an augmented reality (AR) display interface, wherein the first search result associated with the target object is displayed in the target display area corresponding to the target object in the AR display interface;

[0012] The second display unit is configured to, in response to an update display instruction triggered for the target object, display a second search result associated with the target object within the target display area. The second search result is different from the first search result, and the second search result includes information corresponding to the information category specified by the update display instruction.

[0013] Optionally, the AR display interface also displays a first page trigger control for an information selection page; then the device further includes an information selection unit, used for:

[0014] In response to a trigger operation performed on the trigger control of the first page, the information selection page is displayed; wherein, the information selection page displays multiple search options, each search option corresponds to the target object type of the target object, and each search option corresponds to an information category;

[0015] The update display instruction is generated in response to a selection operation for at least one of the plurality of search options.

[0016] Optionally, the information selection unit is specifically used for:

[0017] Based on the recognition results of the image currently displayed on the AR display interface, the target object type to which the target object belongs is determined; wherein, the currently displayed image includes the target object;

[0018] Based on the information dimensions associated with the target object type, determine each target information dimension associated with the target object;

[0019] In the respective display areas of each target information dimension on the information selection page, the search options corresponding to each target information dimension are displayed.

[0020] Optionally, the information selection unit is further configured to:

[0021] In response to a selection operation of at least one of the multiple search options displayed on the AR display interface, the update display instruction is generated, wherein the multiple search options correspond to the target object type of the target object, and each search option corresponds to an information category.

[0022] Optionally, the device further includes an emphasis display unit for:

[0023] In response to an object emphasis command triggered for the target object, the target object attribute specified by the object emphasis command is set as an emphasis attribute; wherein, the target object attribute is at least one of the object attributes possessed by the target object;

[0024] Emphasize the associated objects with the emphasis attribute in the AR display interface.

[0025] Optionally, if the information selection page displays a second page trigger control corresponding to the attribute emphasis page, then the emphasis display unit is specifically used for:

[0026] In response to a trigger operation performed on the trigger control of the second page, the attribute emphasis page is displayed; wherein, the attribute emphasis page displays multiple attribute emphasis options, the multiple attribute emphasis options correspond to the target object type of the target object, and each attribute emphasis option corresponds to an object attribute;

[0027] The object emphasis instruction is generated in response to a triggering operation performed on at least one of the plurality of attribute emphasis options.

[0028] Optionally, the emphasis display unit is specifically used for:

[0029] In response to a triggering operation performed on the target object, the attribute emphasis operation option corresponding to the target object is displayed, the attribute emphasis operation option being used to set the object attribute of the target object as an emphasized attribute;

[0030] In response to a triggering action on the attribute emphasis operation option, the object emphasis instruction is generated.

[0031] Optionally, the emphasis display unit is specifically used for:

[0032] The AR display interface performs object recognition on the currently displayed image to obtain corresponding recognition results; wherein, the recognition results include the object types of each object included in the currently displayed image;

[0033] Based on the recognition results, the object attributes corresponding to each object are determined;

[0034] If the emphasis attribute is in a valid state, then the object with the emphasis attribute among the various objects is determined as the associated object.

[0035] Optionally, the emphasis display unit is specifically used to emphasize the associated objects with the emphasis attribute in the AR display interface through the following display method:

[0036] The edge line of the associated object is displayed in the AR display interface to present the effect of the associated object being displayed separately from other objects; and / or,

[0037] The emphasis information is displayed in the display area corresponding to the associated object in the AR display interface.

[0038] Optionally, the device further includes an object recognition unit, used for:

[0039] In response to the trigger operation of activating the AR display function, object recognition is performed on the acquired image to be scanned to obtain the recognition result corresponding to the image to be scanned; wherein, the recognition result includes the target object and the target object type of the target object;

[0040] Based on the identification results and the historical behavior data of the currently logged-in target account, the first search result is determined.

[0041] Optionally, the object recognition unit is specifically used for:

[0042] In response to a trigger operation that activates the camera device, an image acquisition interface is displayed, wherein the image acquisition interface displays a function trigger control for activating the AR display function;

[0043] In response to a trigger operation performed on the function trigger control, object recognition is performed on the image to be scanned to obtain the recognition result corresponding to the image to be scanned.

[0044] Optionally, the object recognition unit is specifically used for:

[0045] Image feature extraction is performed on the image to be scanned to obtain the corresponding image feature vector;

[0046] Text detection is performed on the image to be scanned, and character recognition is performed on the detected text regions to obtain the corresponding text feature vectors;

[0047] Based on the image feature vector and the text feature vector, the recognition result corresponding to the image to be scanned is obtained.

[0048] Optionally, the object recognition unit is specifically used for:

[0049] Based on pre-defined information categories, candidate content corresponding to the target object is retrieved from the information retrieval channels corresponding to each information category;

[0050] Based on the historical behavior data, the obtained candidate content is sorted.

[0051] Based on the sorting results, the first search result is determined from each of the candidate contents.

[0052] Optionally, the second display unit is specifically used for:

[0053] In the AR display interface, the second search result is overlaid and displayed in the display area corresponding to the target object, and when the position of the target object changes in the AR display interface, the relative positional relationship between the second search result and the target object remains unchanged.

[0054] On one hand, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the above methods.

[0055] On the one hand, a computer storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the steps of any of the above methods.

[0056] On one hand, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of any of the methods described above.

[0057] In this embodiment, after displaying the first search result associated with the target object within the target display area corresponding to the target object in the AR display interface, an update display operation can be triggered for the target object. In response to the update display command triggered for the target object, a second search result associated with the target object is displayed within the target display area. The second search result differs from the first search result and includes information corresponding to the information category specified by the update display command. In other words, this embodiment provides a way to customize search results. When the first search result displayed in the AR display interface does not meet the requirements, an update display operation can be performed to customize the information to be displayed, thus assisting in custom searches. This solves the problem that related technologies can only meet fixed and single search needs, avoids the cumbersome operation caused by switching apps multiple times, and improves search efficiency. Furthermore, this embodiment also directly displays relevant information about the target object in the AR display interface, enabling more intuitive and efficient information display. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0059] Figure 1 This is a schematic diagram illustrating an application scenario provided in the embodiments of this application;

[0060] Figure 2 A schematic flowchart of an augmented reality target display method provided in an embodiment of this application;

[0061] Figures 3a-3b A schematic diagram illustrating the AR display interface for displaying search results provided in this application embodiment;

[0062] Figure 4 Another flowchart illustrating the augmented reality target display method provided in this application embodiment;

[0063] Figures 5a-5e This is a schematic diagram illustrating the operation process of the AR display interface provided in the embodiments of this application;

[0064] Figure 6 This is a flowchart illustrating the process of highlighting the attributes of a target object according to an embodiment of this application.

[0065] Figures 7a to 7d This is a schematic diagram of the operation process highlighted in the embodiments of this application;

[0066] Figure 8 A flowchart illustrating the target demonstration process provided in this application embodiment;

[0067] Figure 9 A schematic diagram of the structure of the augmented reality target display device provided in the embodiments of this application;

[0068] Figure 10 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0069] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Unless otherwise specified, the embodiments and features in the embodiments of this application can be arbitrarily combined with each other. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0070] To facilitate understanding of the technical solutions provided in the embodiments of this application, some key terms used in the embodiments of this application will be explained below:

[0071] Augmented Reality (AR), also known as Augmented Reality, is a relatively new technology that integrates real-world and virtual-world information. It uses computer technology to simulate and overlay virtual information onto the real world, making it perceptible to human senses and creating a sensory experience that transcends reality. The real environment and virtual objects can coexist in the same scene and space after being superimposed.

[0072] AR technology identifies objects, planes, and faces in the current environment and overlays virtual information. The virtual information includes a simple user interface (UI) as well as other additional information, such as the information in the search results page involved in the embodiments of this application.

[0073] Target object: In this embodiment of the application, the target object refers to any content that can be displayed in the AR display interface, including objects, planes, faces, or text. For example, when an image is currently displayed in the AR display interface, the target object can be the content included in that image. The target object can be different in different scenarios. For example, when a user wants to search for information related to a certain book, the target object can be the book itself; or, in an AR-assisted driving scenario, the target object can be surrounding vehicles, roadside signs, and obstacles.

[0074] Object recognition: Generally speaking, it refers to the identification of objects in an image. It is a fundamental research area in the field of computer vision. The task of object recognition is to identify what objects exist in an image and indicate the position and orientation of the object in the scene represented by the image.

[0075] This application relates to artificial intelligence (AI) technology, primarily computer vision technology within AI.

[0076] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0077] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and intelligent transportation.

[0078] Computer vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in recognizing and measuring targets, and then performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), autonomous driving, intelligent transportation, and other technologies, as well as common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0079] With the research and advancement of artificial intelligence (AI) technology, AI is being studied and applied in various fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, robots, smart healthcare, smart customer service, vehicle networking, and intelligent transportation. It is believed that with the development of technology, AI will be applied in more fields and play an increasingly important role.

[0080] The solutions provided in this application involve technologies such as computer vision technology in artificial intelligence. Specifically, the embodiments of this application use computer vision technology to extract corresponding object information from images, and then use the identified information to search from relevant retrieval channels to display the searched information on an AR display interface.

[0081] Specifically, in the AR display mode of this application embodiment, AR scanning of the image to be scanned can be performed based on computer vision technology to obtain relevant information of the target object in the image to be scanned, such as the object type, and then a search can be performed from the default search channel based on the relevant information of the target object to obtain search results, which are then displayed on the AR display interface.

[0082] The design concept of the embodiments of this application will be briefly introduced below.

[0083] Currently, in real-world scenarios, to learn about information about a physical object, one needs to input the object's name or photo into a terminal device to retrieve the relevant information from the internet. For example, when selecting books in a bookstore, to find out a book's rating or price, one needs to input the book's photo or name into the relevant application (APP) to view the information and aid in deciding whether to purchase it.

[0084] AR search methods based on related technologies can only meet fixed and relatively simple search needs. When other search intentions need to be met, it may be necessary to switch between apps multiple times to satisfy different search needs, which is cumbersome and inefficient. Furthermore, the display method is also very limited, usually only able to display a single guide label.

[0085] Furthermore, in real-world scenarios, when users want to find other objects with the same attributes as the target object, they cannot find them quickly and accurately. For example, if a user wants to quickly find books from the same publisher or category as a particular book among multiple books, the relevant technology can only determine this by flipping through each one individually, which is obviously a cumbersome process and extremely inefficient when there are many books in the bookstore.

[0086] Considering the aforementioned problems in meeting user needs, the root cause is that current AR search methods can only provide recommendations in a fixed direction, leaving users without a way to express their own needs. Furthermore, since the intent is obvious in specific apps, there is usually no other search intent. In order to meet users' search needs, a customization function can be provided, allowing users to express their own search needs.

[0087] In view of this, this application provides a target display method based on augmented reality. This method offers a way to customize search results. When the first search result displayed in the AR display interface does not meet the requirements, the display can be updated to customize the information to be displayed, thus assisting in customized searches. This solves the problem that related technologies can only meet fixed and single search needs, avoids the cumbersome operation caused by switching apps multiple times, and improves search efficiency. Furthermore, this application embodiment also directly displays relevant information about the target object in the AR display interface, enabling a more intuitive and efficient information display.

[0088] In other words, after making initial recommendations to users, if users are not satisfied with the default search results, they can update the display by customizing the information displayed. This allows the system to display information corresponding to the target object based on specified information categories, thereby meeting the user's search needs.

[0089] Furthermore, to help users quickly find other objects with the same attributes as the target object in real-world scenarios, users can perform an emphasis operation. This involves highlighting a specific attribute of the target object. When the AR display scans other objects with that emphasized attribute, they will be highlighted, allowing users to find other objects with the same attribute more quickly. For example, when a user is selecting books in a bookstore and emphasizes the author of book A, the author of book A can be set as the emphasized attribute. When AR scans other books by that author, these books will be highlighted, allowing users to find the book they need more quickly.

[0090] The following is a brief introduction to the application scenarios to which the technical solutions of the embodiments of this application are applicable. It should be noted that the application scenarios described below are only for illustrating the embodiments of this application and are not intended to limit the scope. In specific implementation, the technical solutions provided by the embodiments of this application can be flexibly applied according to actual needs.

[0091] The solution provided in this application can be applied to most AR display scenarios, such as AR search scenarios and AR driving scenarios. Figure 1 The diagram shown is an application scenario provided by an embodiment of this application. In this scenario, a terminal device 101 and a server 102 may be included.

[0092] Terminal device 101 can be, for example, a mobile phone, tablet computer (PAD), laptop computer, desktop computer, smart TV, smart in-vehicle device, and smart wearable device. Terminal device 101 can be equipped with an AR display application. The application involved in this embodiment can be a software client, or a webpage, mini-program, etc. The server is the backend server corresponding to the software, webpage, mini-program, etc., and there is no limitation on the specific type of client.

[0093] Server 102 can be the backend server corresponding to the AR display application installed on terminal device 101. It can realize the backend functions of AR display to implement the steps of the augmented reality target display method provided in this application embodiment. For example, it can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, i.e., content delivery network (CDN), and big data and artificial intelligence platforms, but it is not limited to these.

[0094] The augmented reality target display method provided in this application embodiment can be executed by a terminal device 101 or a server 102, or by a combination of terminal device 101 and server 102. Terminal device 101 or server 102 may include one or more processors, memory, and I / O interfaces for interaction with the terminal. Furthermore, terminal device 101 or server 102 may be configured with a database, which can be used to store model parameters related to object recognition models. The memory of terminal device 101 or server 102 may also store program instructions for the augmented reality target display method provided in this application embodiment. When these program instructions are executed by the processor, they can be used to implement the steps of the augmented reality target display method provided in this application embodiment, thereby achieving AR-based target display.

[0095] In one implementation, specifically in an AR search scenario, taking a browser as an example of the aforementioned AR display application, the user can enable the AR search function in the browser and display an AR display interface. Upon initial entry into the AR display interface, an image can be captured via a camera, and then object recognition can be performed on the captured image to identify the information of the target object in the image. Based on the recognition result, information is searched through default search channels to display the first search result on the AR display interface. If the user is not satisfied with the default search results, i.e., the information they need is not displayed, the user can update the display to customize the information categories to be displayed, thereby retrieving search results from the corresponding information category's search channels to display a second search result on the AR display interface, different from the first search result.

[0096] When displaying search results, they can be overlaid directly on the AR display interface for a more efficient and intuitive information presentation. Furthermore, when searching for other objects with the same attributes as the target object, a specific attribute of the target object can be emphasized. When other objects with the same attributes as the target object are scanned, that attribute will be highlighted, allowing users to find other objects with similar attributes more quickly.

[0097] Taking book selection as an example, users can open their device's camera and activate the AR guidance function. The camera recognizes the current object, and the system can present some default cards or tags to the user. If the default cards are what the user needs, no further action is required. If the user wants to customize the card content, they can click "edit" to select custom card content. Then, the user scans the current object again with the AR camera, and the information loaded on the object will be the user's customization. Furthermore, at a certain time, such as before the user's next entry, if the system recognizes that objects 'a' and 'b' have similar categories, it will display the user's customized information card for object 'a'. The user can click "edit" in the upper right corner at any time to reset these settings. Alternatively, based on the user's pre-defined tag 'a', the system can highlight the boundaries of recognized objects containing 'a' in yellow, separating them from other parts of the image to enhance the prompting effect. The corresponding component name will be displayed in the upper left corner within the dividing line, along with the text content of tag 'a'.

[0098] In one implementation, the application scenario can also be an AR driving scenario. In this scenario, the terminal device can specifically be an in-vehicle intelligent terminal. The in-vehicle intelligent terminal can enable the AR display function, and the AR display interface can display the scene around the vehicle, such as the scene in front of or behind the vehicle. Object recognition is performed on objects in the scene, and information about these objects is searched from relevant search channels. The search results are then overlaid and displayed on the scene. Similarly, when the user is not satisfied with these search results, they can also customize the search channels to overlay and display the information they need on the scene.

[0099] In one implementation, the AR display application described above in this application can specifically be a map application. This allows for real-world map display within the map application, and the enhanced display target display method provided in this application can be used to search for information about target objects on the map from relevant search channels, overlaying the search results onto the map screen. For example, the target object can be a map element, such as a building or a shop.

[0100] Terminal device 101 and server 102 can communicate directly or indirectly through one or more networks 103. The network 103 can be a wired network or a wireless network. For example, the wireless network can be a mobile cellular network or a Wireless-Fidelity (WIFI) network. Of course, it can also be other possible networks, and this embodiment of the invention does not limit them.

[0101] It should be noted that in this embodiment of the application, the number of terminal devices 101 can be one or more, and similarly, the number of servers 102 can also be one or more. That is to say, there is no limitation on the number of terminal devices 101 or servers 102.

[0102] In one possible application scenario, to reduce communication latency during retrieval, servers 102 can be deployed in various regions. Alternatively, for load balancing, different servers 102 can serve terminal devices 101 in different regions. For example, terminal device 101 located at location a establishes a communication connection with server 102 serving location a, and terminal device 101 located at location b establishes a communication connection with server 102 serving location b. Multiple servers 102 form a data sharing system, and data sharing is achieved through blockchain.

[0103] Each server 102 in the data sharing system has a corresponding node identifier. Each server 102 can also store node identifiers of other servers 102 in the data sharing system, so that generated blocks can be broadcast to other servers 102 in the data sharing system based on their node identifiers. Each server 102 can maintain a node identifier list, storing the server 102 name and node identifier in this list. The node identifier can be an Internet Protocol (IP) address or any other information that can be used to identify the node.

[0104] Of course, the methods provided in the embodiments of this application are not limited to... Figure 1 The application scenarios shown can also be used in other possible scenarios, and this application embodiment does not impose any limitations. Figure 1 The functions that each device in the application scenario shown can achieve will be described in subsequent method embodiments, and will not be elaborated on here.

[0105] The method flow provided in the various embodiments of this application will be described below with reference to the accompanying drawings. Figure 1 The execution can be performed by server 102 or terminal device 101, or by both server 102 and terminal device 101. Here, we will mainly take the execution by terminal device 101 as an example.

[0106] See Figure 2 The diagram shown is a flowchart illustrating the augmented reality target display method provided in an embodiment of this application.

[0107] Step 201: The terminal device displays the augmented reality (AR) display interface. Within the target display area corresponding to the target object in the AR display interface, the first search result associated with the target object is displayed.

[0108] In this embodiment of the application, the terminal device can be any electronic device, which can display an AR display interface through an AR display application with AR display function installed on it. For example, the AR display application can be a browser or a map application, but is not limited to this.

[0109] The target object can be any object displayed in the AR display interface. The AR display function can display one or multiple target objects, which can be an object, a plane, a face, or text, but is not limited to these. Furthermore, it should be noted that the target object can refer to a specific object, such as a specific object selected by the user; or, the target object can refer to all objects identified in the AR display interface. An update operation targeting the target object is effective for all currently displayed objects, and remains effective for other objects identified after the update operation.

[0110] Specifically, the AR display interface can display the image to be scanned, and the target object can be located within the image. The image to be scanned can be captured by a camera device included in the terminal device and then displayed in the AR display interface; alternatively, the image to be scanned can also be captured by a camera device external to the terminal device and then displayed in the AR display interface; or the image to be scanned can also be obtained from the terminal device or a network and then displayed in the AR display interface. This application embodiment does not limit the method of acquiring the image to be scanned.

[0111] In this embodiment of the application, during the initial display, the corresponding search results can be displayed by default according to the pre-set information categories. The pre-set information categories can be categories set by the system in the AR display application or categories pre-set by the user in the application settings.

[0112] See Figure 3a The image shows a schematic diagram of the AR display interface showing the first search result. In this AR display interface, the target object is a door, and the default information category displayed is the door's brand and rating. As you can see, after entering the AR display interface, you will see that the door's brand is A and its rating is 3 stars.

[0113] In this embodiment, the target display area can be any area in the AR display interface. For example, it can be located in the area where the target object is located, or it can be located in the area surrounding the target object. Alternatively, a dedicated display area can be set in the AR display interface to display the first search result of the target object. This embodiment does not limit this.

[0114] In this embodiment, the first search result can be displayed overlaid on the AR display interface, that is, virtual information is overlaid on the image of the display scene. Specifically, the virtual information refers to the first search result, thereby achieving an augmented reality display effect. For example... Figure 3a As shown, the corresponding first search result can be overlaid and displayed within the area where the door is located. The first search result can be overlaid and displayed in the form of cards or labels.

[0115] Step 202: In response to the update display instruction triggered for the target object, the terminal device displays a second search result associated with the target object in the target display area. The second search result is different from the first search result and includes information corresponding to the information category specified by the update display instruction.

[0116] In this embodiment of the application, when a user is not satisfied with the first search result, that is, when the user cannot obtain the information they need, the user can customize the information category to display information of the specified information category on the AR display interface.

[0117] Specifically, users can trigger an update display operation for a target object, and the terminal device can obtain an update display instruction based on the update display operation. The update display instruction specifies the information category to be displayed, and the device can obtain a second search result based on these information categories and display the corresponding second search result in the target display area corresponding to the target object.

[0118] Similarly, the second search result can also be overlaid and displayed within the display area corresponding to the target object. Furthermore, to enhance the AR display effect, the displayed search results can be bound to the target object; that is, the relative positional relationship between the search results and the target object is bound. When the position of the target object changes within the AR display interface, the relative positional relationship between the search results and the target object remains unchanged, thus the position of the search results changes accordingly. For example, when the search results are presented as cards, the effect is that the cards are pasted onto the target object and move as the target object moves.

[0119] See Figure 3bThe image shows a schematic diagram of the AR display interface showing the second search result. In this AR display interface, the target object is still the door. The information category specified by the update command has been expanded to include the price of the door on shopping platforms 1 and 2. As a result, in addition to displaying the brand name A and the rating of 3 stars, the AR display interface also shows the price of the door on shopping platforms 1 and 2 to help users compare prices and decide whether to buy.

[0120] In practical applications, the first and second search results can use the same display style, such as... Figure 3a and Figure 3b The same display style was used to display the first and second search results.

[0121] Alternatively, different display styles can be used to display the first and second search results.

[0122] In one implementation, the user can specify the display style of the second search result, thereby displaying it using the user-specified display style.

[0123] In one implementation, the display style of the second search result can also be determined based on the various information categories in the second search result. For example, if the second search result includes information in categories A and B, and category A needs to be displayed in style 1 while category B needs to be displayed in style 2, then the information in categories A and B can be displayed in their respective styles 1 and 2.

[0124] In one implementation, the display style of the second search result can be determined based on the amount of information in each piece of information in the second search result. For example, if the second search result includes a large amount of text content, it can be displayed in a style that facilitates the display of text content; or if the second search result includes images, it can be displayed in a style that facilitates the display of images.

[0125] In the embodiments of this application, see Figure 4 The diagram shown is another flowchart illustrating the augmented reality target display method provided in this application embodiment.

[0126] Step 401: In response to the trigger operation of starting the AR display function, perform object recognition on the acquired image to be scanned to obtain the recognition result corresponding to the image to be scanned.

[0127] In this embodiment of the application, the user can perform a trigger operation in the AR display application to trigger the opening of the AR display interface.

[0128] In one implementation, a user can trigger the camera device in an AR display application. The terminal device then responds to the trigger by displaying an image acquisition interface. This image acquisition interface can display a function trigger control for activating the AR display function. The user can then trigger the function trigger control again, and the terminal device can respond to the trigger by performing object recognition on the image to be scanned and obtaining the recognition result corresponding to the image to be scanned.

[0129] See Figure 5a This is a diagram illustrating the process of entering the AR display interface. Taking a browser as an example, on a designated page in the browser, you can see... Figure 5a The search box area shown may include a photo search entry point. Users can access the image capture interface by interacting with this entry point to capture images for searching. The image capture interface displays a function trigger control to activate the AR display feature, such as... Figure 5b As shown, options may include "Search Questions", "AR", "Scan ID Cards", and "Scan Documents". The icon area corresponding to AR is the function trigger control to start the AR display function, so that users can trigger it to enter the AR display interface.

[0130] In one implementation, to reduce the number of steps required from the user, the AR display function can be activated by default after the trigger operation to start the camera device, thus allowing the user to perform actions such as... Figure 5a After the trigger operation shown, you can directly enter the AR display interface.

[0131] In one implementation, a shortcut to the AR display interface can also be provided within the AR display application. For example, this shortcut can be displayed on the homepage of the AR display application; or, it can be displayed in the menu bar of the terminal device; or, a shortcut key or gesture can be set to provide a way to quickly access the AR display page.

[0132] See Figure 5a As shown, upon receiving a trigger operation to activate the AR display function, the system can respond to this operation and enter the AR display interface. On the AR display interface, the user can preview the scene captured by the current camera device. This scene can then be used as the image to be scanned. Alternatively, see [link to documentation]. Figure 5a As shown, users can also select locally stored images or videos as images to be scanned. That is, users can operate on the local image acquisition entry to enter the local image selection page and select at least one local image or video to acquire the image to be scanned.

[0133] Specifically, in order to perform object recognition on the image to be scanned, the terminal device can generate an object recognition instruction based on the image to be scanned, so as to call the background object recognition service to perform object recognition on the image to be scanned.

[0134] The AR display interface can display images captured by the camera device in real time and perform AR scanning on each image, that is, object recognition. Since the processing of each image to be scanned is similar, only one image to be scanned will be used as an example for introduction here.

[0135] Specifically, the first step can be to denoise the image to be scanned. For example, median filtering can be used. Median filtering is a non-linear filtering method based on sorting statistics theory, which is a non-linear signal processing technique that can effectively suppress noise. Its idea is to replace the gray value of a pixel with the median of its neighborhood gray values, making the values ​​of surrounding pixels closer to the true values, thereby eliminating isolated noise points. This method can preserve the edge details of the image while removing impulse noise and salt-and-pepper noise.

[0136] For the denoised image to be scanned, image features are extracted to obtain the corresponding image feature vectors. For example, features such as shape, texture, and color in the image to be scanned can be extracted. Feature extraction can be implemented using a pre-trained feature extraction model based on machine learning methods, such as Convolutional Neural Networks (CNNs) or a network model containing a CNN structure. The specific structure of the feature extraction model is not limited in this embodiment.

[0137] Considering that the acquired images may contain text, and text plays an important role in object recognition, for example, the text on a book can intuitively reflect the content of the book, such as the book title, author and publisher, text detection can also be performed on the scanned image, and character recognition can be performed on the detected text regions to obtain the corresponding text feature vectors.

[0138] Specifically, for text regions, character preprocessing can be performed, such as segmenting the text region into characters, normalizing the size of each character and the stroke width, and then performing character recognition based on enhancement algorithms to obtain the corresponding text feature vectors. These text feature vectors can indicate the characters contained in the image to be scanned.

[0139] It should be noted that the above-mentioned image feature extraction process and text detection and recognition process do not have a substantial sequential order. That is to say, they can be performed simultaneously or sequentially. This application embodiment does not impose any restrictions on this.

[0140] Furthermore, based on the aforementioned image feature vectors and text feature vectors, the recognition result corresponding to the image to be scanned can be obtained. The recognition result may include the target object and the target object type of the target object.

[0141] Step 402: Based on the identification results and the historical behavior data of the currently logged-in target account, determine the first search result.

[0142] In one implementation, taking an AR display application as a browser as an example, image feature vectors and text feature vectors can be input into the browser's search engine to identify and obtain the target object and the target object type. At the same time, the search engine can also perform related searches for the target object to obtain search results related to the target object.

[0143] In one implementation, the recognition results obtained based on image feature vectors and text feature vectors can also be input into the browser's search engine to perform an association search on the target object and obtain candidate content associated with the target object.

[0144] Specifically, upon initial entry into the AR display interface, candidate content corresponding to the target object can be retrieved from the information retrieval channel corresponding to each information category, based on the pre-defined information category. For example, if the default information retrieval channel is the price of the target object on shopping platforms 1 and 2, then the price of the target object can be retrieved from the shopping platforms.

[0145] Taking a book as the target object as an example, the identification results can include information such as the book's name, author, publisher, and publication date. This information can then be input into a search engine to retrieve candidate search results related to the book. Specifically, candidate results can include one or more of the following:

[0146] (1) Evaluation content of the book on various book review websites, such as ratings and comments.

[0147] (2) A brief introduction to the book

[0148] (3) Prices of the book on various shopping platforms

[0149] (4) Have any of your friends read the book? If so, which of your friends have read the book?

[0150] It should be noted that the above content can also include any candidate search content related to books, which will not be listed one by one in this application embodiment. In addition, the above content is candidate content for when the target object is a book, and when the target object is other object types, the content can also be changed accordingly.

[0151] In this embodiment of the application, in order to make the displayed content more in line with the user's usage habits, after obtaining the above-mentioned candidate content, the user can also select content that is more in line with the user's usage habits from the candidate content based on the historical behavior data of the currently logged-in target account.

[0152] Specifically, for the candidate content obtained above, each candidate can be sorted based on the user's historical behavior data, and then the first search result can be determined from the sorted results. For example, taking books as the target, the candidate content can be sorted based on the creators the user has followed or the articles they have viewed, and the top 1 to 5 items can be selected as the first search result.

[0153] Step 403: Display the first search result in the target display area corresponding to the target object on the AR display interface.

[0154] Specifically, the first search result can be rendered as an AR card or label and overlaid on the target display area. See also Figure 5a As shown, for the books displayed in this AR display interface, the book's rating can be overlaid on the book, and the book's reviews on book review websites and its price on shopping platforms can be overlaid around the book. Of course, in actual applications, the display style of AR cards or tags can be adjusted according to actual needs.

[0155] In one implementation, when the first search result involves multiple pieces of information, the display order of these pieces of information can be adjusted according to the user's historical behavior data.

[0156] Step 404: In response to the trigger operation performed on the trigger control of the first page, display the information to select the page.

[0157] See Figure 5a As shown, the AR display interface also includes a first-page trigger control for the information selection page. Therefore, if a user is not satisfied with the search results displayed for the current target object, they can trigger the first-page trigger control to enter the information selection page and choose the search results they want to see. The information selection page displays multiple search options, each corresponding to a specific target object type and information category.

[0158] In this embodiment of the application, the search options in the information selection page corresponding to different object types can be different. Therefore, the target object type to which the target object belongs can be determined based on the recognition result of the image currently displayed on the AR display interface, and the various target information dimensions associated with the target object can be determined based on the information dimensions associated with the target object type.

[0159] Specifically, the information dimensions associated with each object type can be pre-configured. For example, when the target object is a book, information dimensions such as rating and price can be associated. When the target object is a building, information dimensions such as location, shopping mall, rating and shop can be associated. After determining the target object type, the various target information dimensions associated with the target object can be determined based on the information dimensions associated with the target object type.

[0160] Then, in the respective display area of ​​each target information dimension on the information selection page, the search options corresponding to each target information dimension are displayed.

[0161] See Figure 5b The image shows a schematic of the information selection page when the target object is a book. When the target object is a book, search options related to books can be displayed, such as the object attributes associated with the book. Figure 5b The text displays information such as the book title, author, and publisher; or, for example, price information related to the book, it can display search options for the book's price on various shopping platforms within its display area, such as... Figure 5b The search options for shopping platforms 1 through 5 are displayed. When a user selects a shopping platform, they can search for the book's price on that platform. Similarly, books can also be associated with rating information, such as... Figure 5b The display area corresponding to the rating information dimension also shows the search options for book review websites 1 to 3. When a user selects a book review website, they can search for reviews from that website and obtain the book's ratings on that website. The same applies to other search options, which will not be listed one by one.

[0162] Step 405: In response to a selection operation for at least one of the multiple search options, generate an update display instruction.

[0163] See Figure 5c The diagram shows the display of the second search result on the AR display interface. After the user completes the selection and confirms the selection, the terminal device can generate an update display instruction based on the confirmation.

[0164] It should be noted that steps 404 and 405 described above are only one possible way to perform a custom search. In practical applications, other possible custom search methods may also be included, and this application embodiment does not limit this. For example, multiple search options can be directly displayed on the AR display interface. These multiple search options correspond to the target object type of the target object, and each search option corresponds to an information category. Thus, the user can directly select at least one search option on the AR display interface to generate corresponding update display instructions.

[0165] Step 406: In response to the update display instruction, display the second search result associated with the target object within the target display area.

[0166] In this embodiment, based on the update display command, the terminal device can scan the image displayed in the AR display interface again to identify objects in the image. Then, based on the identification results and the information category specified by the update display command, it performs another search to obtain a corresponding second search result. The second search result associated with the target object is then displayed within the target display area. The process of obtaining the second search result is similar to that of obtaining the first search result, the only difference being the information category. Therefore, please refer to the description of the first search result above, which will not be repeated here.

[0167] Specifically, based on the user-defined information display mode, the identified object recognition information is imported into the relevant search channels for searching, the datasets from each search channel are obtained and then returned to the front end to be rendered into card components, which are then loaded onto the target object in AR form.

[0168] Specifically, depending on the information category, the second search result may include one or more of the following information:

[0169] (1) Pure UI type, which only annotates the current target object to help users quickly understand the object attributes of the current target object. Taking the above books as an example, this type of information can include the book title, category, publisher, publication time and content summary, so that users can intuitively perceive the object attributes of each book in the current AR display interface. For example, when users want to find design books, they can intuitively see which books belong to the design category through the AR display interface, helping users quickly find the books they want.

[0170] (2) Includes related information. The source of the related information can be defined by the user. For example, the user can view the book's rating on a book review website and comments from various users on the book.

[0171] See Figure 5cAs shown, the user selected shopping platforms 1, 2, 4, and 5 as search options in the price information dimension. After confirmation, the prices of shopping platforms 1, 2, 4, and 5 can be displayed on the AR display interface. At the same time, the user selected to view the ratings of book review websites 1 and 2 in the rating information dimension. After confirmation, the reviews of book review websites 1 and 2 can be displayed on the AR display interface.

[0172] See Figure 5d The image shown is another schematic diagram illustrating the display of the second search result on the AR display interface. As mentioned above, the AR display interface can display images captured by the camera device in real time. Since the images captured by the camera device may change, after customization and confirmation, object recognition and searching can be performed again on the current image. See [link to documentation]. Figure 5d As shown, the current image frame is from Figure 5a The screen shown has changed Figure 5d The next screen will then display a second search result for the target object in the new screen, such as... Figure 5d The text displays ratings for new books, along with reviews from associated book review website 1, and prices on shopping platforms 1, 2, 4, and 5. It should be noted that... Figure 5d The image only shows the second search result for one target object. In practical applications, other objects can also be used as target objects, and their corresponding second search results can be displayed as well.

[0173] See Figure 5e The image shown is another illustration of displaying the second search result on the AR display interface. Figure 5d If only the reviews from Book Review Website 1 are selected to be displayed on the custom page, then after confirmation, only the reviews from Book Review Website 1 will be displayed for the identified objects.

[0174] In this embodiment of the application, to help users quickly find other objects with the same attributes, this embodiment can also emphasize a certain object attribute of the target object, so as to highlight other objects with the same attributes in the AR display screen. See also Figure 6 The diagram shows a flowchart illustrating the process of highlighting the target object's attributes.

[0175] Step 601: In response to an object emphasis instruction triggered for the target object, set the target object property specified by the object emphasis instruction as the emphasis property.

[0176] Among them, the target object attribute is at least one of the object attributes that the target object has.

[0177] In this embodiment of the application, a second page trigger control corresponding to the attribute emphasis page can be set in the AR display application, so that the user can operate on it to enter the attribute emphasis page and then emphasize the setting operation.

[0178] In one implementation, see Figure 7a The diagram illustrates one operation for setting an emphasized attribute. The information selection page displays a second-page trigger control corresponding to the attribute emphasis page. Users can trigger this control, and the terminal device responds by displaying the attribute emphasis page. The attribute emphasis page displays multiple attribute emphasis options, each corresponding to a target object type. Each attribute emphasis option represents a specific object attribute. Users can select at least one attribute emphasis option and trigger it. The terminal device then generates an object emphasis command to set the target object attribute corresponding to at least one attribute emphasis option as the emphasized attribute.

[0179] See Figure 7a As shown, when the target object is a book, its object attributes can be correspondingly the book title, author, publisher, and publication date. However, when the target object is another type of object, the object attributes can also change accordingly.

[0180] In one implementation, see Figure 7b The diagram illustrates another method for setting emphasized attributes. Specifically, users can select a target object and then perform a trigger action on it. For example, this trigger action could be a long press on the target object, which would then display the attribute emphasis options for that target object. Each attribute emphasis option is used to set the target object's attribute as the emphasized attribute. (See also...) Figure 7b As shown, you can choose to set attributes such as book title as emphasis attributes, and then the terminal device will respond to the trigger operation of the attribute emphasis operation option and generate the corresponding object emphasis instruction.

[0181] Step 602: Perform object recognition on the image currently displayed on the AR display interface to obtain the corresponding recognition results; wherein, the recognition results include the object types of each object included in the currently displayed image.

[0182] Step 603: Based on the recognition results, determine the object attributes corresponding to each object.

[0183] Step 604: If the emphasis attribute is in a valid state, then identify the objects with the emphasis attribute as associated objects.

[0184] In this embodiment, after emphasizing the target object attributes of the target object, object recognition can be performed again on the currently displayed image to identify the object attributes of each object in the currently displayed image, thereby identifying the objects with emphasized attributes as associated objects. The process of identifying object attributes can be found in the description of the corresponding sections above, and will not be repeated here.

[0185] In this embodiment, users typically search for items with the same attribute immediately; therefore, the emphasized attribute can have an expiration date, during which the emphasized attribute remains valid. Specifically, the expiration date can include one or more of the following:

[0186] (1) The time period between entering the AR display interface and not exiting the current AR display interface, that is, from entering the AR display interface this time to entering the AR display interface again.

[0187] (2) The time period between setting the emphasis attribute this time and setting the emphasis attribute again next time.

[0188] When the emphasis attribute is active and other objects displayed on the current AR display interface are identified as having the emphasis attribute, the object with the emphasis attribute can be identified as the associated object.

[0189] Step 605: Emphasize the associated objects with emphasis attributes in the AR display interface.

[0190] In this embodiment of the application, the purpose of highlighting is to make the associated objects more easily perceived by the user. Therefore, any method that can play a highlighting role can be used as a method of highlighting.

[0191] Specifically, one or more of the following methods can be used for emphasis display:

[0192] (1) Display the edge lines of related objects in the AR display interface to present the effect of separating related objects from other objects. For example, see Figure 7c and Figure 7d The diagram illustrates an emphasis display process. Based on user-defined emphasis attributes, the boundaries of identified objects containing these attributes are highlighted in yellow to separate them from the rest of the image. See also... Figure 7d As shown, when there are a large number of books, finding a specific book requires flipping through them one by one. However, this embodiment of the application, by selecting an emphasis attribute, can highlight and display books with that attribute among a large number of books. Figure 7d As you can see, users can more intuitively perceive the books they want to find and find them more quickly, making it extremely efficient.

[0193] (2) Display emphasis information in the corresponding display area of ​​the associated object in the AR display interface. See also Figure 7c As shown, for example, the corresponding item name can be displayed at a specified position within the dividing line box, such as the upper left corner, and the attribute content of the emphasized attribute can be presented in the form of a pop-up window, such as using a toast or other possible methods to display the pop-up window.

[0194] In one implementation, when the emphasized attribute is in a valid state, and it is recognized that other objects displayed on the current AR display interface share common attributes with the target object, and the user can display a customized information card for the target object, the user can click the edit button in the upper right corner at any time during use to reset it.

[0195] In this embodiment, the technical solution involves two parts: front-end display and back-end data processing. The front-end can be implemented by a terminal device, and the back-end can be implemented by the terminal device and / or a back-end server. The processes executed at each end are described below. See [link to documentation]. Figure 8 The diagram shown is a flowchart illustrating the target demonstration process provided in an embodiment of this application.

[0196] Step 801a: Based on the user's trigger operation, turn on the camera device of the terminal device and turn on AR mode.

[0197] Step 802a: The front end identifies objects in the currently displayed image.

[0198] Upon receiving a trigger to activate the AR display function, the system can respond to this trigger and enter the AR display interface. On the AR display interface, the user can preview the scene captured by the current camera device. This scene can then be used as the image to be scanned. Alternatively, see [link to other documentation]. Figure 5a As shown, users can also select locally stored images or videos as images to be scanned. That is, users can operate on the local image acquisition entry to enter the local image selection page and select at least one local image or video to acquire the image to be scanned.

[0199] Specifically, in order to perform object recognition on the image to be scanned, the terminal device can generate an object recognition instruction based on the image to be scanned, so as to call the background object recognition service to perform object recognition on the image to be scanned.

[0200] The front end can identify objects in the currently displayed image by sending object recognition instructions to the back end.

[0201] Step 803b: The backend performs object recognition on the currently displayed image, including image recognition and text recognition, in order to recommend relevant content for the target object.

[0202] Step 804a: The front end displays the default card corresponding to the target object.

[0203] The front-end renders content retrieved from the back-end search to display the default card or label corresponding to the target object.

[0204] Step 805b: The backend adjusts the order in which the cards are presented based on the user's previous usage habits.

[0205] Step 806a: If the user is not satisfied with the default card, the user can perform a customization operation on the front end to customize the card content.

[0206] Step 807a: The front end displays a custom menu based on the information dimensions that the current target object can identify.

[0207] Step 808b: The backend analyzes the information dimensions related to the target object based on the object recognition results.

[0208] Specifically, the front end can obtain the information dimensions that the current target object can recognize based on the information dimension data transmitted from the back end, in order to display a custom menu.

[0209] Step 809a: The user performs an emphasis operation on the front end, selecting to emphasize attribute tag A of the target object.

[0210] Step 810a: The front end continues to scan in AR mode. When an object with attribute label A is detected, it is highlighted.

[0211] Step 811b: The backend identifies the currently displayed image. If attribute label A appears, it is emphasized. The display screen is rendered according to the emphasis data and emphasized on the frontend.

[0212] Step 812a: The user performs a custom selection operation on the front end and chooses to display information of category B.

[0213] Step 813a: The front end continues to scan using AR mode and displays the B-class information of each object.

[0214] Step 814b: The backend identifies the currently displayed image and renders the display screen based on the identification results and related search results, displaying Category B information on the frontend.

[0215] It should be noted that the steps marked 'a' and 'b' in the above steps are the steps executed by the front end and the back end, respectively. However, there is no actual sequential relationship between the steps corresponding to 'a' and 'b'. They can be executed simultaneously or sequentially.

[0216] In summary, this application provides an AR search method that allows users to customize their needs and displays targets using AR technology. Specifically, when a user sees an item they want to search for in a real-world scene, they can activate the AR mode of the AR display application. Based on the current recognition results, the application will recommend some default information categories. If the user is not satisfied, they can customize the tags they want to see in the current scene, helping them search for the target item more quickly. By customizing the user's needs for objects, the application's boundaries can be significantly expanded, improving user efficiency and the accuracy of search result card recommendations in AR mode. Furthermore, by utilizing AR's feature of directly overlaying virtual information onto real-world objects, the display of targets can be more efficient.

[0217] Please see Figure 9 Based on the same inventive concept, embodiments of this application also provide an augmented reality target display device 90, which includes:

[0218] The first display unit 901 is used to display an augmented reality (AR) display interface. Within the target display area corresponding to the target object in the AR display interface, the first search result associated with the target object is displayed.

[0219] The second display unit 902 is used to respond to an update display instruction triggered for a target object, and to display a second search result associated with the target object in the target display area. The second search result is different from the first search result, and the second search result includes information corresponding to the information category specified by the update display instruction.

[0220] Optionally, the AR display interface may also show a first page trigger control for an information selection page; in this case, the device may further include an information selection unit 903, used for:

[0221] In response to a trigger operation performed on the trigger control on the first page, an information selection page is displayed; the information selection page displays multiple search options, each search option corresponds to the target object type of the target object, and each search option corresponds to an information category;

[0222] In response to a selection operation for at least one of a plurality of search options, an update display instruction is generated.

[0223] Optionally, the information selection unit 903 is specifically used for:

[0224] Based on the recognition results of the image currently displayed on the AR display interface, the target object type is determined; wherein, the currently displayed image includes the target object;

[0225] Based on the information dimensions associated with the target object type, determine the various target information dimensions associated with the target object;

[0226] In the information selection page, each target information dimension's corresponding display area will show the search options for that target information dimension.

[0227] Optionally, the information selection unit 903 is also used for:

[0228] In response to a selection operation of at least one of the multiple search options displayed on the AR display interface, an update display instruction is generated, wherein the multiple search options correspond to the target object type of the target object, and each search option corresponds to an information category.

[0229] Optionally, the device also includes an emphasis display unit 904, for:

[0230] In response to an object emphasis instruction triggered on a target object, the target object property specified by the object emphasis instruction is set as the emphasis property; wherein, the target object property is at least one of the object properties that the target object has;

[0231] Emphasize related objects with emphasis attributes in the AR display interface.

[0232] Optionally, if the information selection page displays a second page trigger control corresponding to the attribute-emphasized page, then the emphasis display unit 904 is specifically used for:

[0233] In response to a trigger operation performed on the trigger control on the second page, an attribute emphasis page is displayed; the attribute emphasis page displays multiple attribute emphasis options, each of which corresponds to the target object type of the target object, and each attribute emphasis option corresponds to an object attribute;

[0234] In response to a triggering action performed on at least one of a plurality of attribute emphasis options, an object emphasis instruction is generated.

[0235] Optionally, the emphasis display unit 904 is specifically used for:

[0236] In response to a triggering operation performed on the target object, display the attribute emphasis operation options corresponding to the target object. The attribute emphasis operation options are used to set the object property of the target object as the emphasized property.

[0237] In response to a trigger action on an attribute emphasis option, an object emphasis instruction is generated.

[0238] Optionally, the emphasis display unit 904 is specifically used for:

[0239] Perform object recognition on the image currently displayed on the AR display interface to obtain the corresponding recognition results; wherein, the recognition results include the object type of each object included in the currently displayed image;

[0240] Based on the recognition results, determine the object attributes corresponding to each object;

[0241] If the emphasized attribute is in a valid state, then the objects with the emphasized attribute among the various objects are identified as associated objects.

[0242] Optionally, the emphasis display unit 904 is specifically used to emphasize associated objects with emphasis attributes in the AR display interface through the following display methods:

[0243] Display the edge lines of related objects in the AR display interface to present the effect of separating related objects from other objects; and / or,

[0244] In the AR display interface, display the corresponding display area of ​​the associated object and show the emphasis information.

[0245] Optionally, the device further includes an object recognition unit 905, used for:

[0246] In response to the trigger operation of starting the AR display function, object recognition is performed on the acquired image to be scanned to obtain the recognition result corresponding to the image to be scanned; wherein, the recognition result includes the target object and the target object type of the target object;

[0247] Based on the identification results and the historical behavior data of the currently logged-in target account, the first search result is determined.

[0248] Optionally, the object recognition unit 905 is specifically used for:

[0249] In response to the trigger operation of activating the camera device, an image acquisition interface is displayed, in which a function trigger control for activating the AR display function is displayed;

[0250] In response to a trigger operation performed on a function trigger control, object recognition is performed on the image to be scanned to obtain the recognition result corresponding to the image to be scanned.

[0251] Optionally, the object recognition unit 905 is specifically used for:

[0252] Image feature extraction is performed on the image to be scanned to obtain the corresponding image feature vector;

[0253] Text detection is performed on the image to be scanned, and character recognition is performed on the detected text regions to obtain the corresponding text feature vectors;

[0254] Based on image feature vectors and text feature vectors, the recognition results corresponding to the image to be scanned are obtained.

[0255] Optionally, the object recognition unit 905 is specifically used for:

[0256] Based on pre-defined information categories, candidate content corresponding to the target object is retrieved from the information retrieval channels corresponding to each information category;

[0257] Based on historical behavioral data, the obtained candidate content is sorted.

[0258] Based on the ranking results, the first search result is determined from all candidate content.

[0259] Optionally, the second display unit 902 is specifically used for:

[0260] In the AR display interface, the second search result is overlaid and displayed in the display area corresponding to the target object, and the relative positional relationship between the second search result and the target object remains unchanged when the position of the target object changes in the AR display interface.

[0261] With the aforementioned device, when the first search result displayed in the AR display interface does not meet the requirements, an update operation can be performed to customize the information to be displayed, thereby assisting in customized searches. This solves the problem that related technologies can only meet fixed and single search needs, avoids the cumbersome operation caused by switching apps multiple times, and improves search efficiency. Furthermore, in this embodiment, relevant information about the target object is also directly displayed in the AR display interface, enabling a more intuitive and efficient information presentation.

[0262] In addition, to help users quickly find other objects with the same attributes as the target object in real-world scenarios, users can perform an emphasis operation, that is, emphasize a certain object attribute of the target object. When the AR display interface scans other objects with that emphasized attribute, they will be highlighted and displayed, so that users can find other objects with the same attribute more quickly.

[0263] This device can be used to execute the methods shown in the various embodiments of this application. Therefore, the functions that each functional module of this device can achieve can be referred to the description of the foregoing embodiments, and will not be repeated here.

[0264] Please see Figure 10 Based on the same technical concept, this application also provides a computer device 100, which can be used for... Figure 1 The terminal device or server shown may include a memory 1001 and a processor 1002.

[0265] The memory 1001 is used to store computer programs executed by the processor 1002. The memory 1001 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function, etc.; the data storage area may store data created based on the use of the computer device, etc. The processor 1002 may be a central processing unit (CPU), or a digital processing unit, etc. This application embodiment does not limit the specific connection medium between the memory 1001 and the processor 1002. This application embodiment... Figure 10 The memory 1001 and the processor 1002 are connected via a bus 1003, and the bus 1003 is in Figure 10 The connections between other components are shown in bold and are for illustrative purposes only, not as limiting information. The bus 1003 can be divided into address bus, data bus, control bus, etc. For ease of illustration, Figure 10 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0266] Memory 1001 may be volatile memory, such as random-access memory (RAM); memory 1001 may also be non-volatile memory, such as read-only memory, flash memory, hard disk drive (HDD), or solid-state drive (SSD); or memory 1001 may be any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. Memory 1001 may be a combination of the above-mentioned memories.

[0267] The processor 1002 is used to execute the methods performed by the device in various embodiments of this application when calling the computer program stored in the memory 1001.

[0268] In some possible implementations, various aspects of the methods provided in this application may also be implemented as a program product comprising program code that, when run on a computer device, causes the computer device to perform the steps of the methods described above according to various exemplary embodiments of this application. For example, the computer device may perform the methods performed by the device in various embodiments of this application.

[0269] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0270] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0271] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. An augmented reality target display method, characterized in that, The method includes: The augmented reality (AR) display interface is displayed, and the first search result associated with the target object is displayed in the target display area corresponding to the target object in the AR display interface; In response to a trigger operation performed on the first page trigger control in the AR display interface, an information selection page is displayed; wherein, in the display area corresponding to each target information dimension in the information selection page, search options corresponding to each target information dimension are displayed, each search option corresponds to an information category, and each target information dimension is determined according to the information dimension associated with the target object type of the target object, and the target object type is determined based on the recognition result of the image currently displayed in the AR display interface; In response to a selection operation for at least one search option in the information selection page, an update display instruction is generated; In response to the update display instruction, a second search result associated with the target object is displayed within the target display area. The second search result is different from the first search result and includes information corresponding to the information category specified by the update display instruction.

2. The method as described in claim 1, characterized in that, Before displaying a second search result associated with the target object within the target display area in response to the update display instruction, the method further includes: In response to a selection operation of at least one of the multiple search options displayed on the AR display interface, the update display instruction is generated, wherein the multiple search options correspond to the target object type of the target object, and each search option corresponds to an information category.

3. The method as described in claim 1, characterized in that, The method further includes: In response to an object emphasis command triggered for the target object, the target object attribute specified by the object emphasis command is set as an emphasis attribute; wherein, the target object attribute is at least one of the object attributes possessed by the target object; Emphasize the associated objects with the emphasis attribute in the AR display interface.

4. The method as described in claim 3, characterized in that, The information selection page displays a second page trigger control corresponding to the attribute emphasis page; Before setting the target object property of the target object specified by the object emphasis instruction as the emphasis property in response to the object emphasis instruction triggered for the target object, the method further includes: In response to a trigger operation performed on the trigger control of the second page, the attribute emphasis page is displayed; wherein, the attribute emphasis page displays multiple attribute emphasis options, the multiple attribute emphasis options correspond to the target object type of the target object, and each attribute emphasis option corresponds to an object attribute; The object emphasis instruction is generated in response to a triggering operation performed on at least one of the plurality of attribute emphasis options.

5. The method as described in claim 3, characterized in that, Before setting the target object property of the target object specified by the object emphasis instruction as an emphasis property in response to an object emphasis instruction triggered for the target object, the method further includes: In response to a triggering operation performed on the target object, the attribute emphasis operation option corresponding to the target object is displayed, the attribute emphasis operation option being used to set the object attribute of the target object as an emphasized attribute; In response to a triggering action on the attribute emphasis operation option, the object emphasis instruction is generated.

6. The method as described in claim 3, characterized in that, Before emphasizing the associated objects with the emphasis attribute in the AR display interface, the method further includes: The AR display interface performs object recognition on the image currently displayed to obtain corresponding recognition results; wherein, the recognition results include the object types of each object included in the currently displayed image; Based on the recognition results, the object attributes corresponding to each object are determined; If the emphasis attribute is in a valid state, then the object with the emphasis attribute among the various objects is determined as the associated object.

7. The method as described in claim 3, characterized in that, The associated objects with the emphasis attribute in the AR display interface are highlighted using the following display method: The edge line of the associated object is displayed in the AR display interface to present the effect of the associated object being displayed separately from other objects; and / or, In the AR display interface, the display area corresponding to the associated object displays emphasis information, which includes the attribute content of the emphasis attribute.

8. The method as described in claim 1, characterized in that, Before displaying the AR display interface, the method further includes: In response to the trigger operation of activating the AR display function, object recognition is performed on the acquired image to be scanned to obtain the recognition result corresponding to the image to be scanned; wherein, the recognition result includes the target object and the target object type of the target object; Based on the identification results and the historical behavior data of the currently logged-in target account, the first search result is determined.

9. The method as described in claim 8, characterized in that, In response to the trigger operation of activating the AR display function, object recognition is performed on the acquired image to be scanned to obtain the recognition result corresponding to the image to be scanned, including: In response to a trigger operation that activates the camera device, an image acquisition interface is displayed, wherein the image acquisition interface displays a function trigger control for activating the AR display function; In response to a trigger operation performed on the function trigger control, object recognition is performed on the image to be scanned to obtain the recognition result corresponding to the image to be scanned.

10. The method as described in claim 8, characterized in that, In response to the trigger operation of activating the AR display function, object recognition is performed on the acquired image to be scanned to obtain the recognition result corresponding to the image to be scanned, including: Image feature extraction is performed on the image to be scanned to obtain the corresponding image feature vector; Text detection is performed on the image to be scanned, and character recognition is performed on the detected text regions to obtain the corresponding text feature vectors; Based on the image feature vector and the text feature vector, the recognition result corresponding to the image to be scanned is obtained.

11. The method as described in claim 8, characterized in that, Based on the identification results and the historical behavior data of the currently logged-in target account, the first search result is determined, including: Based on pre-defined information categories, candidate content corresponding to the target object is retrieved from the information retrieval channels corresponding to each information category; Based on the historical behavior data, the obtained candidate content is sorted. Based on the sorting results, the first search result is determined from each of the candidate contents.

12. The method according to any one of claims 1 to 11, characterized in that, In response to an update display instruction triggered for the target object, a second search result associated with the target object is displayed within the target display area, including: In the AR display interface, the second search result is overlaid and displayed in the display area corresponding to the target object, and when the position of the target object changes in the AR display interface, the relative positional relationship between the second search result and the target object remains unchanged.

13. An augmented reality target display device, characterized in that, The device includes: The first display unit is used to display an augmented reality (AR) display interface, wherein the first search result associated with the target object is displayed in the target display area corresponding to the target object in the AR display interface; The second display unit is configured to display an information selection page in response to a trigger operation performed on a first page trigger control in the AR display interface. Within the display area corresponding to each target information dimension in the information selection page, search options corresponding to each target information dimension are displayed. Each search option corresponds to an information category. The target information dimensions are determined based on the information dimensions associated with the target object type, which is determined based on the recognition result of the image currently displayed on the AR display interface. In response to a selection operation performed on at least one search option in the information selection page, an update display instruction is generated. In response to the update display instruction triggered on the target object, a second search result associated with the target object is displayed in the target display area. The second search result is different from the first search result, and the second search result includes information corresponding to the information category specified by the update display instruction.

14. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 12.

15. A computer storage medium storing computer program instructions thereon, characterized in that, When executed by a processor, the computer program instructions implement the steps of the method according to any one of claims 1 to 12.

16. A computer program product comprising computer program instructions, characterized in that, When executed by a processor, the computer program instructions implement the steps of the method according to any one of claims 1 to 12.

Images