A method, device, and storage medium for displaying scene guidance

Abstract

This application discloses a scene display guidance method, device, and storage medium. The scene display guidance method includes acquiring a scene image in a scene space using an image sensor; identifying image feature points in the scene image to obtain the distribution of these feature points in the scene image; and determining guidance information to change the position of the image sensor in the scene space based on the distribution of the image feature points. Through the above implementation, the region of interest in the scene is automatically analyzed from the identified distribution of image feature points in the scene image, and the guidance information is used to change the position of the image sensor in the scene space to track the region of interest in the scene.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a scene display guidance method, device, and storage medium. Background Technology

[0002] Augmented Reality (AR) is a technology that cleverly integrates virtual information with the real world. It widely utilizes various techniques such as multimedia, 3D modeling, real-time tracking and registration, intelligent interaction, and sensing. AR simulates computer-generated text, images, 3D models, music, and videos, and then applies this simulation to the real world, allowing the two types of information to complement each other, thereby "enhancing" the real world.

[0003] Augmented reality technology has been developing for many years, and we hope that one day we can use this technology in our daily lives and work to provide convenience and improve work efficiency. Currently, many work scenarios have emerged. With the advancement of technology, there are also global positioning methods based on visual high-precision maps, signal positioning, and other multi-sensor fusion. Various science fiction concepts such as digital twins and parallel worlds are gradually becoming possible.

[0004] In existing technologies, there may be situations where the number of feature points in the image acquired by the camera is too small to be successfully located, resulting in poor final results. Summary of the Invention

[0005] To address the aforementioned problems in the prior art, this application provides a scene display guidance method, device, and storage medium.

[0006] To address the technical problems existing in the prior art, this application provides a scene display guidance method, which includes: acquiring a scene image in a scene space using an image sensor; identifying image feature points in the scene image to obtain the distribution of the image feature points in the scene image; and determining guidance information for changing the position of the image sensor in the scene space based on the distribution of the image feature points in the scene image.

[0007] To address the technical problems existing in the prior art, this application provides a scene display guidance device, which includes a processor and a memory. The memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0008] To address the technical problems existing in the prior art, this application provides a computer-readable storage medium storing program instructions that, when executed by a processor, implement the above-described method.

[0009] Compared with existing technologies, the scene display guidance method of this application includes acquiring a scene image in a scene space using an image sensor; identifying image feature points in the scene image to obtain the distribution of image feature points in the scene image; and determining guidance information to change the position of the image sensor in the scene space based on the distribution of image feature points in the scene image. Through the above implementation method, the region of interest in the scene is automatically analyzed from the identified distribution of image feature points in the scene image, and the region of interest in the scene is tracked by changing the position of the image sensor in the scene space using the guidance information.

[0010] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic flowchart of an embodiment of the scene display guidance method provided in this application;

[0013] Figure 2 yes Figure 1 A schematic flowchart of an embodiment of step S101;

[0014] Figure 3 yes Figure 1 A schematic flowchart of an embodiment of step S102;

[0015] Figure 4 yes Figure 1 A schematic flowchart of an embodiment of step S103;

[0016] Figure 5 yes Figure 4 A schematic flowchart of an embodiment of step S403;

[0017] Figure 6 This is a schematic diagram of an embodiment of the scene display guidance device provided in this application;

[0018] Figure 7 This is a schematic diagram of the structure of an embodiment of the scene display guidance device provided in this application;

[0019] Figure 8 This is a schematic diagram of the structure of an embodiment of the computer storage medium provided in this application. Detailed Implementation

[0020] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be particularly noted that the following embodiments are for illustrative purposes only and do not limit the scope of the application. Similarly, the following embodiments are only some, not all, embodiments of the present application, and all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of the present application.

[0021] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0022] In the description of this application, it is necessary to specify that, unless otherwise expressly stated and limited, the terms "installation," "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or a connection through an intermediate medium. Those skilled in the art will understand the specific meanings of the above terms within the context of this application.

[0023] This disclosure relates to the field of augmented reality (AR). It involves acquiring image information of target objects in a real-world environment and then using various visual algorithms to detect or identify the relevant features, states, and attributes of these objects, thereby achieving an AR effect that combines virtual and real elements to suit specific applications. For example, target objects may include human features such as faces, limbs, gestures, and movements; objects such as signs and markers; or venues such as sand tables, display areas, or displayed items. Visual algorithms may include visual localization, SLAM, 3D reconstruction, image registration, background segmentation, keypoint extraction and tracking of objects, and pose or depth detection. Specific applications can include interactive scenarios related to real-world scenes or objects, such as guided tours, navigation, explanations, reconstruction, and virtual effect overlay displays, as well as human-related special effects processing, such as makeup enhancement, body enhancement, special effects displays, and virtual model displays.

[0024] Convolutional neural networks (CNNs) can be used to detect or identify the relevant features, states, and attributes of target objects. The aforementioned CNNs are network models obtained through training using deep learning frameworks.

[0025] Based on the above technical foundation, this application proposes a scene display guidance method, see [link to relevant documentation]. Figure 1 , Figure 1 This is a schematic flowchart of an embodiment of the scene display guidance method provided in this application. Specifically, it may include the following steps S101 to S103.

[0026] Step S101: Acquire scene images in the scene space using an image sensor.

[0027] An image sensor is located in the display device, and the image sensor can be a camera. The display device uses the image sensor to acquire scene images. The display device can be any electronic device that supports AR functionality, including but not limited to AR glasses, tablets, and smartphones. For example, the display device can present AR effects, which can be understood as displaying virtual objects integrated into the real-world scene. This can be done by directly rendering the content of the virtual objects and merging them with the real-world scene, such as presenting a set of virtual buildings whose display effect is that of real buildings placed in the real-world scene. Alternatively, it can be by merging the content of the virtual objects with the real-world scene image and then displaying the merged image. In this embodiment, the display device can be a mobile terminal or other display device.

[0028] The scene space can include a variety of environmental information. For example, the scene space can include an indoor scene space or an outdoor scene space. For example, when an outdoor scene space is included, it can include a road, buildings on both sides of the road, and plants between the road and the buildings. When an indoor scene space is included, it can include elevators, corridors, lighting fixtures, furniture, shops, etc.

[0029] Scene images are pictures taken by a camera in a scene space that contain part of the scene content.

[0030] Step S102: Identify image feature points in the scene image to obtain the distribution of image feature points in the scene image.

[0031] Image feature points can be corner points in a scene image. These feature points can be identified from the scene image using a pre-defined recognition algorithm. For example, when the image feature points include corner points, corner detection algorithms can be used to identify them. These algorithms include grayscale image-based corner detection, binary image-based corner detection, and contour curve-based corner detection. Grayscale image-based corner detection can be further divided into three categories: gradient-based, template-based, and template-gradient combination-based methods. Template-based methods primarily consider the grayscale changes of neighboring pixels, i.e., changes in image brightness, defining points with sufficiently large brightness contrast to their neighbors as corner points. Common template-based corner detection algorithms include the Kitchen-Rosenfeld corner detection algorithm, the Harris corner detection algorithm, the KLT corner detection algorithm, and the SUSAN corner detection algorithm. Among these, the SUSAN corner detection algorithm is characterized by its simplicity, accurate location, and strong noise resistance compared to other corner detection algorithms.

[0032] After identifying image feature points from the scene image, the distribution of image feature points in the scene image can be obtained based on the positions of all identified image feature points in the scene image. For example, most image feature points are distributed in the middle of the scene image, or distributed in other areas of the scene image.

[0033] Step S103: Based on the distribution of image feature points in the scene image, determine the guidance information for changing the position of the image sensor in the scene space.

[0034] In some scenarios, such as AR interactive games or AR navigation applications, location operations are required during application use. However, when there are few image feature points in the scene, location is difficult to achieve. In such cases, the user needs to move the image sensor to re-identify the scene image with more image feature points to finally achieve location. However, during an immersive user experience, without guidance on moving the image sensor, it is difficult to quickly align the image sensor with an area with more image feature points in the scene space, which will seriously affect the user experience.

[0035] Guidance information can be presented in the form of text or voice broadcast. For example, when a large number of image feature points are located in the scene image at the 12 o'clock position, guidance information can be issued to prompt the user to move the image sensor in the 12 o'clock direction, or to prompt the user to adjust the angle of the image sensor so that the image sensor faces the 12 o'clock direction. This allows for the prediction of areas with abundant feature points in the scene space based on the distribution of feature points in the scene image, and the guidance information can then prompt the user to move the image sensor towards these areas, thus displaying scene areas with more image feature points. This avoids the possibility that the image feature points in the current scene image do not meet the criteria, preventing subsequent positioning and other operations from being performed.

[0036] In one embodiment, the scene display guidance method includes: determining a virtual display space based on the position of the image sensor when acquiring a scene image; and displaying image feature points in the virtual display space.

[0037] A virtual display space can be established by acquiring a high-precision map. For example, a virtual display space can be constructed based on a high-precision map of the scene space. The high-precision map data can be images taken of the actual environment where the scene space is located. These images can be captured by intelligent vision sensors or other types of cameras. The number of images can be multiple, such as 10, 20, 50, etc. The more images, the more accurate the constructed virtual display space; however, this also means more time is required to construct the virtual space. The images can be 3D map data, which can be constructed from 3D point clouds.

[0038] The virtual display space can be determined by the position of the image sensor in the scene space, where the position of the virtual display space is the same as the position where the scene image is acquired by the sensor. For example, when a mobile terminal is used as a display device, the user is located in the scene space and holding the mobile terminal. When the mobile terminal is at a certain point in the scene space, a virtual display space can be generated within the field of view of the image sensor, and a scene image can be acquired at that point. The acquired scene image can then be used to analyze image feature points in the scene image. After obtaining the image feature points in the scene image, these feature points can be displayed at the corresponding positions in the virtual display space. This allows the user to easily understand what image feature points are through the virtual display space, reducing the learning cost for users using the virtual display space and enhancing the application experience to a certain extent, thus effectively increasing the fun of using the virtual display space for entertainment.

[0039] In one embodiment, the step of displaying image feature points in a virtual display space includes: comparing the spacing between image feature points in the scene image with a preset distance threshold; merging several image feature points with a spacing less than the preset distance threshold into fused feature points; and displaying the fused feature points and the image feature points together in the virtual display space.

[0040] When there are many image feature points, displaying them randomly in the scene image results in a poor final display effect. Therefore, after identifying image feature points in the scene image, the distance between them can be obtained. Then, several image feature points with a distance less than a preset threshold are merged into a fused feature point, and both the original image feature points and the fused feature point are displayed simultaneously. The representation of the fused feature point can differ from that of the original image feature points. For example, the color of the fused feature point can differ from the color of the original image feature points, and the color can be determined based on the number of merged image feature points; the more merged image feature points, the darker the color of the fused feature point. The shape of the fused feature point can also differ from that of the original image feature points; for example, the original image feature points are represented by circles, while the fused feature points are represented by triangles or other shapes.

[0041] In one embodiment, the scene display guidance method includes: generating a prompt animation according to guidance information; and displaying the prompt animation in a virtual display space.

[0042] When the guidance information includes instructions for the user to move the image sensor up, down, left, right, or in other directions, a prompt animation can be generated and displayed. This animation can be a dynamic visual prompt to guide the user to move the image sensor in the direction indicated by the guidance information, such as using an arrow pointing in a certain direction. Similarly, when the guidance information includes instructions for the user to tilt the image sensor up, down, left, right, or in other directions to change its orientation, a prompt animation can be generated and displayed. This animation can be a dynamic visual prompt to guide the user to rotate the image sensor in the direction indicated by the guidance information. Thus, by using animation to guide the user in moving the image sensor, the user experience is made more engaging.

[0043] See Figure 2 , Figure 2 yes Figure 1 A schematic diagram of an embodiment of step S101. Specifically, it includes the following steps S201 to S203.

[0044] Step S201: Use an image sensor to acquire multiple selected images in the scene space.

[0045] The selected images from multiple frames can be acquired by an image sensor at different points in the scene space. For example, when a mobile terminal is used as a display device, the user holds the mobile terminal in the scene space. The mobile terminal acquires the first selected image at a certain point in the scene space. However, the first selected image may not meet expectations, that is, it cannot be guaranteed that each selected image is a scene image that meets the conditions. In this case, the image sensor can be used to acquire other selected images at different points to select the scene image that best meets the conditions from multiple selected images.

[0046] Step S202: Identify image feature points in multiple selected images and obtain the number of feature points in each selected image.

[0047] Feature point recognition algorithms can be used to identify image feature points in each selected frame from multiple selected images, and the number of feature points in each selected frame can be calculated. For example, when image feature points include image corners, corner detection algorithms can be used to identify the image corners from the scene image. Examples of corner detection algorithms include corner detection based on grayscale images, corner detection based on binary images, and corner detection based on contour curves.

[0048] Step S203: Select images with a number of feature points greater than or equal to a preset threshold as scene images.

[0049] After identifying the number of feature points in each selected image frame, the number of feature points in each selected image frame can be compared with a preset threshold to obtain the comparison result. Then, at least one selected image frame with a number of feature points greater than or equal to the preset threshold is selected from the multiple selected images as the scene image, so that guidance information can be obtained accurately and conveniently using the selected scene image.

[0050] See Figure 3 , Figure 3 yes Figure 1 A schematic flowchart of an embodiment of step S102 is shown below. Specifically, it includes the following steps S301 to S304.

[0051] Step S301: Identify image feature points in the scene image.

[0052] Image feature points can be corner points in a scene image. These feature points can be identified from the scene image using a pre-defined recognition algorithm. For example, when the image feature points include corner points, corner detection algorithms can be used to identify them. These algorithms include grayscale image-based corner detection, binary image-based corner detection, and contour curve-based corner detection. Grayscale image-based corner detection can be further divided into three categories: gradient-based, template-based, and template-gradient combination-based methods. Template-based methods primarily consider the grayscale changes of neighboring pixels, i.e., changes in image brightness, defining points with sufficiently large brightness contrast to their neighbors as corner points. Common template-based corner detection algorithms include the Kitchen-Rosenfeld corner detection algorithm, the Harris corner detection algorithm, the KLT corner detection algorithm, and the SUSAN corner detection algorithm. Among these, the SUSAN corner detection algorithm is characterized by its simplicity, accurate location, and strong noise resistance compared to other corner detection algorithms.

[0053] Specifically, when the number of image feature points identified in the scene image is greater than the first preset threshold, the recognition accuracy of the image feature points can be improved, thereby reducing the number of image feature points identified; when the number of image feature points identified in the scene image is less than the second preset threshold, the recognition accuracy of the image feature points can be reduced, thereby increasing the number of image feature points identified, wherein the second preset threshold is less than the first preset threshold.

[0054] Step S302: Divide the scene image into multiple image regions according to a preset division rule.

[0055] The preset division rules include dividing the scene image into multiple image regions in the form of a nine-square grid, or dividing the scene image into multiple image regions in the form of a clock. In other embodiments, the preset division rules may also include other forms, which are not limited here.

[0056] In this case, steps S302 and S301 can be executed in any order. That is, the scene image can be divided into multiple image regions according to a preset division rule first, and then the image feature points in the scene image can be identified.

[0057] Step S303: Calculate the number of image feature points in each image region.

[0058] After identifying image feature points in the scene image and dividing the scene image into multiple image regions, the number of image feature points in each image region can also be determined. The number of image feature points in each image region is the sum of all image feature points in that region.

[0059] Step S304: Determine the distribution of image feature points in the scene image based on the number of image feature points in each image region.

[0060] After obtaining the number of image feature points in each image region, we can determine which image region has the most and which has the fewest identified image feature points. This allows us to use the distribution of image feature points in the scene image to perform subsequent steps, thereby improving the accuracy of the distribution of image feature points in the scene image.

[0061] In this embodiment, the step of determining guidance information for changing the position of the image sensor in the scene space based on the distribution of image feature points in the scene image (step S103) includes: comparing the number of image feature points in multiple image regions and determining the image region with the most image feature points in the scene image; determining the movement direction from the center position of the scene image to the image region with the most image feature points based on the position information; and determining guidance information for changing the position of the image sensor in the scene space based on the movement direction.

[0062] After calculating the number of image feature points in each image region, the number of image feature points in each image region can be compared to determine the image region with the highest number of at least one image feature point in the scene image. After determining the location information of the image region with the highest number of image feature points, guidance information can be issued to the user to change the orientation of the image sensor. For example, if the scene image is divided into nine image regions arranged in a 3x3 grid, and the image region with the highest number of image feature points is located in the upper left corner of the nine image regions, then the image region with the highest number of image feature points is located in the upper left corner of the scene image. At this time, the user can move towards the upper left corner based on the center position of the scene image to obtain the direction of movement, and thus obtain guidance information to change the position of the image sensor in the scene space. The guidance information at this time includes moving the image sensor towards the upper left corner or rotating the image sensor towards the upper left corner.

[0063] The above embodiment divides a scene image into multiple image regions after acquiring one, thereby determining the distribution of image feature points in the scene graphic. In other embodiments, guidance information can also be determined using multiple images. See also Figure 4 , Figure 4 yes Figure 1 A schematic flowchart of an embodiment of step S103. Specifically, it may include the following steps S401 to S403.

[0064] Step S401: Use an image sensor to acquire a prediction image at the location where the scene image is acquired, wherein the field of view of the image sensor when acquiring the prediction image is different from the field of view when acquiring the scene image.

[0065] Image sensors inherently possess a field of view, the size of which determines the sensor's field of view range. A larger field of view results in a wider field of view, and the image sensor can adjust its field of view within this range to acquire images. For example, if the image sensor's field of view ranges from 0 to 60 degrees, the sensor can be adjusted to acquire a first frame image at a 60-degree field of view. This first frame image can be used as the aforementioned scene image. Then, the sensor can be adjusted to acquire a second frame image at a 50-degree field of view, which can be used as the prediction image. Both the scene image and the prediction image can be acquired at the same location in the scene space.

[0066] It is important to note that the same position here does not refer to an absolutely identical point, but rather a range of positions. In other words, when the image sensor acquires the predicted image and the scene image, the positions of the two can fluctuate within a small range.

[0067] Step S402: Identify image feature points in the predicted image to obtain the distribution of image feature points in the predicted image.

[0068] Image feature points in the predicted image can be image corner points. Image feature points can be identified from the predicted image based on a preset recognition algorithm. For example, when the image feature points include image corner points, the image corner points can be identified from the predicted image through a corner detection algorithm. For example, corner detection algorithms include corner detection based on grayscale images, corner detection based on binary images, and corner detection based on contour curves.

[0069] After identifying image feature points from the predicted image, the distribution of image feature points in the predicted image can be obtained based on the positions of all identified image feature points in the predicted image. For example, most image feature points are distributed in the middle of the predicted image, or distributed in other areas of the predicted image.

[0070] Step S403: Based on the distribution of image feature points in the scene image and the distribution of image feature points in the predicted image, determine the guidance information for changing the position of the image sensor in the scene space.

[0071] The guidance information may be presented in the form of text information or voice broadcast. In this embodiment, the distribution of image feature points in the scene image and the distribution of image feature points in the prediction image are considered at the same time to more accurately predict the area with rich feature points in the scene space. The guidance information is used to prompt the user to move the image sensor to the area with rich feature points so as to display the scene area with more image feature points. This avoids the possibility that the image feature points in the current scene image do not meet the conditions, which may lead to the inability to perform subsequent positioning and other operations.

[0072] See Figure 5 , Figure 5 yes Figure 4 A schematic flowchart of an embodiment of step S403. Specifically, it may include the following steps S501 to S506.

[0073] Step S501: Based on the distribution of image feature points in the predicted image, determine the first position in the predicted image, wherein the first position is the point in the image region with the largest number of image feature points in the predicted image.

[0074] In this embodiment, the method for determining the first position in the predicted image can be the same as... Figure 3 The illustrated embodiment determines the distribution of image feature points in the scene image in a similar manner. That is, image feature points in the prediction image can be identified first; then the prediction image can be divided into multiple image regions according to a preset division rule; then the number of image feature points in each image region can be calculated; the number of image feature points in multiple image regions can be compared to determine the image region with the most image feature points; then a point can be selected from the image region with the most image feature points as the first position. For example, the first position can be any point selected from the image region; or an image feature point selected from the image region; or the center of the image region selected from the image region.

[0075] Step S502: Based on the distribution of image feature points in the scene image, determine the second position in the scene image, wherein the second position is the point in the image region with the largest number of image feature points in the scene image.

[0076] In this embodiment, the method for determining the second position in the scene image can be the same as... Figure 3The illustrated embodiment determines the distribution of image feature points in the scene image in a similar manner. That is, image feature points in the scene image can be identified first; then the scene image can be divided into multiple image regions according to a preset division rule; then the number of image feature points in each image region can be calculated; the number of image feature points in multiple image regions can be compared to determine the image region with the most image feature points; then a point can be selected from the image region with the most image feature points as the second position. For example, the second position can be any point selected from the image region; or an image feature point selected from the image region; or the center of the image region selected from the image region, etc.

[0077] Step S503: Map the first position in the predicted image to the scene image.

[0078] In this embodiment, the field of view of the image sensor when acquiring the predicted image can be smaller than the field of view when acquiring the scene image; and the image sensor acquires the predicted image at the location where the scene image is acquired, meaning that all image content in the predicted image can be reflected in the scene image. After determining the first position, the first position can be mapped onto the scene image to obtain the first position in the scene image.

[0079] Step S504: Determine the third position in the scene image based on the mapped first position and second position.

[0080] The first and second positions in the mapped scene image are located at different points in the scene image, meaning they are spaced apart. A third position can then be determined based on the first and second positions. The third position could be the midpoint of the line connecting the first and second positions. Alternatively, the third position can be determined based on the weights of the scene image and the predicted image. For example, if the weight of the scene image is greater than the weight of the predicted image, the determined third position would lie on the line connecting the first and second positions, but be closer to the second position.

[0081] Step S505: Using the third position and the center position of the scene image, determine the prediction direction from the center position of the scene image to the third position.

[0082] After determining the third position, the predicted direction can be based on the direction from the center of the scene image to the third position. For example, if the third position is located at the 12 o'clock position of the center position, the predicted direction is the direction of movement from the center position to the 12 o'clock position.

[0083] Step S506: Determine guidance information for changing the position of the image sensor in the scene space based on the predicted direction.

[0084] After determining the prediction direction, guidance information can be determined based on the prediction direction, so that the user can change the position of the image sensor according to the guidance information. Thus, the third position is determined by the first position and the second position, and the prediction direction is more accurate. This allows the user to change the position of the image sensor according to the guidance information, so that the image sensor faces the area with more feature points in the scene space. This avoids the possibility that the image feature points in the current scene image do not meet the conditions, which may prevent subsequent operations (such as positioning operations based on feature points) from being performed.

[0085] The scene display guidance method in this embodiment can be applied to a scene display guidance device. The scene display guidance device of this application can be a server, a mobile device, or a system in which the server and mobile device cooperate with each other. Accordingly, the various parts of the mobile device, such as various units, sub-units, modules, and sub-modules, can all be set in the server, all in the mobile device, or separately in the server and the mobile device.

[0086] Furthermore, the aforementioned server can be either hardware or software. When the server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software programs or software modules, such as software or software modules used to provide distributed server functionality, or as a single software program or software module; no specific limitations are made here.

[0087] To implement the scene display guidance method of the above embodiments, this application provides a scene display guidance device. See also... Figure 6 , Figure 6 This is a schematic diagram of an embodiment of the scene display guidance device 60 provided in this application.

[0088] Specifically, the scene display guidance device 60 may include: an acquisition module 61, an identification module 62, and a determination module 63.

[0089] The acquisition module 61 is used to acquire scene images in the scene space using an image sensor.

[0090] The recognition module 62 is used to identify image feature points in the scene image and obtain the distribution of image feature points in the scene image.

[0091] The determination module 63 is used to determine guidance information for changing the position of the image sensor in the scene space based on the distribution of image feature points in the scene image.

[0092] The above scheme determines guidance information based on the distribution of image feature points in the identified scene image, so that the user can change the position of the image sensor in the scene space through the guidance information, thereby avoiding the possibility that the image feature points in the current scene image do not meet the conditions, which may prevent subsequent operations from being performed.

[0093] In one embodiment of this application, Figure 6 The various modules in the scene display guidance device 60 shown can be individually or entirely combined into one or more units, or some of the units can be further divided into multiple functionally smaller sub-units to achieve the same operation without affecting the technical effects of the embodiments of this application. The above modules are based on logical function division. In practical applications, the function of one module can also be implemented by multiple units, or the function of multiple modules can be implemented by one unit. In other embodiments of this application, the scene display guidance device 60 may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0094] The above method is applied to scene display guidance devices. Please refer to [link / reference] for details. Figure 7 , Figure 7 This is a schematic diagram of an embodiment of the scene display guidance device provided in this application. The scene display guidance device 70 in this embodiment includes a processor 71 and a memory 72. The memory 72 stores a computer program, and the processor 71 executes the computer program to implement the above-described scene display guidance method.

[0095] The processor 71 can be an integrated circuit chip with signal processing capabilities. The processor 71 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor.

[0096] for Figures 1-5 The scene display guidance method shown in the embodiment can be presented in the form of a computer program. This application proposes a computer storage medium for carrying the computer program. Please refer to [link to relevant documentation]. Figure 8 , Figure 8 This is a schematic diagram of a computer storage medium according to an embodiment of the present application. The computer storage medium 80 in this embodiment includes a computer program 81, which can be executed to implement the above-described scene display guidance method.

[0097] In this embodiment, the computer storage medium 80 can be a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, or a medium that can store program instructions. Alternatively, it can be a server that stores the program instructions. The server can send the stored program instructions to other devices for execution, or it can execute the stored program instructions itself.

[0098] Furthermore, if the aforementioned functions are implemented as software functions and sold or used as independent products, they can be stored in a mobile terminal-readable storage medium. That is, this application also provides a storage device storing program data, which can be executed to implement the methods of the above embodiments. This storage device can be, for example, a USB flash drive, an optical disc, or a server. In other words, this application can be embodied in the form of a software product, which includes several instructions to cause a smart terminal to execute all or part of the steps of the methods described in the various embodiments.

[0099] In the description of this application, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0100] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0101] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as will be understood by those skilled in the art to which embodiments of this application pertain.

[0102] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (which may be a personal computer, server, network device, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0103] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A scene display guidance method characterized by comprising: The method comprises: acquiring a scene image in a scene space by using an image sensor; identifying image feature points in the scene image to obtain a distribution of the image feature points in the scene image; determining guiding information for changing a position of the image sensor in the scene space based on the distribution of the image feature points in the scene image; wherein the determination of the guiding information for changing the position of the image sensor in the scene space based on the distribution of the image feature points in the scene image comprises: acquiring a prediction image at a position for acquiring the scene image by using the image sensor, wherein a field of view angle of the image sensor when acquiring the prediction image is different from a field of view angle when acquiring the scene image; identifying image feature points in the prediction image to obtain a distribution of the image feature points in the prediction image; determining a first position in the prediction image based on the distribution of the image feature points in the prediction image, wherein the first position is a point position in an image region with the largest number of image feature points in the prediction image; determining a second position in the scene image based on the distribution of the image feature points in the scene image, wherein the second position is a point position in an image region with the largest number of image feature points in the scene image; mapping the first position in the prediction image to the scene image; determining a third position in the scene image based on the mapped first position and the second position, wherein the third position is located on a line connecting the first position and the second position; determining a prediction direction from a center position of the scene image to the third position by using the third position and the center position of the scene image; determining the guiding information for changing the position of the image sensor in the scene space based on the prediction direction.

2. The method of claim 1, wherein, The identification of the image feature points in the scene image to obtain the distribution of the image feature points in the scene image comprises: identifying the image feature points in the scene image; dividing the scene image into a plurality of image regions according to a preset division rule; calculating the number of the image feature points in each image region respectively; determining the distribution of the image feature points in the scene image based on the number of the image feature points in each image region.

3. The method of claim 2, wherein, The determination of the guiding information for changing the position of the image sensor in the scene space based on the distribution of the image feature points in the scene image comprises: determining an image region with the largest number of image feature points in the scene image; acquiring position information of the image region with the largest number of image feature points in the scene image; determining a moving direction of a center position of the scene image to the image region with the largest number of image feature points based on the position information; determining the guiding information for changing the position of the image sensor in the scene space based on the moving direction.

4. The method of claim 1, wherein, The acquisition of the scene image in the scene space by using the image sensor comprises: acquiring a plurality of selected images in the scene space by using the image sensor; Identify image feature points of multiple frames of the selected images, to obtain a number of feature points in each frame of the selected images; Select a selected image with the number of feature points greater than or equal to a preset number threshold as the scene image.

5. The method according to any one of claims 1 to 4, characterized in that, The method comprises: Determine a virtual display space based on a position of the image sensor when the scene image is acquired; Display the image feature points in the virtual display space.

6. The method of claim 5, wherein, The display of the image feature points in the virtual display space comprises: Compare a distance between image feature points in the scene image with a preset distance threshold; Merge a plurality of image feature points with a distance less than the preset distance threshold into a fusion feature point; Display the fusion feature point and the image feature points together in the virtual display space.

7. The method of claim 6, wherein, The method comprises: Generate a prompt animation according to the guidance information; Display the prompt animation in the virtual display space.

8. A scene display guide device characterized by comprising: Comprise: A processor and a memory, the memory storing a computer program, the processor being configured to execute the computer program to implement the method of any one of claims 1 to 7.

9. A computer-readable storage medium having stored thereon program instructions, wherein, The program instructions are executed by the processor to implement the method of any one of claims 1 to 7.

Images