Method, apparatus and electronic device for acquiring attention information of object
By displaying 3D virtual images in a virtual scene, the focus of user attention can be determined, solving the problem of accurately locating the user's focus in existing technologies, and achieving precise acquisition of user attention data and product optimization.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- LENOVO (BEIJING) LTD
- Filing Date
- 2022-02-08
- Publication Date
- 2026-05-29
AI Technical Summary
Existing technologies make it difficult to accurately identify the product aspects that users care about, resulting in insufficient precision in product optimization.
By displaying a 3D virtual image in a virtual scene, the target object presented in the 3D virtual image within the display area is identified, and the focus of attention is determined based on the target object. Digital twin technology is then used to map the physical object onto the virtual scene, reflecting the part that the user is interested in.
It enables accurate identification of user focus areas, provides precise user attention data, and supports product optimization and big data analysis.
Smart Images
Figure CN114445603B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, and more specifically, to a method, apparatus, and electronic device for acquiring attention information of an object. Background Technology
[0002] After a product is released, it is often necessary to collect user feedback and optimize or maintain the parts of the product that users are most concerned about.
[0003] Currently, user feedback on products is generally obtained through questionnaires or product reviews. However, this feedback covers a wide range of product components, making it difficult to accurately identify the areas that most users care about, which hinders more effective product optimization. Summary of the Invention
[0004] This application provides a method, apparatus, and electronic device for obtaining information of interest to an object.
[0005] One method for obtaining the attention information of an object includes:
[0006] Displaying a stereoscopic virtual image in a virtual scene, wherein the stereoscopic virtual image is a stereoscopic image of a physical object mapped to a virtual object in the virtual scene;
[0007] Identify the target object presented within the display area in the stereoscopic virtual image;
[0008] Based on the target object, a focus object in the stereoscopic virtual image is determined, wherein the focus object belongs to at least a portion of the stereoscopic virtual image.
[0009] In one possible implementation, determining the target object presented in the display area within the stereoscopic virtual image includes:
[0010] Identify the target object within the sampling area of the display area in the stereoscopic virtual image.
[0011] In another possible implementation, determining the target object within the sampling region of the display area in the stereoscopic virtual image includes:
[0012] Determine the camera position of the virtual camera in the virtual scene at the current moment;
[0013] Using the camera position as the viewpoint, the target object within the sampling area of the display area in the stereoscopic virtual image is determined.
[0014] In another possible implementation, determining the target object within the sampling area of the display area in the stereoscopic virtual image, using the camera position as the viewpoint position, includes:
[0015] Determine the image mapping area within the display area onto which the stereoscopic virtual image is mapped;
[0016] Starting from the camera position, determine multiple rays that pass through multiple sampling points in the display area from the starting point;
[0017] By combining the sub-mapping areas in the image mapping area that are picked up by the multiple rays, the target object in the stereoscopic virtual image that is picked up by the multiple rays is determined.
[0018] In another possible implementation, the target object includes: at least one target object region in the stereoscopic virtual image and display features of the at least one target object region;
[0019] The step of determining the focus object in the stereoscopic virtual image based on the target object includes:
[0020] Based on the display characteristics of the at least one target object region, determine the focus object from the at least one target object region.
[0021] In yet another possible implementation, before determining the focus object in the stereoscopic virtual image, the method further includes:
[0022] Determine the distribution characteristics of each region within the sampling area of the target object;
[0023] The step of determining the focus object in the stereoscopic virtual image based on the target object includes:
[0024] Based on the distribution characteristics of each region of the target object within the sampling area, the focus of attention object in the stereoscopic virtual image is determined.
[0025] In another possible implementation, after determining the focus object in the stereoscopic virtual image, the method further includes:
[0026] Increase the number of times the focus object is viewed;
[0027] Store the number of times the focus is viewed.
[0028] In another possible implementation, after determining the focus object in the stereoscopic virtual image, the method further includes:
[0029] Obtain the duration of attention for the object of focus;
[0030] The cumulative attention duration of the focus object is increased by the attention duration.
[0031] One of the devices for acquiring attention information of an object includes:
[0032] An image display unit is used to display a stereoscopic virtual image in a virtual scene, wherein the stereoscopic virtual image is a stereoscopic image of a physical object mapped to a virtual object in the virtual scene;
[0033] The target determination unit determines the target object presented in the display area of the stereoscopic virtual image;
[0034] A focus determination unit is used to determine a focus object in the stereoscopic virtual image based on the target object, wherein the focus object belongs to at least a portion of the stereoscopic virtual image.
[0035] One of the electronic devices includes at least a memory and a processor;
[0036] The processor is configured to execute the method for obtaining the attention information of an object as described in any one of the claims of this application;
[0037] The memory is used to store the programs required for the processor to perform operations.
[0038] As can be seen from the above scheme, this application, by displaying a stereoscopic virtual image in a virtual scene, determines the target object presented within the display area of the stereoscopic virtual image, and thus determines the user's focus object in the stereoscopic virtual image based on the target object. Since the stereoscopic virtual image is a stereoscopic image corresponding to a physical object mapped onto a virtual object in the virtual scene, the focus object in the stereoscopic virtual image can reflect the part of the physical object that the user is interested in. Therefore, it is possible to obtain the user's focus on physical objects such as products more accurately, and then optimize physical objects such as products based on the user's attention to them. Attached Figure Description
[0039] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0040] Figure 1 A flowchart illustrating a method for obtaining information of interest of an object provided in an embodiment of this application;
[0041] Figure 2A schematic diagram of a stereoscopic virtual image presented in the display area provided in an embodiment of this application;
[0042] Figure 3 Another schematic diagram of a stereoscopic virtual image presented in the display area provided in the embodiments of this application;
[0043] Figure 4 A schematic diagram illustrating the relationship between a stereoscopic virtual image presented within a display area and a sampling area, provided in an embodiment of this application.
[0044] Figure 5 A flowchart illustrating a method for obtaining information of interest of an object provided in an embodiment of this application;
[0045] Figure 6 This is a flowchart illustrating a process for determining a target object within a sampling area in a stereoscopic virtual image, as described in this application embodiment.
[0046] Figure 7 This is a schematic diagram of the image mapping area of a stereoscopic virtual image in an embodiment of this application;
[0047] Figure 8 This diagram shows the location of the sampling points in the display area;
[0048] Figure 9 This diagram shows the areas picked up by the ray within the display area;
[0049] Figure 10 A schematic diagram of the structural composition of a device for obtaining information of interest of an object provided in an embodiment of this application;
[0050] Figure 11 This is a schematic diagram of the composition structure of an electronic device provided in an embodiment of this application.
[0051] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings are used to distinguish similar parts and are not necessarily used to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in a sequence other than that illustrated herein. Detailed Implementation
[0052] The solution proposed in this application can be applied to product display platforms or e-commerce platforms to display three-dimensional virtual images of products or objects in virtual scenes. It can determine the user's focus on the physical object corresponding to the three-dimensional virtual object based on the user's focus on the three-dimensional virtual image, thereby providing accurate and reliable user attention data for optimizing physical objects such as products or conducting big data analysis.
[0053] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.
[0054] like Figure 1 The diagram illustrates a flowchart of a method for obtaining information about an object's interests, as provided in an embodiment of this application. This method can be applied to electronic devices, such as servers (e.g., servers of e-commerce platforms or product display platforms). The electronic device can also be a terminal device, such as a mobile phone or laptop.
[0055] In one possible scenario, the terminal device can establish a communication connection with a server that provides virtual scene data. For example, the terminal device can establish a communication connection with an e-commerce platform or product display platform to obtain data of stereoscopic virtual images in the virtual scene from the server of the e-commerce platform or product display platform, and so on.
[0056] In another possible scenario, the terminal device can be a standalone computer device that can store data such as the virtual scene to be presented, without any restrictions.
[0057] The method in this embodiment may include:
[0058] S101 displays a 3D virtual image in a virtual scene.
[0059] Among them, the stereoscopic virtual image is the stereoscopic image corresponding to the virtual object in the virtual scene that is mapped from the physical object.
[0060] For example, a three-dimensional virtual image of a virtual object is a digital twin of a physical object; that is, a three-dimensional virtual object is a three-dimensional virtual image created for a physical object using digital twin technology. The composition and shape of this three-dimensional virtual image are consistent with the composition and shape of the real physical object, and the size of the three-dimensional virtual object can also be proportional to the size of the real physical object.
[0061] For example, to obtain information about a user's attention to different parts of a product, a virtual scene containing a 3D virtual image of the product can be constructed. The composition and shape of the 3D virtual image can be consistent with the product, while the size of the 3D virtual image can be the same as the product's size or at a predetermined ratio.
[0062] The type of stereoscopic virtual image can vary depending on the spatial dimension of the virtual scene. For example, a stereoscopic virtual image can be a three-dimensional virtual image or a virtual image of other dimensions, without any restrictions.
[0063] It is understandable that displaying a stereoscopic virtual image of a virtual scene is actually the process of mapping the stereoscopic virtual image to a two-dimensional display area, that is, the process of converting each pixel in the stereoscopic virtual image from the world coordinate system to the screen coordinate system. This application does not impose any restrictions on the specific implementation process.
[0064] In this application, the virtual object can be displayed on the screen of the electronic device, or it can be projected by the electronic device onto a display area outside the electronic device or into a display device, without limitation.
[0065] For example, taking electronic devices as terminal devices, the terminal devices can output stereoscopic virtual images to their displays, or output them to other displays or project them onto a designated display area (such as the display area of a wall or screen).
[0066] For example, if the electronic device is a server, then the server can output the stereoscopic virtual image to the terminal device so that the stereoscopic virtual image can be displayed on the terminal device's screen.
[0067] S102, Determine the target object presented in the display area in the stereoscopic virtual image.
[0068] This display area is used to display stereoscopic virtual images. For example, when an electronic device displays a stereoscopic virtual image on its own screen or the screen of another electronic device, this display area can be the screen (or the display area of the screen); if the electronic device projects a stereoscopic virtual image onto a projection screen or wall, then the image display area on the projection screen or wall is this display area.
[0069] It is understood that when a stereoscopic virtual image is displayed within a two-dimensional display area, a portion of the stereoscopic virtual image is presented within the display area and can be seen by the user; while the portion of the stereoscopic virtual image not presented within the display area is invisible to the user. In this application, the target object can belong to the target image area presented within the display area of the stereoscopic virtual image, or it can belong to the constituent objects contained within the target image area presented within the display area of the stereoscopic image.
[0070] For example, the target object can be a target image region presented in the display area of a stereoscopic virtual image, or all the constituent objects contained in that image region.
[0071] For example, if the stereoscopic virtual image is a stereoscopic virtual image of a laptop, then the stereoscopic virtual image presented in the display area may only contain a portion of the image area of the laptop's screen and keyboard area. In this case, the target object represents the portion of the stereoscopic virtual image of the laptop that is mapped into the display area, or it can represent the screen and keyboard components of the stereoscopic virtual image.
[0072] For example, the target object can be a portion of the image area presented within the display area of a stereoscopic virtual image, or a component object within that portion of the image area. For instance, this application can combine the positional distribution characteristics of the target image area presented within the display area of the stereoscopic virtual image to determine a portion of the image area that is likely to attract user attention, or a component object contained within that portion of the image area.
[0073] There are several possible ways to determine the target image region that appears within the display area in a stereoscopic virtual image.
[0074] For example, in one possible implementation, the target image portion of the stereoscopic virtual image that can be displayed in the display area can be determined by combining the current viewpoint position in the virtual scene.
[0075] In this context, the viewpoint position in the virtual scene is a virtual viewing point within the virtual scene, which serves as a reference point for mapping the stereoscopic virtual image in the virtual scene onto a two-dimensional screen plane.
[0076] As an alternative approach, a virtual camera can be set up within the virtual scene. The initial position of the virtual camera is fixed, but its position changes continuously as the user rotates or zooms the virtual scene. Based on this, the current spatial position of the virtual camera within the virtual scene can be used as the viewpoint position to determine the portion of the target image that can be displayed within the display area of the stereoscopic virtual image.
[0077] It should be noted that the virtual camera is not visible in the virtual scene; it is merely a fictional reference point used to determine the viewpoint position and target objects.
[0078] S103, Based on the target object, determine the focus object in the stereoscopic virtual image.
[0079] The object of focus is at least a part of the stereoscopic virtual image.
[0080] It is understandable that the focus of attention can be the image region in the stereoscopic virtual image that the user is paying attention to, or it can be the components within the image region that the user is paying attention to.
[0081] It is understandable that all the target objects presented in the display area of a stereoscopic virtual image may be objects that the user needs to focus on. In this case, the target objects can be the focus of attention.
[0082] In some cases, users may only focus on a portion of the presented target object. Therefore, it is necessary to combine some characteristics of the target object to determine the focus of the user's attention.
[0083] For example, in one possible implementation, the identified target object may consist of multiple parts. Specifically, the target object may include at least one target object region in a stereoscopic virtual image and the display features of the at least one target object region.
[0084] The target object region is either the image region or the component region that makes up the target object.
[0085] The display characteristics of the target object area can include: the display position of the target object area within the display area and the area ratio of the target object area to the image display area of the stereoscopic virtual image, etc., without any restrictions. The image display area of the stereoscopic virtual image can also be called the image mapping area, which is the display area occupied by the stereoscopic virtual image within the mapped display area.
[0086] It is understood that the display characteristics of a target object area can reflect the degree to which it is likely to attract user attention. Based on this, this application can determine the focus object from the at least one target object area based on its display characteristics.
[0087] For example, the target object area located in the center of the display area can be identified as the focus of attention. Alternatively, at least one target object area with a relatively large area ratio can be identified as the focus of attention.
[0088] For example, taking a stereoscopic virtual image as a stereoscopic virtual object of a laptop computer, assuming that the front part of the laptop computer is presented in the stereoscopic virtual image of the laptop computer within the display area, such as the target object presented within the display area including the screen area and the keyboard area. Based on this, this application can obtain the positions of the screen area and the keyboard area within the display area, as well as the area ratio of the screen area and the keyboard area within the image display area of the stereoscopic virtual image of the laptop computer, etc.
[0089] If the screen area is positioned closer to the center of the display area than the keyboard area, and the screen area occupies a much larger area than the keyboard area, then users are more likely to focus on the screen area, and the screen can be made the focus of user attention.
[0090] For example, if the keyboard area and the screen area are both located in the center of the display area, and the difference in the area ratio of these two areas is very small, then the user may pay attention to both areas at the same time. In this case, the keyboard area and the screen area (or a virtual keyboard and screen) can both be used as the focus of attention.
[0091] Understandably, since 3D virtual images are mappings of real physical objects into a virtual scene, the focus of a user's attention on a 3D virtual object can reflect the part of the physical object that the user is interested in, thus accurately identifying the user's main focus on the physical object. Based on this, collecting data on the focus of attention from a large number of different users on physical objects can be used for big data analysis to predict user behavior or obtain specific analytical results, etc., without any limitations.
[0092] As can be seen from the above scheme, this application, by displaying a stereoscopic virtual image in a virtual scene, determines the target object presented within the display area of the stereoscopic virtual image, and thus determines the user's focus object in the stereoscopic virtual image based on the target object. Since the stereoscopic virtual image is a stereoscopic image corresponding to a physical object mapped onto a virtual object in the virtual scene, the focus object in the stereoscopic virtual image can reflect the part of the physical object that the user is interested in. Therefore, it is possible to obtain the user's focus on physical objects such as products more accurately, and then optimize physical objects such as products based on the user's attention to them.
[0093] To facilitate understanding of the solution in this application, the following explanation will focus on several possible scenarios for determining the target object presented in the display area within the stereoscopic virtual image.
[0094] In one possible scenario, all objects presented within the display area in the stereoscopic virtual image can be identified as the target objects.
[0095] As mentioned earlier, the target image portion presented within the display area of the stereoscopic virtual image can be identified as the target object. Alternatively, after identifying the target image portion, each constituent object contained within the target image portion can be identified as the target object.
[0096] In this possible implementation, all objects within the display area are considered objects that may be of interest to the user.
[0097] For example:
[0098] like Figure 2 and Figure 3 The diagrams show a schematic representation of a stereoscopic virtual image displayed in the display area.
[0099] Figure 2 and Figure 3 The following is an example illustrating the mapping of a 3D virtual image on a laptop computer to the display area.
[0100] exist Figure 2 In this context, the portion of the stereoscopic virtual image of the laptop computer presented within the display area 201 may include the screen image area 201 and the keyboard image area 203 of the stereoscopic virtual image of the laptop computer. Therefore, both the screen image area and the keyboard image area can be used as target objects.
[0101] Considering that the screen image area represents the screen (virtual screen) in the stereoscopic virtual image that makes up the laptop, similarly, the keyboard image area represents the keyboard (or virtual keyboard) in the stereoscopic virtual image, which can also be the keyboard component and screen component that make up the stereoscopic virtual image as the target object.
[0102] And in Figure 3 In the image, the 3D virtual image of the laptop is displayed in the side portion of display area 301. Figure 3 As can be seen, the portion of the 3D virtual image of the laptop computer that is displayed within the display area may include: the back panel image area 302, the chassis image area 303, and the keyboard image area 304. Therefore, the back panel image area 302, the chassis image area 303, and the keyboard image area 304 can all be used as target objects.
[0103] In practical applications, after a stereoscopic virtual image is displayed, the user may only focus on a portion of the target image area. Therefore, to accurately identify the object within the stereoscopic virtual image that the user is focusing on, it is necessary to determine the target object within key areas of the stereoscopic virtual image displayed on the screen.
[0104] For example, in another possible scenario, this application can identify a target object within a sampling area presented in the display area of a stereoscopic virtual image.
[0105] The sampling area can be a specific area within the display area that the user is focusing on.
[0106] For example, the sampling area could be a portion of the display area located in the center. See also... Figure 4 The sampling area 402, set in the display area 401, is located in the central part of the display area. In this case, the only objects within the sampling area in the stereoscopic virtual image of the laptop are the screen image area 403 and the keyboard image area 404. The chassis image area 405 is not within the sampling area and therefore will not be considered as a target object.
[0107] Understandably, after displaying a stereoscopic virtual image, users will generally rotate or scale it according to their needs, so that the parts of the stereoscopic virtual image that the user is most interested in are located in the center of the display area or other important areas of interest. Therefore, identifying the object part of the sampling area presented in the display area of the stereoscopic virtual image as the target object can more accurately reflect the part of the stereoscopic virtual image that the user is interested in.
[0108] The following section describes one possible implementation method where the object portion of the stereoscopic virtual image presented in the sampling area of the display area is taken as the target object, and introduces the solution of this application.
[0109] like Figure 5 As shown, it illustrates another flowchart of obtaining the attention information of an object according to this application. The method of this embodiment can be applied to the aforementioned electronic device.
[0110] The method in this embodiment includes:
[0111] S501 displays stereoscopic virtual images in a virtual scene.
[0112] Among them, the stereoscopic virtual image is the stereoscopic image corresponding to the virtual object in the virtual scene that is mapped from the physical object.
[0113] Displaying a stereoscopic virtual image from a virtual scene onto the display area is essentially the process of mapping a three-dimensional or multi-dimensional stereoscopic virtual image onto a two-dimensional display area.
[0114] To facilitate understanding, we will use one implementation method as an example. Taking a virtual camera set in a virtual scene as an example, the virtual camera has a set initial spatial position. When displaying the stereoscopic virtual image to the display area, a camera coordinate system can be constructed based on the initial spatial position. Then, by combining the camera coordinate system and coordinate mapping relationships, the stereoscopic virtual image can be mapped onto the display area.
[0115] After the 3D virtual image is displayed, the camera position (spatial position in the world coordinate system) of the virtual camera in the virtual scene will change accordingly based on the user's adjustments such as rotation or scaling of the 3D virtual image. Consequently, the camera coordinate system constructed based on the camera position of the virtual camera will also change.
[0116] The following describes the process of displaying a stereoscopic virtual image using a camera coordinate system constructed by combining the camera position of the virtual camera:
[0117] First, by combining the camera coordinate system and the world coordinate system of the virtual scene, a first transformation matrix is determined to transform from the world coordinate system to the camera coordinate system. Based on the first transformation matrix, the world coordinates of each pixel in the stereoscopic virtual image are transformed into coordinates in the camera coordinate system.
[0118] Secondly, by combining the camera coordinate system, a second transformation matrix is determined to transform the camera coordinate system to the standardized device coordinate system. Based on the second transformation matrix, the coordinates of each pixel in the stereoscopic virtual image in the camera coordinate system are converted to standardized device coordinates.
[0119] Finally, according to the mapping relationship from standardized device coordinates to screen coordinates, the standardized device coordinates of each pixel in the stereoscopic virtual image are converted into screen coordinates in the screen coordinate system, and then the stereoscopic virtual image is displayed in the display area by combining the screen coordinates of each pixel in the stereoscopic virtual image.
[0120] For example, if the normalized device coordinates (x1, y1, z1) of a pixel in a 3D virtual image of a virtual object are known, the screen coordinates (x2, y2) can be obtained by the following formula:
[0121] x2 = (0.5 + x1 / 2) * w;
[0122] y2 = (0.5 - y1 / 2) * h;
[0123] Where w is the screen width, which is the width of the display area used to display the stereoscopic virtual image. For example, if the stereoscopic virtual image of a virtual object is displayed on the screen, the screen width can be the display width of the screen. If the stereoscopic virtual image of a virtual object is projected onto other types of display areas through projection or other means, the screen width is the width of the display area used to display the projected image. h is the screen height, which can be the display height of the screen or the height of the display area.
[0124] Of course, in practical applications, there are multiple possible implementations for mapping stereoscopic virtual images to the screen coordinate system, which is based on the camera coordinate system corresponding to the virtual camera. This application does not impose any restrictions on this.
[0125] S502, determine the camera position of the virtual camera in the virtual scene at the current moment.
[0126] As mentioned above, the position of the virtual camera in the virtual scene will change as the user adjusts the 3D virtual image, etc. Therefore, in order to determine the objects in the display area or the area to be used, it is necessary to first determine the current position of the virtual camera.
[0127] S503 uses the camera position as the viewpoint to determine the target object within the sampling area of the display area in the stereoscopic virtual image.
[0128] In this context, the viewpoint position in the virtual scene is a virtual viewing point within the virtual scene, which is the user's perspective position from which the virtual user can view the virtual scene.
[0129] Correspondingly, it is possible to simulate the target object within the sampling area when viewing the stereoscopic virtual image from that viewpoint.
[0130] The following example illustrates the specific implementation of determining the target object within the sampling region. Figure 6 As shown, it illustrates a flowchart for determining a target object within a sampling area in a stereoscopic virtual image, which may include:
[0131] S601, determine the image mapping area within the display area to which the stereoscopic virtual image is mapped.
[0132] The image mapping area is the display area within the display area after the stereoscopic virtual image is mapped onto the display area.
[0133] This image mapping area can be the smallest area in the display area that can completely contain the stereoscopic virtual image.
[0134] For example, in one possible implementation, the screen coordinates of each pixel in the stereoscopic virtual image can be determined. Then, from the screen coordinates of each pixel in the stereoscopic virtual image, the maximum and minimum values of the x-axis and y-axis can be determined. Using these maximum and minimum values on the x-axis and y-axis, the image mapping area of the stereoscopic virtual image in the display area can then be determined.
[0135] For example, such as Figure 7 As shown, Figure 7 Let's take a three-dimensional virtual image as a spatial cube as an example.
[0136] After determining the screen coordinates of each pixel in the spatial cube projected onto the display area, it can be seen that in the screen coordinate system of the display area, the minimum and maximum values of each pixel in the displayed spatial cube 701 on the x-axis are x0 and x1, respectively, while the maximum and minimum values on the y-axis are y0 and y1, respectively. Combining the maximum and minimum values of the displayed spatial cube on the x-axis and y-axis, the smallest square region 702 containing the spatial cube 701 can be constructed. The screen coordinates of the four vertices of the square region 702 are (x0, y0), (x1, y0), (x0, y1), and (x1, y1), respectively.
[0137] Of course, the area enclosed by the outline of the stereoscopic virtual image in the display area can also be directly defined as the image mapping area of the stereoscopic virtual image. In practical applications, there are many possibilities for determining the image mapping area of the stereoscopic virtual image in the display area, and there are no restrictions on this.
[0138] S602, taking the camera position as the starting point, determines multiple rays that pass through multiple sampling points in the display area from the starting point.
[0139] In this context, the area formed by multiple sampling points within the display area can be considered the sampling area within the display area. Correspondingly, these multiple sampling points can represent the parts of the display area that the user is primarily focused on.
[0140] The number of sampling points and their positions in the display area can be set as needed without any restrictions.
[0141] like Figure 8 As shown, it illustrates the location of sampling points in the display area.
[0142] Depend on Figure 8 It can be seen that the positions of multiple sampling points 802 can be set in the display area 801. These sampling points are located in a portion of the display area that is relatively close to the center.
[0143] It should be noted that the sampling point is not a point displayed in the display area, but only a sampling position set to determine the target object.
[0144] S603, combining the sub-mapping areas picked up by the multiple rays in the image mapping area, determine the target object picked up by the multiple rays in the stereoscopic virtual image.
[0145] The image mapping area may include at least one sub-mapping area, each sub-mapping area being a portion of the image mapping area. In an alternative embodiment, each sub-mapping area corresponds to a mapping area of a component in the stereoscopic virtual image within the display area.
[0146] The sub-mapping area picked up by the ray is the sub-mapping area through which the ray passes. Since the ray is emitted from the camera position of the virtual camera towards the sampling point, the sub-mapping area picked up by the ray is actually the sub-mapping area within the sampling area that can be of interest to the user.
[0147] Each ray can capture a sub-mapping region, and multiple rays may capture the same sub-mapping region.
[0148] This application can identify objects in the stereoscopic virtual image corresponding to the sub-mapping areas picked up by multiple rays as target objects.
[0149] The target object can be viewed as including: multiple object regions in a stereoscopic virtual image, each object region being an image region of the stereoscopic virtual image within a sub-mapping area; or, each object region being a region of constituent components contained in a sub-mapping area of the stereoscopic virtual image.
[0150] For example, still combined Figure 8 Explanation:
[0151] exist Figure 8 The following example uses a stereoscopic virtual image on a laptop computer. Figure 8 As can be seen from the multiple sampling points 802 shown and the image mapping area 804 of the stereoscopic virtual image 803 of the laptop, some of the sampling points are located in the image mapping area of the laptop, while others are located outside the image mapping area 804.
[0152] Furthermore, the areas with sampling points distributed within the image mapping area 804 of the laptop computer include: the screen image area and the keyboard image area in the stereoscopic virtual image.
[0153] In this case, after emitting rays from the virtual camera's position as the viewpoint to multiple sampling points, the areas picked up by the rays in the display area can be divided into two main categories: one is the area within the image mapping area, and the other is the area within the image mapping area. The area within the image mapping area can be divided into multiple sub-mapping areas according to the composition of the laptop, and each sub-mapping area corresponds to a component of the stereoscopic virtual image in the laptop.
[0154] See Figure 9 It shows a schematic diagram of the areas picked up by the ray in the display area.
[0155] contrast Figure 8 and Figure 9 As can be seen, the ray picks up three regions. These three regions belong to the mapping areas of the laptop's stereoscopic virtual image and correspond to a component object in the stereoscopic virtual image. Regions 901 and 902 are therefore sub-mapping areas. Region 903 does not contain any component of the laptop's stereoscopic virtual image.
[0156] based on Figure 8 and Figure 9 It can be seen that region 901 belongs to the screen image area in the stereoscopic virtual image of the laptop, while region 902 belongs to the keyboard image area in the stereoscopic virtual image of the laptop.
[0157] Based on this, by combining these two regions 901 and 902, the partial screen image area corresponding to region 901 and the partial keyboard image area corresponding to region 902 can be identified as the target objects. Therefore, the partial screen image area corresponding to region 901 and the partial keyboard image area corresponding to region 902 are two object regions in the target objects.
[0158] Alternatively, the keyboard and screen in the 3D virtual image can be directly identified as the target objects.
[0159] S504, Based on the target object, determine the focus object in the stereoscopic virtual image.
[0160] This step can be referred to in the relevant description of the previous embodiments.
[0161] For example, if the target object includes at least one target object area, the focus object can be determined from the at least one target object area by combining the display features based on the at least one target object area.
[0162] In this embodiment, the display features of the target object area may include: the position of the target object in the display area or sampling area, and the area ratio of the target object area in the image mapping area or sampling area of the stereoscopic virtual image, etc.
[0163] For example, considering that the larger the area of the target object region in the sampling area or the more sampling points it contains, the greater the likelihood that the user will focus on that target object region. This application can determine one or more target object regions that contain a large number of sampling points or have the largest area proportion in the sampling area as the focus of attention; alternatively, it can determine the constituent objects in the stereoscopic virtual image corresponding to that target object region as the focus of attention.
[0164] In another possible implementation, where the target object is a whole and includes at least one component (part or component, etc.) of the stereoscopic virtual image, this embodiment can further determine the distribution characteristics of each region within the sampling area of the target object. These distribution characteristics characterize the relative positional relationship of each region within the sampling area and the area size of each region within the sampling area. This area size can also be determined by the number of sampling points contained in each region of the target object (or the number of rays passing through the region of the target object).
[0165] Correspondingly, the focus of attention in the stereoscopic virtual image can be determined based on the distribution characteristics of each part of the target object within the sampling area.
[0166] like Figure 8 and Figure 9As shown, the target object in the 3D virtual image of a laptop includes the screen image area and part of the keyboard image area (or, in other words, the target object includes both the screen and the keyboard, the principle is the same). However, the part of the screen image area has a larger area in the sampling area and is relatively closer to the center area, so the screen image area, or the screen itself, can be identified as the focus of attention.
[0167] It is understood that, in the above embodiments of this application, after determining the focus object in the stereoscopic virtual image, the number of times the focus object is observed can be increased, and the number of times the focus object is observed can be stored. For example, if a certain object portion in the stereoscopic virtual image is the focus object, the number of times that object portion is observed will increase by one.
[0168] Similarly, in the above embodiments of this application, after determining the focus object in the stereoscopic virtual image, the duration of attention given to the focus object can also be obtained.
[0169] It is understandable that after displaying a stereoscopic virtual image and identifying a certain object in the stereoscopic virtual image as the focus of attention, if no adjustment is made to the displayed stereoscopic virtual image, the object in the stereoscopic virtual image will continue to be the focus of attention. In this application, the duration for which the object is the focus of attention can be counted until the object is no longer the focus of attention, thereby obtaining the duration for which the object was the focus of attention this time.
[0170] Furthermore, this application can add the cumulative attention duration of the focus object to the attention duration, and finally continuously calculate the cumulative attention duration of each focus object.
[0171] It is understood that, in the embodiments of this application, by determining the focus of attention in the stereoscopic virtual image of the physical object, the user's focus of attention on the physical object can be obtained. Based on this, by statistically analyzing the focus of attention of different users on the physical object, it can be used to analyze and determine the parts of the physical object with higher user attention, so as to prioritize upgrading the parts of the physical object with higher attention or optimize the problems existing in the parts.
[0172] In addition, identifying the focus object in the 3D virtual image of the physical object, the cumulative attention duration corresponding to the focus, and the number of attention, etc., is beneficial for analyzing and comparing user attention from multiple different dimensions. It can also provide reliable data support for determining user attention information and some big data analysis due to big data analysis in other scenarios.
[0173] In accordance with the method for obtaining the attention information of an object provided in this application, this application also provides an apparatus for obtaining the attention information of an object.
[0174] like Figure 10 The diagram illustrates a possible structural composition of an apparatus for acquiring information about an object, as described in this application. This apparatus is applied to an electronic device and may include:
[0175] The image display unit 1001 is used to display a stereoscopic virtual image in a virtual scene, wherein the stereoscopic virtual image is a stereoscopic image of a physical object mapped to a virtual object in the virtual scene;
[0176] Target determination unit 1002 determines the target object presented in the display area of the stereoscopic virtual image;
[0177] The focus determination unit 1003 is used to determine the focus object in the stereoscopic virtual image based on the target object, wherein the focus object belongs to at least a part of the stereoscopic virtual image.
[0178] In one possible implementation, the target determination unit includes:
[0179] The target determination subunit is used to determine the target object in the sampling area of the display area in the stereoscopic virtual image.
[0180] In one possible implementation, the target-determining sub-unit includes:
[0181] The camera determination subunit is used to determine the camera position of the virtual camera in the virtual scene at the current moment;
[0182] The object determination subunit is used to determine the target object in the stereoscopic virtual image that is located within the sampling area of the display area, with the camera position as the viewpoint position.
[0183] In one possible implementation, the object defines sub-units, including:
[0184] A mapping area determination subunit is used to determine the image mapping area onto which the stereoscopic virtual image is mapped within the display area;
[0185] A ray constructing subunit is used to determine multiple rays originating from the camera position and passing through multiple sampling points in the display area.
[0186] The picking sub-unit is used to combine the sub-mapping areas picked up by the multiple rays in the image mapping area to determine the target object picked up by the multiple rays in the stereoscopic virtual image.
[0187] In one possible implementation, the target object determined by the target determination unit or the target determination subunit includes: at least one target object region in the stereoscopic virtual image and the display features of the at least one target object region;
[0188] The focus determination unit includes:
[0189] The first focus determination unit is used to determine the focus object from the at least one target object region based on the display features of the at least one target object region.
[0190] In yet another possible implementation, the device further includes:
[0191] The feature determination unit is used to determine the distribution characteristics of each part of the target object within the sampling area before the focus determination unit determines the focus object in the stereoscopic virtual image;
[0192] The focus determination unit includes:
[0193] The second focus determination unit is used to determine the focus object in the stereoscopic virtual image based on the distribution characteristics of each part of the target object within the sampling area.
[0194] In any of the above embodiments of this application, the device further includes:
[0195] The number of times incrementing unit is used to increase the number of times the focus object is focused on after the focus determination unit determines the focus object in the stereoscopic virtual image;
[0196] A count storage unit is used to store the number of times the focus of attention is received.
[0197] In any of the above embodiments of this application, the device further includes:
[0198] The duration acquisition unit is used to obtain the duration of attention of the focus object in the stereoscopic virtual image after the focus determination unit determines the focus object in the stereoscopic virtual image;
[0199] The duration accumulation unit is used to add the accumulated attention duration of the focus object to the attention duration.
[0200] Furthermore, this application also provides an electronic device, such as Figure 11 As shown, it illustrates a schematic diagram of the composition structure of the electronic device. The electronic device can be any type of electronic device, and the electronic device includes at least a memory 1101 and a processor 1102.
[0201] The processor 1101 is used to execute the method for obtaining the attention information of an object as described in any of the above embodiments.
[0202] The memory 1102 is used to store the programs required for the processor to perform operations.
[0203] It is understood that the electronic device may also include a display unit 1103 and an input unit 1104.
[0204] Of course, the electronic device can also have more than Figure 11 There are no restrictions on the number of more or fewer components.
[0205] On the other hand, this application also provides a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the method for obtaining the interest information of an object as described in any of the above embodiments.
[0206] This application also proposes a computer program comprising computer instructions stored in a computer-readable storage medium. When executed on an electronic device, the computer program performs the method for acquiring information of interest about an object as described in any of the above embodiments.
[0207] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. Furthermore, the features described in the various embodiments of this specification can be substituted or combined with each other, enabling those skilled in the art to implement or use this application. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.
[0208] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.
[0209] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for obtaining the attention information of an object, comprising: Displaying a stereoscopic virtual image in a virtual scene, wherein the stereoscopic virtual image is a stereoscopic image of a physical object mapped to a virtual object in the virtual scene; Based on the current viewpoint position in the virtual scene, the target object presented in the two-dimensional display area of the stereoscopic virtual image is determined, and the viewpoint position is the reference point for mapping the stereoscopic virtual image in the virtual scene onto the two-dimensional screen display area; The target object includes: at least one target object region in the stereoscopic virtual image and display features of the at least one target object region; the display features include at least: the display position of the target object region within the display area and the area ratio of the target object region to the image display area of the stereoscopic virtual image; Determining the focus object in the stereoscopic virtual image based on the target object includes: determining the focus object from the at least one target object region based on the display characteristics of the at least one target object region; the focus object belongs to at least a portion of the stereoscopic virtual image.
2. The method according to claim 1, wherein determining the target object presented in the two-dimensional display area of the stereoscopic virtual image includes: Identify the target object within the sampling area of the two-dimensional display region in the stereoscopic virtual image.
3. The method according to claim 2, wherein determining the target object within the sampling area of the two-dimensional display area in the stereoscopic virtual image includes: Determine the camera position of the virtual camera in the virtual scene at the current moment; Using the camera position as the viewpoint, the target object within the sampling area of the display area in the stereoscopic virtual image is determined.
4. The method according to claim 3, wherein determining the target object within the sampling area of the display area in the stereoscopic virtual image, using the camera position as the viewpoint position, comprises: Determine the image mapping area within the display area onto which the stereoscopic virtual image is mapped; Starting from the camera position, determine multiple rays that pass through multiple sampling points in the display area from the starting point; By combining the sub-mapping areas in the image mapping area that are picked up by the multiple rays, the target object in the stereoscopic virtual image that is picked up by the multiple rays is determined.
5. The method according to claim 2, further comprising, before determining the focus object in the stereoscopic virtual image: Determine the distribution characteristics of each region within the sampling area of the target object; The step of determining the focus object in the stereoscopic virtual image based on the target object includes: Based on the distribution characteristics of each region of the target object within the sampling area, the focus of attention object in the stereoscopic virtual image is determined.
6. The method according to claim 1, further comprising, after determining the focus object in the stereoscopic virtual image: Increase the number of times the focus object is viewed; Store the number of times the focus is viewed.
7. The method according to claim 1, further comprising, after determining the focus object in the stereoscopic virtual image: Obtain the duration of attention for the object of focus; The cumulative attention duration of the focus object is increased by the attention duration.
8. An apparatus for acquiring attention information of an object, comprising: An image display unit is used to display a stereoscopic virtual image in a virtual scene, wherein the stereoscopic virtual image is a stereoscopic image of a physical object mapped to a virtual object in the virtual scene; The target determination unit, in conjunction with the current viewpoint position in the virtual scene, determines the target object presented in the two-dimensional display area of the stereoscopic virtual image, wherein the viewpoint position is a reference point for mapping the stereoscopic virtual image in the virtual scene onto the two-dimensional screen display area; The target object includes: at least one target object region in the stereoscopic virtual image and display features of the at least one target object region; the display features include at least: the display position of the target object region within the display area and the area ratio of the target object region to the image display area of the stereoscopic virtual image; A focus determination unit is configured to determine, based on the target object, a focus object in the stereoscopic virtual image, wherein the focus object belongs to at least a portion of the stereoscopic virtual image; The focus determination unit is specifically used to: determine the focus object from the at least one target object region based on the display characteristics of the at least one target object region.
9. An electronic device, comprising at least a memory and a processor; in, The processor is configured to execute the method for acquiring the attention information of an object as described in any one of claims 1 to 7 above; The memory is used to store the programs required for the processor to perform operations.