An augmented virtual item placement system

The augmented virtual item placement system addresses the inefficiency in arranging virtual items by using an interface controller and geometric fitting techniques to provide real-time feedback and optimize placement within a storage space, ensuring efficient and visually appealing arrangements.

WO2024227221A1PCT designated stage expired Publication Date: 2024-11-07BUGGY KIRSTIE
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
PCT/AU2024/050420
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-04
Filing Date
2024-05-02
Publication Date
2024-11-07

AI Technical Summary

Technical Problem

Current systems lack an efficient method for visually and spatially arranging virtual items within a storage space, failing to provide real-time feedback on space constraints and optimizing item placement effectively.

Method used

An augmented virtual item placement system that uses an interface controller to display and arrange virtual items within a storage space, incorporating a space constraint calculating controller and a placement controller, which allows users to configure dimensions and shapes, and employs geometric fitting techniques to optimize item arrangement and provide feedback on space availability.

Benefits of technology

Enables users to efficiently arrange virtual items within a storage space by providing real-time feedback on space constraints and optimizing placement, ensuring items are placed in a visually and spatially efficient manner, while preventing overfilling and allowing for automatic optimization of item placement.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure AU2024050420_07112024_PF_FP_ABST
    Figure AU2024050420_07112024_PF_FP_ABST
Patent Text Reader

Abstract

The disclosed augmented virtual item placement system comprises an electronic device that utilises an interface controller for displaying a user interface where virtual items can be visually arranged within an imaged storage space. A space constraint calculating controller calculates the remaining space within the storage space after placing virtual items, while a placement controller augments the display of these items within the user interface. Additionally, the system can connect to a server and potentially an e-commerce server over the Internet, accessing a database of product images and models for virtual placement consideration. The interface allows users to configure the dimensions and shapes of virtual items and the storage space itself, enabling placement of items in a visually and spatially efficient manner. The system optimises item arrangement using geometric fitting techniques and provides feedback on space availability and constraints during the placement process.
Need to check novelty before this filing date? Find Prior Art

Description

An augmented virtual item placement systemField of the Invention

[0001] This invention relates generally to a system which is configured to display a user interface with an augmented image of a storage space for the placement of virtual items within the displayed storage space.Summary of the Disclosure

[0002] The disclosed augmented virtual item placement system comprises an electronic device that utilises an interface controller for displaying a user interface where virtual items can be visually arranged within an imaged storage space.

[0003] A space constraint calculating controller calculates the remaining space within the storage space after placing virtual items, while a placement controller augments the display of these items within the user interface.

[0004] Preferably, the system can connect to a server and potentially an e-commerce server over the Internet, accessing a database of product images and models for virtual placement consideration.

[0005] The interface may allow users to configure the dimensions and shapes of virtual items and the storage space itself, enabling placement of items in a visually and spatially efficient manner.

[0006] The system may optimise item arrangement using geometric fitting techniques and provides feedback on space availability and constraints during the placement process.

[0007] Other aspects of the invention are also disclosed.Brief Description of the Drawings

[0008] Notwithstanding any other forms which may fall within the scope of the present invention, preferred embodiments of the disclosure will now be described, by way of example only, with reference to the accompanying drawings in which:

[0009] Figure 1 shows an augmented virtual item placement system ;

[0010] Figure 2 shows an exemplary placement interface displayed by the system of Figure 1 ; and

[0011] Figure 3 shows exemplary processing by the system of Figure 1 .Description of Embodiments

[0012] Figure 1 shows an augmented virtual item placement system 100 in accordance with an embodiment. The system 100 comprises an electronic device 101 which may take the form of a personal computer device, mobile phone device or the like.

[0013] The electronic device 101 comprises a processor 102 for processing digital data. A memory device 107 in operable communication with the processor 102 via a system bus 1 15 is configured for storing digital data including computer program code instructions. In use, the processor 102 fetches these computer program code instructions and associated data 104 for implementing the functionality described herein.

[0014] The computer program code instructions may be logically divided into a plurality of computer program code instruction controllers 105. These controls 105 may comprise an interface controller which is configured to display a placement interface 200, a space constraint calculating controller which is configurable with dimensions of a storage space 201 and which is further configured to calculate remaining space within the storage space 201 when placing virtual items 203 and a placement controller which is configured to augment the display of a plurality of virtual items 203 over the storage space 201 displayed within the interface 200.

[0015] The electronic device 101 may comprise an image sensor 106 configured for capturing image data, including for capturing image data of the storage space 201.

[0016] The electronic device 101 may further comprise a display 107 which is controlled by the controllers 105 to display the user interface 200.

[0017] In embodiments, the electronic device 101 may be in operable communication with a server 109 across a wide area network, such as the Internet.

[0018] In certain embodiments, the system 100 may involve an e-commerce server 1 10. The server 109 and / or the e-commerce server 1 10 being in operable communication with a database 1 1 1 of products 1 12. Each product 1 12 may be stored in relation to image data 1 13 and / or a 2D or 3D model 1 14.

[0019] Figure 2 shows the exemplary placement interface 200 which is controlled by the system 100 for the placement of virtual items 203 within a storage space 201 .

[0020] The interface 200 displays image data taken of a storage space 201. In this example, the storage space 201 is an image taken of a pantry and which may therefore comprise a plurality of shelves dividing the storage space 201 into a plurality of compartments 202.

[0021] The interface 200 may further display a plurality of virtual items 203 for placement or which have been placed within the available storage space 201 . These virtual items 203 may be 2D virtual items but, in a preferred embodiment, are displayed as 3D items. According to the 3D embodiment, the system 100 takes into account three dimensions of each virtual item 203 with respect to 3 dimensions of the storage space 201 or compartments 202.

[0022] The interface 200 is usable by a user to place the virtual items within the available storage space 201. Where the storage space 201 displays an image taken of pantry shelves, the virtual items 203 may be virtual items of pantry items, such as pantry item containers.

[0023] In embodiments, the interface 200 may comprise an item configuration input 204 wherein the dimensions of each virtual item may be configured. The item configuration input 204 may allow the input of width, depth and height dimensions.

[0024] The item configuration input 204 may further allow the specification of the shape of each item 203, such as whether the item 203 is cylindrical, rectangular or the like.

[0025] The placement interface 200 may further comprise a storage space dimension configuration input operable to configure dimensions of the storage space 201 or the compartments 102 thereof.

[0026] In one embodiment, the storage space dimension configuration input allows the specification of the width, depth and height of the storage space 201 or each compartment 202.

[0027] In alternative embodiments, the storage space dimension configuration input comprises a plurality of on-screen placement vertices 205 which are positioned onscreen to allow the space constraint calculating controller to calculate the dimensions of the storage space.

[0028] In a preferred embodiment, these vertices 205 comprise at least four vertices, three of these usable to specify the front dimensions of a rectangular storage space 201 and at least one rear vertices for specifying the depth of the storage space 201.

[0029] For example, in the example given, reference vertex 205A is placed at the top left-hand corner of the displayed storage space 201. A width reference vertex 205B is placed to the right of the reference vertex 205A to allow the space constraint calculating controller to calculate the width of the storage space 201.

[0030] A height reference vertex 205C is placed below the reference vertex 205A to allow the space constraint calculating controller to calculate the height of the storage space 201 .

[0031] Finally, for the three-dimensional embodiment of the system 100, a depth reference vertex 205D is placed diagonally with respect to the reference vertex 205 to allow the space constraint calculating controller to calculate the depth of the storage space 201 .

[0032] These vertices 205 could be positioned within the user interface by drag-and- drop gesture functionality.

[0033] The user would place these vertices 205 to coincide with the visual corners of the image of the storage space 201 .

[0034] In alternative embodiments, the space constraint calculating controller may automate the calculation of the internal volume of the storage space using image recognition techniques.

[0035] The interface 200 may have an available space indication 206. The available space indication 206 may indicate the total space available (i.e., the total internal volume of the storage space 201 ) and / or the remaining space available which diminishes with the placement of each virtual item 203.

[0036] The interface 200 may further comprise an item selection area 207 wherein a plurality of virtual items 203 are placed and available for placement. Placement of virtual items 203 may also be by way of drag-and-drop gesture functionality.

[0037] The interface 200 may also comprise an auto place input 208 which causes the placement controller to automatically place the virtual items 203.

[0038] The placement controller 203 may optimise the placement of items using known geometric fitting techniques.

[0039] The placement controller 203 may place virtual items 203 in the order they are selected by the user.

[0040] In alternative embodiments, available virtual items 203 may be assigned a placement hierarchy wherein the placement controller 203 places the items according to the hierarchy. In this regard, if there is insufficient remaining space, the placement controller may remove the virtual item 203 from the storage space in favour of another virtual item 203 having a greater hierarchy.

[0041] Figure 3 shows an exemplary process implement by the system 100. At step 301 , the user would use the image sensor 106 to capture image data of a storage space 201. As alluded to above, the user may use electronic device 101 to take a photograph of pantry storage space 201 .

[0042] At step 302, the user uses the interface 200 to configure the storage space dimensions 302.

[0043] As alluded to above, in one embodiment, the user may use the storage space configuration input to specify the width, height and depth of the storage space 201. In embodiments, the placement interface 200 is usable to specify the number of compartments 202 within the storage space and wherein the user is able to specify the width, height and depth of each compartment 202.

[0044] As also alluded to above, in alternative embodiments, the user may drag the storage space configuration placement vertices 205 on-screen to the respective corners of the storage space 201 . In a preferred embodiment, for vertices 205 are used to allow the space constraint calculating controller to calculate the width, height and depth of the storage space 205.

[0045] Similarly, the placement interface 200 may be usable to specify the number of compartments 202 within the storage space 201 and wherein the user is able to place respective vertices 205 in relation to each compartment 202.

[0046] At step 303, the item selection area 207 is populated with a virtual items 203 available for placement. In embodiments, the user may take a photograph of each item for placement using the image sensor 106 of the electronic device 101 . The item configuration input 204 may be used to specify the dimensions and shape of each item 203. For example, the user may specify the rectangular dimensions of a cereal box or the cylindrical dimensions of a can of beans.

[0047] Alternatively, the virtual items 203 may be obtained from the server 109 or e- commerce server 1 10. In accordance with this embodiment, the user may be considering purchasing items for placement within the storage space 201 but only if the items would physically fit within the storage space 201.

[0048] For example, the e-commerce server 1 10 may be selling various pantry items. In this regard, the server 109 or the e-commerce server 1 10 may comprise a database 1 1 1 of these pantry item products 1 12. Each product 1 12 may be stored in relation to image data 1 13 of the product and a 3D model (i.e., shape and dimensions).

[0049] For example, the user may wish to purchase a box of cereal wherein the database 1 1 1 would be queried to obtain at least one of an image and / or dimensions thereof. As such, the virtual item 203 representation of the cereal box may be virtually placed using the interface 200.

[0050] In embodiments, the e-commerce server implements a shopping cart and wherein the placement of selected items within the shopping cart is restricted by the space constraint calculating controller. For example, if there is insufficient space within the storage space 201 to fit an item available for sale through the e-commerce server 1 10, an indication may be provided to the user that the item will not fit or, the item cannot be added to the shopping cart.

[0051] With reference to Figure 2, the item selection may be displayed within the item selection field 207, each available for selection.

[0052] At step 304, the user is able to select an item for placement at step 306. As alluded to above, each item may be dragged and dropped in the display storage space201.

[0053] At step 306, the interface controller displays the placed virtual item 203 at the respective position within the storage space 201 . The placement controller may control the depth of placement of each virtual item 203 so that the interface controller visually displays virtual items 203 behind each other depending on the depth position. For example, a virtual item 203 of a tin of beans may be displayed in front of a virtual item 203 of a box of cereal.

[0054] Upon placement of each virtual item 203, the space constraint calculating controller may calculate the remaining space available by subtracting the volume of the virtual item 203 determined by the dimensions thereof from the calculated volume of the storage space 201 .

[0055] In embodiments, the selection may be constrained at step 305. For example, if the user is attempting to place a virtual item 203 but the space constraint calculating controller determines that there is insufficient space remaining within the storage space 201 , the selection thereof may be prevented.

[0056] In embodiments, the ability to place items may be indicated within the item selection area 207. For example, items which can no longer fit within the remaining storage space may be displayed in red.

[0057] At step 307, the placement of a virtual item 203 may be constrained. For example, the placement controller may prevent placement of virtual item 203 at a location within the storage space at which the virtual item 203 would not fit.

[0058] The user may drag the virtual item 203 around the user interface over the storage space 201 wherein, in embodiments, the interface controller may display the virtual item 203 in red if it would not fit but in green if it would fit. As such, when the user receives this visual indication of the ability of the virtual item 203 to fit at the proposed location, the user may release the item for placement.

[0059] In embodiments, as the user is dragging the virtual item 203 over the storage space 201 for placement, the interface controller may reposition already placed virtual items 203 in real-time. For example, a virtual item 203 of a tin of beans placed at the back of a compartment 202 may be moved to the front to allow for the placement of a virtual item 203 of a box of cereal.

[0060] As also alluded to above, the placement controller may reference the placement hierarchy assigned to the virtual items 203 to remove items from the storage space during proposed placement by the user. For example, virtual item 203 having higher placement hierarchy would displace a virtual item 203 with a lower hierarchy. The interface controller may indicate that an already placed virtual item is about to be displaced such as in a different colour wherein, after placement of the higher ranking virtual item 203, the lower ranking virtual item is removed back to the item selection area 207.

[0061] At step 308, the user may use the auto place input 208 or auto place the remaining items wherein the placement controller will optimise the placement of the remaining virtual items 203.

[0062] In embodiments, the placement controller may be configured not to reposition already placed virtual items 203. In alternative embodiments, the placement controller may be configured to allow repositioning of lower ranking virtual items 203.

[0063] During auto placement, the placement controller may be configured to optimise the placement of items using known geometric fitting techniques.

[0064] The foregoing description, for purposes of explanation, used specific nomenclature to provide a thorough understanding of the invention. However, it will be apparent to one skilled in the art that specific details are not required in order to practise the invention. Thus, the foregoing descriptions of specific embodiments of the invention are presented for purposes of illustration and description. They are not intended to be exhaustive or to limit the invention to the precise forms disclosed as obviously many modifications and variations are possible in view of the above teachings. The embodiments were chosen and described in order to best explain the principles of the invention and its practical applications, thereby enabling others skilled in the art to best utilize the invention and various embodiments with various modifications as are suited to the particular use contemplated. It is intended that the following claims and their equivalents define the scope of the invention.

Claims

Claims1. An augmented virtual item placement system comprising: an electronic device comprising a digital display and a processor in operable communication with the digital display, the processor executing computer program code instructions controllers, including: an interface controller configured to display a placement interface including image data taken of a storage space; a space constraint calculating controller configurable with dimensions of the storage space; a placement controller configured to augment the display of a plurality of virtual items over the displayed storage space, wherein the space constraint calculating controller is configured to calculate a remaining space after the placement of each virtual item and wherein placement of each virtual items is constrained by the remaining space and dimensions thereof.

2. The system as claimed in claim 1 , wherein the placement interface comprises an item dimension configuration input for the input of dimensions of a selected virtual item.

3. The system as claimed in claim 2, wherein the item dimension configuration further allows the specification of a shape of a virtual item and wherein the dimensions correspond to the shape.

4. The system as claimed in claim 1 , wherein the placement interface comprises a storage space dimension configuration input for input of dimensions of the storage space.

5. The system as claimed in claim 4, wherein the storage space configuration input comprise a plurality of configurable on-screen placement vertices.

6. The system as claimed in claim 5, wherein the vertices are placed using drag- and-drop gestures.

7. The system as claimed in claim 5, wherein the vertices comprise at least two vertices for specifying a width of the storage space.

8. The system as claimed in claim 5, wherein the vertices comprise at least two vertices for specifying a height of the storage space.

9. The system as claimed in claim 5, wherein the vertices comprise at least two vertices for specifying a depth of the storage space.

10. The system as claimed in claim 5, wherein the vertices comprise at four vertices for specifying a width, depth and height of the storage space.1 1 . The system as claimed in claim 1 , wherein the storage space comprises a plurality of compartments and wherein the space constraint calculating controller configurable with dimensions of at least one compartment.

12. The system as claimed in claim 1 , wherein the system comprises an e- commerce shopping cart and a database of products stored in relation to dimensions and at least one of a 3D model and image data, and wherein adding of items in the shopping cart is constrained by the space constraint calculating controller.

13. The system as claimed in claim 1 , wherein the placement controller optimises the placement of items using a geometric fitting technique.

14. The system as claimed in claim 1 , wherein the placement controller places virtual items according to a selection order.

15. The system as claimed in claim 1 , wherein available virtual items are assigned a placement hierarchy wherein the placement controller places the items according to the hierarchy.

16. The system as claimed in claim 15, wherein, if there is insufficient remaining space, the placement controller removes a virtual item from the storage space in favour of another virtual item having a greater hierarchy.

17. The system as claimed in claim 1 , wherein the system is configured to receive image data of a virtual item for placement using the image sensor of the electronic device.

18. The system as claimed in claim 1 , wherein virtual items are obtained from a product database of virtual items.

19. The system as claimed in claim 18, when the database comprises image data and the dimension data stored in relation to each virtual item.

20. The system as claimed in claim 19, wherein the database is in operable communication with an e-commerce server which implements a shopping cart and wherein the placement of selected items within the shopping cart is restricted by the space constraint calculating controller.21 . The system as claimed in claim 1 , wherein a virtual item can be dragged and dropped in the displayed storage space.

22. The system as claimed in claim 1 , wherein the interface controller displays a placed virtual item at the respective position within the storage space.

23. The system as claimed in claim 22, wherein the placement controller controls the depth of placement of each virtual item so that the interface controller visually displays virtual items behind each other depending on the depth position.

24. The system as claimed in claim 1 , wherein, upon placement of each virtual item, the space constraint calculating controller calculates a remaining space available by subtracting the volume of the virtual item determined by the dimensions thereof from the volume of the storage space.

25. The system as claimed in claim 1 , wherein selection of virtual items is constrained by the space constraint calculating controller.

26. The system as claimed in claim 25, wherein ability to place items is indicated within an item selection area of the interface.

27. The system as claimed in claim 1 , wherein placement of a virtual item is constrained by the space constraint calculating controller.

28. The system as claimed in claim 27, wherein the placement controller is configured to prevent placement of virtual item at a location within the storage space at which the virtual item would not fit.

29. The system as claimed in claim 27, wherein the interface is configured to allow the repositioning of a virtual item over the storage space and the interface controller displays the virtual item according to if the virtual item would fit at the location .

30. The system as claimed in claim 1 , wherein, as a virtual item is repositioned over the storage space for placement, the interface controller repositions already placed virtual items in real-time.31 . The system as claimed in claim 30, wherein the placement controller references the placement hierarchy assigned to the virtual items to remove items from the storage space during placement wherein a virtual item having higher placement hierarchy would displace a virtual item with a lower hierarchy.

32. The system as claimed in claim 31 , wherein the virtual item with the lower hierarchy is removed back to the item selection area.

33. The system as claimed in claim 1 , wherein the interface has auto place functionality wherein the placement controller optimises the placement of the remaining virtual items.

34. The system as claimed in claim 33, wherein the placement controller is configured not to reposition already placed virtual items.

35. The system as claimed in claim 33, wherein the placement controller is configured to allow repositioning of lower ranking virtual items.

Citation Information

Patent Citations

  • Method for automatically adjusting seats in vehicles based upon cargo size

    US20070265754A1

  • Container Selection in a Materials Handling Facility

    US20130218799A1

  • Dynamic user interactions for display control and identifying dominant gestures

    US20140201674A1

  • Method of providing virtual space, method of providing virtual experience, program and recording medium therefor

    US20180025531A1

  • Item selection based on dimensional criteria

    US9965793B1