Social network using augmented reality

Abstract

Systems and methods for social networking using augmented reality are disclosed. A network can receive requests from a plurality of client devices to connect to a server. The server can be associated with an object, such as a landmark or a product. The server can serialize an AR world including the object and content associated with the object and send data indicative of the AR world to the plurality of client devices. The server can receive, from a first client device of the plurality of client devices via the network, an indication of user input associated with the AR world. The indication of user input can include textual data, digital image data, or user feedback. The indication of user input can be broadcast to the plurality of client devices.

Description

Background Technology

[0001] Communication is increasingly facilitated by internet-based tools. These tools can be any software or platform. Existing social media platforms, such as YouTube and Facebook, enable users to communicate with each other via static web applications by sharing images, videos, and other information. As communication devices, such as mobile phones, become increasingly sophisticated, people continue to expect new ways of social networking and communication. Attached Figure Description

[0002] The following detailed description will be better understood when read in conjunction with the accompanying drawings. For illustrative purposes, exemplary embodiments of various aspects of this disclosure are illustrated in the drawings; however, the invention is not limited to the specific methods and means disclosed.

[0003] Figure 1 An example system for using augmented reality in a social network is shown.

[0004] Figure 2 An example process for using a social network with augmented reality, which can be executed by a cloud network according to this disclosure, is shown.

[0005] Figure 3 An example process for using a social network with augmented reality, which can be executed by a client device according to this disclosure, is shown.

[0006] Figures 4a to 4b An example object is shown that is tracked by an object tracking model according to this disclosure.

[0007] Figures 5a to 5b Examples of real-life environments enhanced by various social networking tools are shown.

[0008] Figure 6 An example user interface for an application using augmented reality social networks, according to this disclosure, is shown.

[0009] Figure 7 Another example user interface for an application using an augmented reality social network, according to this disclosure, is shown.

[0010] Figure 8 An example computing device is shown that can be used to perform any of the techniques disclosed herein. Detailed Implementation

[0011] Figure 1 The illustration shows an example system 100 for distributing content. System 100 may include a cloud network 102 and multiple client devices 104a-d. The cloud network 102 and the multiple client devices 104a-d may communicate with each other via one or more networks 120.

[0012] Cloud network 102 may be located in a data center, such as a single location, or distributed across different geographical locations (e.g., in several locations). Cloud network 102 may provide services via one or more networks 120. Network 120 includes various network devices, such as routers, switches, multiplexers, hubs, modems, bridges, repeaters, firewalls, proxy devices, etc. Network 120 may include physical links, such as coaxial cable links, twisted-pair cable links, fiber optic links, combinations thereof, etc. Network 120 may include wireless links, such as cellular links, satellite links, Wi-Fi links, etc.

[0013] Cloud network 102 may include multiple computing nodes 118 hosting various services. In one embodiment, node 118 hosts augmented reality (AR) service 112. AR service 112 may augment graphics, video, sound, and other inputs onto real-world objects. For example, AR service 112 may place digital elements, such as images and user input, onto objects in a real-world environment. AR service 112 may be configured to receive and distribute AR data 116 via various transmission technologies. AR data 116 may include image, video, audio data, text data, and / or other data. In an example, AR data 116 includes data indicating objects in a real-world environment, such as landmarks or products. AR data 116 may further include content associated with the objects, such as text, graphics, or information indicating user feedback. AR data 116 may be used to generate an AR world that includes objects and content associated with those objects. AR data 116 may be stored in database 114. For example, AR service 112 may include social networking services, content sharing platforms, collaborative gaming platforms, etc.

[0014] In embodiments, at least one of the AR service 112 or client devices 104a-d includes an object tracking model 110. The object tracking model 110 can be configured to automatically extract information from images (such as photographs or posters), scenes (such as real-world environments), or video streams. In embodiments, the object tracking model 110 can use positive and negative training applied by machine learning algorithms or computer vision algorithms to extract information, such as labels or recognition features, from images, scenes, or video streams. Once extracted, the information can be used to detect at least one object, feature, or element of interest present in the image, scene, or video stream. For example, the information can be used to detect products or landmarks present in an image or scene. The object tracking model 110 can be configured to identify objects, features, or elements of interest that have already been detected in an image or scene.

[0015] In an embodiment, to identify objects, features, or elements of interest present in an image, scene, or video stream, object tracking model 110 compares detected objects, features, or elements of interest with information stored in a database such as database 114. The information stored in the database may include information about objects, features, or elements of interest that have already been imaged and characterized. If a detected object, feature, or element of interest matches an object, feature, or element of interest stored in the database, object tracking model 110 identifies the detected object, feature, or element of interest as the object, feature, or element of interest. For example, if the object tracking model detects a unique stationary object (such as a landmark) in an image, scene, or video stream, the detected landmark may be compared with information stored in the database to determine that the detected landmark is, for example, the Eiffel Tower.

[0016] In an embodiment, the information extracted by the object tracking model 110 includes one or more measurements associated with an object, feature, or element of interest. For example, one or more measurements may include information about the object's size, its layout relative to other objects in the scene, or geometric or topological information. Such techniques for object identification are known. In one example, the object tracking model 110 may match pixels in a graphics file acquired from a user's camera with a set of points associated with a stored model of the object, feature, or element of interest. The object tracking model 110 may be configured to determine the approximate number of lines of the corresponding line associated with the object, feature, or element of interest. The set of points may be a series of connected line segments or links, such as polylines, that the object tracking model 110 has generated and fitted to the set of pixels. The object tracking model 110 may also determine the tilt angle of each link in the polyline. In short, it is known to use lines in a model associated with an object to identify lines in one or more graphics images in order to identify objects. Other techniques may be used to identify objects in an image.

[0017] In an embodiment, object tracking model 110 is configured to segment an image (such as a 2D image) into multiple regions to determine 3D information about objects, features, or elements of interest. Different features present in an image, scene, or video stream can be distinguished and separated from the image, scene, or video stream, allowing object tracking model 110 to extract useful 3D information about the objects, features, or elements of interest.

[0018] In an embodiment, object tracking model 110 is configured to convert distance information associated with the extracted set of intermediate pixels into coordinate values ​​in a three-dimensional space, such as a real-world environment containing the object, feature, or element of interest. For example, object tracking model 110 may be configured to convert pixel position information into coordinate values ​​in real-world space. The coordinate values ​​in real-world space may be 3D coordinates. By converting the distance information associated with pixels into 3D coordinates, the distance information can be considered as spatial distance information.

[0019] The object tracking model 110 can be further configured to normalize at least a portion of the 3D coordinate information. For example, in one embodiment, the object tracking model 110 is configured to output a normalized coordinate space relative to the detected object. The object tracking model 110 is configured to normalize distances by normalizing at least a portion of the 3D coordinate information. Distance normalization can be performed by decreasing or increasing the distance between the detected object of interest, feature, or element and the client devices 104a-d. For example, if two different client devices 104a-d are located at different distances from the detected object of interest, feature, or element, the object tracking model 110 can normalize these two different distances so that the two different client devices 104a-d are located at the same distance from the detected object of interest, feature, or element. The same distance can be a predetermined distance.

[0020] In an embodiment, object tracking model 110 is configured to normalize the distance relative to an object while maintaining the orientation relative to the object. For example, if two different client devices 104a-d located at different distances from a detected object, feature, or element of interest have different lines of sight to the detected object, feature, or element of interest, object tracking model 110 can normalize these two different distances so that the two different client devices 104a-d are located at the same distance from the detected object, feature, or element of interest, while maintaining the different lines of sight associated with the different client devices 104a-d.

[0021] In this embodiment, the AR data 116 provided by AR service 112 includes user input. User input may include at least one of the following: text data, digital image data, or user feedback data. If the user input includes text data, the text data may include information about text that at least one user of AR service 112 wants to overlay on an object and that will be viewed by other users of AR service 112. The text data may include at least one of letters, words, or phrases, and may include various fonts, colors, or sizes, and may be written in any language. If the user input includes digital image data, the digital image data may include a digital image that at least one user of AR service 112 wants to overlay on an object and that will be viewed by users of AR service 112. The digital image data may include at least one of drawings or emojis. Drawings may be of various shapes or colors. If the user input includes user feedback data, the user feedback may indicate whether at least one user of AR service 112 likes or dislikes the object, or whether at least one user of AR service 112 likes or dislikes user input from other users of AR service 112. User feedback may include "like" or "dislike".

[0022] AR data 116 can be output to different client devices 104a-d via cloud network 102. For example, AR data 116 can be streamed to client devices 104a-d. Multiple client devices 104a-d can be configured to access AR data 116 from AR service 112. In an embodiment, client devices 104a-b may include application 106 and object tracking model 110. Application 106a-b can output (e.g., display, render, present) the AR world generated based on AR data 116 to the user associated with client devices 104a-b.

[0023] Client devices 104a-d can include any type of computing device, such as mobile devices, tablets, laptops, desktop computers, smart TVs or other smart devices (e.g., smartwatches, smart speakers, smart glasses), gaming devices, set-top boxes, digital streaming devices, robots, etc. Client devices 104a-d can be associated with one or more users. A single user can use one or more of client devices 104a-d to access cloud network 102. Client devices 104a-d can travel to various locations and use different networks to access cloud network 102.

[0024] AR service 112 can be configured to receive input from a user. A user can register as a user of AR service 112 and can be a user of application 106 operating on client devices 104a-d. User input can include connection requests and user input data, such as text data, digital image data, or user content. Connection requests can include requests from client devices 104a-d to connect to AR service 112. User input data can include information that the user connected to AR service 112 wants to share with other connected users of AR service 112.

[0025] In this embodiment, a user can use application 106 on client devices 104a-d to send client input data to AR service 112. Client devices 104a-d can access interface 108 of application 106, which allows the user to input text data, digital image data, or user feedback. Interface 108 may include input elements. For example, input elements may be configured to receive text data, digital image data, or user feedback from the user. Application 106 can determine timing information for the input, such as when the user inputs text data. Application 106 can send the input and associated metadata to cloud network 102. For example, application 106 may send text data or digital image data, the identifier of the user inputting the data, and timing information for the text data or digital image data to cloud network 102. AR service 112 can store the input and associated metadata in database 114.

[0026] AR service 112 can be configured to output data to a user. For example, AR service 112 can serialize an AR world including objects and content associated with those objects, and transmit data indicating the AR world to the user via network 120. Users can register as users of AR service 112 to share and view information about objects (such as landmarks or products). Users can be users of application 106 operating on client devices 104a-d. Application 106 can output (display, render, present) the AR world to the user associated with client devices 104a-d. Client devices 104a-d can access interface 108 of application 106. Interface 108 can include output elements. Output elements can be configured to display the AR world, allowing users to input information associated with the AR world.

[0027] In this embodiment, user input associated with the AR world can be output to all users viewing the same AR world. For example, all users connected to a server associated with a particular object can view user input received by AR service 112 from any user connected to the server. User input can be output by AR service 112 in real-time or near real-time. Application 106 can display the user input overlaid on the AR world in various ways on client devices 104a-d. For example, user input can be shown as scrolling across the AR world. As an example and not a limitation, user input can be shown as scrolling across the AR world from right to left and can reappear on the right side of interface 108 of user devices 104a-d after it has scrolled all the way to the left.

[0028] Multiple compute nodes 118 can handle tasks associated with AR service 112. Multiple compute nodes 118 can be implemented as one or more compute devices, one or more processors, one or more virtual compute instances, and / or combinations thereof. Multiple compute nodes 118 can be implemented by one or more compute devices. One or more compute devices can include virtualized compute instances. Virtualized compute instances can include virtual machines, such as simulations of computer systems, operating systems, and / or servers. Virtual machines can be loaded by the compute device based on virtual images and / or other data defining specific software (e.g., operating systems, dedicated applications, servers) used for simulation. As the demand for different types of processing services changes, different virtual machines can be loaded and / or terminated on one or more compute devices. A hypervisor can be implemented to manage the use of different virtual machines on the same compute device.

[0029] Figure 2 An example process 200 executed by a cloud network (e.g., cloud network 102) is illustrated. Cloud network 102 can execute process 200 to enhance real-world environments using information shared by users. Although in Figure 2 The operations are described as a series of operations, but those skilled in the art will understand that various embodiments may add, delete, reorder, or modify the described operations.

[0030] At point 202, cloud network 102 can receive at least one connection request. At least one connection request can be received from at least one client device (such as client devices 104a-d). The at least one connection request can include a request from the at least one client device to connect to a server (such as AR service 112). The request from the at least one client device can include information instructing the at least one client device to approach a real-world environment including an object or to approach an image including an object. The object can be a unique, immovable object, such as a landmark or product. The request to connect to the server from the at least one client device can instruct the camera of the at least one client device to point at (e.g., focus on) the object. If the camera of a client device is pointing at a specific landmark, the connection request from the at least one client device can be a request to join a server associated with the landmark. For example, if the camera of a client device is pointing at a well-known building, the request from the client device can be a request to connect to a server associated with the well-known building.

[0031] In this embodiment, cloud network 102 can receive more than one request to connect to the same server. This may occur, for example, if the cameras of two client devices (such as client devices 104a-b) are pointed at the same object or product. If both client devices are pointed at the same building, cloud network 102 can receive two requests to connect to a server associated with that building. Cloud network 102 can receive requests to connect to the same server simultaneously or at different times. For example, the camera of client device 104a may be pointed at the building at a first time, and the camera of client device 104b may be pointed at the building at a later second time. In that case, cloud network 102 can receive a connection request from client device 104a before it receives a connection request from client device 104b. In another example, the cameras of client devices 104a-b may be pointed at the same well-known building simultaneously. In this case, cloud network 102 can receive connection requests from both devices simultaneously.

[0032] To determine the object being pointed at by the camera of at least one client device, an object tracking model, such as object tracking model 110, can be used. Object tracking model 110 can reside in at least one of a server or at least one client device. Object tracking model 110 can be configured to detect objects, such as landmarks or products, being pointed at by the camera of at least one device. Object tracking model 110 can output a common coordinate space and a 3D coordinate frame, allowing the position of the client device relative to the object to be determined. For example, as... Figures 4a to 4b As shown, objects such as buildings (402) can be detected by the object tracking model.

[0033] In an embodiment, such as Figure 4aAs shown, when the object tracking model detects object 402, the client device's camera can be positioned relative to one of four planes (404a, 404b, 404c, or 404d) positioned around the object. The client device's camera can have multiple different lines of sight to object 402 and can detect object 402 from these different lines of sight. For example, Figure 4a The illustration shows a scene where the cameras of two client devices are positioned such that they are on opposite sides of plane 404a relative to an object. The cameras of the two client devices are positioned such that they are on opposite sides of plane 404a relative to the object, having two different lines of sight to object 402. Figure 4a The illustration also shows a scene in which the camera of the client device is positioned such that it is on the side opposite to plane 404b.

[0034] A plane represents a mathematically constructed representation of an object. User messages and graphics are projected onto planes (such as planes 404a, 404b, 404c, or 404d). Client devices view the messages projected onto the planes based on calculations that determine where the client device is located relative to the object and the plane. When the camera is positioned relative to the object on the side opposite to plane 404a, the camera can be pointed at the first part of the object, and the client device can view the message projected onto that particular plane. When the camera is positioned on the side opposite to one of the other planes 404b-d, the camera can be pointed at different parts of the object, such as different sides of a building.

[0035] In another embodiment, such as Figure 4b As shown, when the object tracking model detects object 402, the client device's camera can be positioned relative to a structure in the form of a cylinder 406 positioned around object 402 (i.e., the building). Cylinder 406 can be a straight cylinder or an oblique cylinder. The base of cylinder 406 can be circular or not. User messages and graphics are projected onto this cylinder 406, and the client device views the messages projected onto this cylinder 406 based on calculations determining where the client device is located relative to object 402 and cylinder 406. When the camera is positioned relative to object 402 on the side opposite to cylinder 406, the camera can be pointed at a first portion of object 402, and the client device can view messages projected onto a specific portion of cylinder 406. Other structures are also envisioned, such as structures with sides arranged in a hexagonal pattern or any other arbitrary geometric grid.

[0036] Return to reference Figure 2At 204, cloud network 102 can serialize the current AR world. The current AR world may include objects pointed at by the camera of at least one client device. For example, the current AR world may include objects identified by an object tracking model, such as buildings. The current AR world may further include content associated with objects and received from user devices. The content associated with objects may include text, graphics, or information indicating user feedback associated with objects. Cloud network 102 can serialize the current AR world into a byte array. In one embodiment, the byte array may be stored in a database (such as database 114). If the byte array is stored in a database, it can be loaded later to continue the AR world session. In another embodiment, the byte array may be sent to at least one client device via a network (such as network 120). If the byte array is sent to at least one client device via a network, the AR world session may be shared by users associated with at least one client device.

[0037] For example, at 206, cloud network 102 can send serialized data indicating the AR world to at least one connected client. Cloud network 102 can send serialized data indicating the AR world to at least one connected client device (such as client devices 104a-d) by sending the serialized data. For example, AR world service 112 can send data indicating an object, content associated with the object, and information indicating the position of the content relative to the object to at least one connected client device for displaying the content. At least one connected client device can include at least one client device from which cloud network 102 receives a connection request at 202. If at least one connected client device includes an application, such as application 106, cloud network 102 can send serialized data indicating the AR world to the application. The application can generate and output the AR world on the interface (such as interface 108) of at least one connected client device. Refer to the following... Figure 6 Describe an exemplary user interface.

[0038] At point 208, cloud network 102 can receive a first indication of user input from at least one connected client. This first indication of user input can be received from at least one connected client device (such as client devices 104a-d). For example, cloud network 102 can receive the first indication of user input from at least one connected client via at least one connected client device. If the at least one connected client device includes an application (such as application 106), the at least one connected client can use the application to input the first indication of user input. Cloud network 102 can store the first indication of the received data in a database (such as database 114).

[0039] The user's initial input may include data, such as text data. Text data may include at least one of letters, words, or phrases. Text data may include information about the text that at least one connected client wants to overlay onto the AR world. Text data may include various fonts, colors, or sizes, and can be written in any language.

[0040] The initial indication input by the user may include a coordinate frame. The coordinate frame can indicate where in the AR world at least one connected client wants the initial indication input to be overlaid. For example, if the AR world displays buildings, at least one connected client might want the text data to be displayed near the bottom of the building. The coordinate frame can indicate the location near the bottom of the building. In another example, at least one connected client might want the text data to be displayed near the top of the building. The coordinate frame can indicate the location near the top of the building.

[0041] At point 210, cloud network 102 can broadcast a first indication entered by the user to at least one connected client. The first indication entered by the user can be broadcast to any client device connected to the same server, such as client devices 104a-d. Each client connected to the same server can view the first indication entered by the user after it has been broadcast by cloud network 102. For example, client devices 104a and 104b can both be connected to the same server, such as a server associated with a building. If text data is received from a connected client using client device 104a, the connected client can view the broadcast text data using client device 104b. If the first indication entered by the user includes a coordinate frame, cloud network 102 can broadcast the first indication entered by the user at the location indicated by the coordinate frame.

[0042] At 212, cloud network 102 can receive a second indication of user input from at least one connected client. The second indication of user input can be received from at least one connected client device (such as client devices 104a-d). For example, cloud network 102 can receive the second indication of user input from at least one connected client device. If at least one connected client device includes an application (such as application 106), the at least one connected client can use the application to input the second indication of user input. Cloud network 102 can store the second indication of the received data in a database (such as database 114). The first and second indications of user input can be received from clients with the same connection or from clients with different connections.

[0043] The second instruction input by the user may include data, such as digital image data. The digital image data may include at least one of drawings or emojis. The digital image data may include digital images that at least one connected client wants to overlay onto the AR world. If the digital image data includes drawings, the drawings may be created by at least one connected client using an application (such as application 106). At least one connected client may use the application to “draw” the drawing onto an object. The drawing may be of various shapes or colors. In one embodiment, the digital image data may include at least one of images or videos stored in or captured in real time by the client device.

[0044] The second instruction input by the user can include a coordinate frame. The coordinate frame can indicate where in the AR world at least one connected client wants the second instruction input by the user to be overlaid. For example, if the AR world displays buildings, at least one connected client might want the digital image data to be displayed near the bottom of the building. The coordinate frame can indicate the location near the bottom of the building. In another example, at least one connected client might want the digital image data to be displayed near the top of the building. The coordinate frame can indicate the location near the top of the building. If the digital image data is a drawing, at least one client can specify the desired coordinate frame by “drawing” the drawing onto the desired location of the object.

[0045] At step 214, cloud network 102 may broadcast a second indication of data to at least one connected client. The user-inputted second indication may be broadcast to any client device connected to the same server, such as client devices 104a-d. Each client connected to the same server is able to view the user-inputted second indication after it has been broadcast by cloud network 102. For example, client devices 104a and 104b may both be connected to the same server, such as a server associated with a building. If digital image data is received from a client connected using client device 104a, the connected client can view the broadcast digital image data using client device 104b. If the user-inputted second indication includes a coordinate frame, cloud network 102 may broadcast the user-inputted second indication at the location indicated by the coordinate frame.

[0046] At point 216, cloud network 102 can receive a third indication of user input from at least one connected client. This third indication can be received from at least one connected client device (such as client devices 104a-d). For example, cloud network 102 can receive the third indication of user input from at least one connected client via at least one connected client device. If at least one connected client device includes an application (such as application 106), the at least one connected client can use that application to input the third indication of user input. Cloud network 102 can store the third indication of the received data in a database (such as database 114). The first, second, and third indications of user input can be received from clients on the same connection or from clients on different connections.

[0047] The third indication input by the user may include user feedback, such as "like" or "dislike". In some embodiments, user feedback may indicate whether at least one connected client likes or dislikes the AR world. In another embodiment, user feedback may include comments about the AR world from at least one connected client. For example, if the AR world includes buildings, landmarks, products, or any other objects, the user feedback may indicate whether at least one connected client likes or dislikes the buildings, landmarks, products, or any other objects. In other embodiments, user feedback may also indicate whether at least one connected client likes or dislikes a first or second indication input by the user that has been broadcast by the cloud network 102.

[0048] If the third instruction entered by the user includes "like", then at 218, cloud network 102 can increase the global "like" count. The global "like" count indicates how many connected clients have "liked" the AR world. If the third instruction entered by the user includes "dislike", then at 218, cloud network 102 can decrease the global "like" count. At 220, cloud network 102 can broadcast the third instruction of the data to at least one connected client. The third instruction entered by the user can be broadcast to at least one connected client via at least one connected client device (such as client devices 104a-d). After cloud network 102 broadcasts the third instruction, each client connected to the same server may be able to view the third instruction entered by the user. For example, both client devices 104a and 104b can be connected to the same server, such as a server associated with a building. If a "like" is received from a client connected using client device 104a, then the connected client can view that "like" using client device 104b. A client connected using client device 104b can view the "like" by seeing the increase in the global "like" count.

[0049] Figure 3 The illustration depicts an example process 300 executed by a client device (e.g., client device 104a-d). Client device 104a-d can execute process 300 to view and share information related to a real-life environment. Although in Figure 3 The operations are described as a series of operations, but those skilled in the art will understand that various embodiments may add, delete, reorder, or modify the described operations.

[0050] At 302, at least one client device (such as client devices 104a-d) can initiate an object tracking model, such as object tracking model 110. In one embodiment, at least one client device can automatically initiate the object tracking model by pointing its camera at an object (such as a landmark or product). In another embodiment, a user of at least one client device can manually initiate the object tracking model. As discussed above, the object tracking model can determine which object the camera of at least one client device is pointing at. In the example, the object tracking model can determine that at least one client device is near a first object or a second object different from the first object. The object tracking model can be located at a server located away from at least one client device or on at least one client device. If at least one connected client device includes an application (such as application 106), the object tracking model can be part of the application. In another embodiment, the object tracking model may not be part of the application. As discussed above and Figure 4a As shown, the object tracking model can be configured to detect objects. In one embodiment, the object tracking model can also be configured to detect the position of the camera relative to the object. For example, as... Figure 4a As shown, when the object tracking model detects object 402, the object tracking model can determine the position of the client device's camera relative to the object, positioned on one side of one of the four planar constructs 404a, 404b, 404c, or 404d. The planar constructs are one embodiment; however, other constructs (such as...) Figure 4b Those shown in the document are also within the scope of this disclosure.

[0051] At point 304, at least one client device (such as client devices 104a-d) can connect to a server, such as AR service 112, based on objects detected by the object tracking model. This at least one client device can connect to the server by sending a connection request to a network including the server (such as cloud network 102). The connection request from the at least one client device can indicate an object detected by the object tracking model. If the object tracking model has determined that the camera of the at least one client device is pointing at a specific landmark or product, the connection request from the at least one client device can include a request to join a server associated with the landmark or product. For example, if the object tracking model has determined that the camera of the at least one client device is pointing at a building, the connection request from the at least one client device can include a request to connect to a server associated with the building.

[0052] In this embodiment, more than one client device can connect to the same server. This may happen, for example, if the cameras of two client devices (such as client devices 104a-b) are pointed at the same object. If the cameras of two client devices are both pointed at the same building, both devices can connect to the server associated with that building. More than one client device can connect to the same server simultaneously or at different times. For example, the camera of client device 104a may be pointed at the building at a first time, and the camera of client device 104b may be pointed at the building at a later second time. In this case, client device 104a can connect to the server before client device 104b connects to the server. In another example, the cameras of client devices 104a-b can be pointed at the building simultaneously. In this case, client devices 104a-b can connect to the server simultaneously.

[0053] At point 306, at least one client device (such as client devices 104a-d) can send a message to a server such as AR service 112. This message can be a "hello" message. The "hello" message can initiate a handshake between at least one client device and the server. During the handshake, at least one client device and the server can exchange information about the protocol used to establish the communication link. For example, during the handshake, at least one client device and the server can signal to each other that they are both active and can agree on which protocols are being used. After the protocol for establishing the communication link has been established, at least one client device and the server can communicate with each other. At least one client device can download AR world data 116 associated with objects detected by the object tracking model from a database (such as database 114).

[0054] At 308, at least one client device (such as client devices 104a-d) can receive AR world data 116 from the server. The at least one client device, including an application (such as application 106), can use the AR world data 116 to recreate the AR world, including objects, in a local AR world view. A user of the at least one client device can view objects in the local AR world view on an interface (such as interface 108) of the at least one client device. Various aspects of the disclosed user interface are described herein with respect to certain examples and embodiments, which are intended to illustrate and not limit this disclosure.

[0055] For example, such as Figures 5a to 5b As shown, at least one client device (such as client device 104a-d) can recreate objects (such as...) in the local AR world view 500. Figure 4a The AR world of object 402. Figure 5a The illustration shows an exemplary local AR world view 500 recreated by a client device when the client device's camera is positioned relative to an object on the side opposite to plane 404a. The local AR world view 500 may include user input previously sent to the server from other client devices. For example, the local AR world view 500 may include text data or digital image data previously sent to the server from other client devices.

[0056] In an embodiment, user messages and graphics can be projected onto at least one of planes 404a-d, and at least one client device can view the messages projected onto at least one plane 404a-d based on calculations determining where the client device is located relative to the object (i.e., the building and the plane). For example, when the camera of a client device is positioned relative to the object on the side opposite to plane 404a, the camera can be pointed at a first part of the building, and the client device can view messages projected onto plane 404a, such as words 508a-b, drawing 506, or building labels. If the cameras of two client devices are both positioned relative to the building on the side opposite to plane 404a, but have different views of the building, they can view different messages projected onto plane 404a. For example, if the camera of a first client device is positioned on the side opposite to plane 404a but angled toward the bottom of the building, the first client device may only be able to view messages positioned toward the bottom of the building. Similarly, if the camera of the second client device is positioned on the side opposite to plane 404a but at an angle toward the center or top of the building, the second client device may only be able to view messages positioned toward the center or top of the building.

[0057] As another example, when the client device's camera is positioned on the side opposite to plane 404b, the camera can be pointed at the second part of the building, and the client device can view messages projected onto plane 404b, such as the word "Howdy". When the client device's camera is positioned on the side opposite to plane 404c, the camera can be pointed at the third part of the building, and the client device can view messages projected onto plane 404c, such as the word "508c". When the client device's camera is positioned on the side opposite to plane 404d, the camera can be pointed at the fourth part of the object, and the client device can view messages projected onto plane 404d, such as the word "Awesome".

[0058] In one embodiment, when the client device's camera is positioned on the side opposite to a specific plane, the client device can view only the user messages and graphics projected onto that specific plane. In another embodiment, when the client device's camera is positioned on the side opposite to a specific plane, the client device may be able to view user messages and graphics projected onto both the specific plane and adjacent planes. For example, when the client device's camera is positioned on the side opposite to plane 404a, the client device may be able to view user messages and graphics projected onto plane 404a, as well as user messages and graphics projected onto at least one of planes 404b to 404d. For example, user messages projected onto plane 404a may appear normally, and viewing user messages projected onto plane 404c (such as user message 508c) may appear in a mirrored manner.

[0059] Broadcast content, such as words 508a-c or picture 506, can be positioned at a static location within the local AR world view 500, or it can move across the local AR world view 500. As an example, and not a limitation, broadcast content can move from right to left within the local AR world view 500. In one example, when broadcast content is moving and reaches the leftmost edge of a particular plane, the broadcast content can reappear at the rightmost edge of the plane and begin moving from right to left again. In another example, broadcast content can move from right to left within the local AR world view 500. In one embodiment, broadcast content can move across the local AR world view 500 along a predetermined trajectory. In yet another example, broadcast content can be at a static location within the local AR world view 500.

[0060] Broadcast content can be positioned within the local AR world view 500 based on coordinate frames. These coordinate frames can be provided by the user inputting the content or by a coordinate frame selected by the server. In this embodiment, the user can pay to select a coordinate frame. If the broadcast content is in a static position within the local AR world view 500, the coordinate frame can indicate that static position. If the broadcast content is moving within the local AR world view 500, the coordinate frame can indicate the frames within which the broadcast content can move. For example, the coordinate frame could indicate that the broadcast content may be moving near the center of the local AR world view 500.

[0061] For example, such as Figures 6 to 7 As shown, a user can view objects (such as building 602) in the local AR world view on the client device's interface. Based on the client device's camera's position relative to the object, the local AR world view can include different parts of the object. The user can view different parts of the object depending on which side of the planar structure the user is on. For example, if the client device's camera is positioned facing the bottom of the planar structure, the user can view the bottom of a portion of the object within that planar structure, as well as a user message covering the bottom of that portion of the object. As another example, if the client device's camera is positioned facing the top of the planar structure, the user can view the top of a portion of the object within that planar structure, as well as a user message covering the top of that portion of the object.

[0062] The interface may include various options for the user to choose from. In an embodiment, the options may include at least one of the following: browsing option 604, speed option 606, filtering option 608, beautification option 610, timer option 612, text option 614, "like" option 616, or drawing option 618. When selected, the various options may allow the user to send different types of user input to the server associated with the local AR world view. For example, the various options may allow the user to send different types of user input to overlay on portions of objects in a particular planar construction. For example, at 310, the user may send user input associated with the local AR world view to the server. At 310, at least one client device (such as client devices 104a-d) may send indications of user input to the server. If at least one client device includes an application (such as application 106), the user of at least one client device may use the application to input the indications of user input. The cloud network 102 may store the received data indications in a database (such as database 114).

[0063] In this embodiment, the user input indication includes text data. The user can send text data associated with the local AR world view to the server using an interface configured to receive text input (such as interface 108). For example, the user can use characters provided on a numeric keypad to input letters, words, or phrases into a text field. The letters, words, or phrases can be associated with objects. If the text data is configured to move across the interface, the user can select the speed for the text data by selecting speed button 606. As another example, the speed of the text data can be selected by the server. In another example, the text data can be statically positioned in the local AR world, with the static position selected by the user or the server. If the text data is moving, the speed of the text data can indicate how quickly the text data will scroll across the interface of the connected user device after being broadcast by the server.

[0064] Text data can be displayed on the local AR world view for a predetermined amount of time. In the example, the user can select the time for the text data by selecting timer button 612. The time for the text data can indicate the duration for which the text data will scroll across the interface of the connected user device after being broadcast by the server, or it can indicate the duration for which the text data will remain in its static position on the local AR world view. In another example, the server can select the time for the text data. In this embodiment, the user can pay to extend the duration of the text data. The duration can be any amount of time, such as 15 seconds, 30 seconds, or one minute. The user can select the color for the text data using color scale 620. The user can select the color for the text data by selecting a color on color scale 620. Multiple colors can be included on the color scale. In another example, the server can select the color of the text data. The user may need to pay to select the color of the text data.

[0065] In this embodiment, text data can be associated with a coordinate frame. The coordinate frame can be selected by the user and can indicate where the user wants the text data to be overlaid on the local AR world view. The user may need to pay to select the coordinate frame. For example, if the local AR world view displays buildings, the user might want the text data to be displayed near the bottom of the building. The coordinate frame can indicate the location near the bottom of the building. In another example, the user might want the text data to be displayed near the top of the building. The coordinate frame can indicate the location near the top of the building. The user may be given the option to pay to select the desired coordinate frame. In another example, the coordinate frame can be selected by the server. If the server selects the coordinate frame, the server can choose whether the text data is displayed near the top or bottom of the building.

[0066] In another embodiment, the user input indication includes digital image data, such as drawings, images, or emojis. The user can send digital image data associated with the local AR world view to the server by selecting drawing option 618. When the user selects drawing option 618, an interface configured to receive digital image input, such as interface 108, can be presented to the user. For example, the user can input a drawing. To input a drawing, the user can draw shapes on the interface using their finger. The digital image data is associated with an object. If the digital image data is configured to move across the interface, the user can select the speed for the digital image data by selecting speed button 606. As another example, the speed of the digital image data can be selected by the server. In another example, the digital image data can be statically positioned in the local AR world, with the static position selected by the user or the server. If the digital image data is moving, the speed of the digital image data can indicate how quickly the digital image data will scroll across the interface of the connected user device after it has been broadcast by the server. In another embodiment, the digital image data may also include at least one of images or videos stored in or captured in real time by the client device.

[0067] Digital image data can be displayed on the local AR world view for a predetermined amount of time. In one example, a user can select the time for the digital image data by selecting timer button 612. The time for the digital image data can indicate the duration for which the digital image data will scroll across the interface of a connected user device after being broadcast by the server, or it can indicate the duration for which the digital image data will remain in a static position on the local AR world view. In another example, the server can select the time for the digital image data. In one embodiment, the user can pay to extend the duration of the digital image data. The duration can be any amount of time, such as 15 seconds, 30 seconds, or one minute. The user can select the color for the digital image data using color scale 620. The user can select the color for the digital image data by selecting a color on color scale 620. Multiple colors can be included on the color scale. In another example, the server can select the color for the digital image data. The user may need to pay to select the color for the digital image data.

[0068] In this embodiment, digital image data can be associated with a coordinate frame. The coordinate frame can be selected by the user and can indicate where the user wants the digital image data to be overlaid on the local AR world view. The user may need to pay to select the coordinate frame. For example, if the local AR world view displays buildings, the user might want the digital image data (such as a drawing) to be displayed near the bottom of the building. The coordinate frame can indicate the location near the bottom of the building. In another example, the user might want the digital image data to be displayed near the top of the building. The coordinate frame can indicate the location near the top of the building. The user may be given the option to pay to select the desired coordinate frame. In another example, the coordinate frame can be selected by the server. If the server selects the coordinate frame, the server can choose whether the digital image data is displayed near the top or bottom of the building.

[0069] In another embodiment, the user input indication includes user feedback data, such as "like" or "dislike," or comments on an object. User feedback can indicate whether the user likes or dislikes the AR world. User feedback can include the user's comments on the AR world. For example, if the AR world includes buildings, the user feedback can indicate whether the user likes or dislikes the building. User feedback can also indicate whether the user likes or dislikes user input that has already been entered by other users. A user can "like" the AR world by selecting the "like" option 616. If a user has already "liked" the AR world, the user can also "dislike" the AR world by selecting the "like" option 616.

[0070] Return to reference Figure 3 At point 312, at least one client device (such as client devices 104a-d) can receive broadcast user input. The broadcast user input may include user input by the user or other users, such as text data or digital image data. If the user input includes a coordinate frame, the user input can be broadcast to at least one client device at the location indicated by the coordinate frame. At point 314, a user of at least one client device (such as client devices 104a-d) can press a "like" button, such as "like" option 616. At point 316, at least one client device (such as client devices 104a-d) can receive an updated value for global likes. The updated value for global likes can reflect an increase or decrease in the global like value. For example, each time the user selects the "like" button, such as "like" option 616, the global like value can increase by one. Whenever the user "dislikes" the AR world, the global like value can decrease by one. The updated value for global likes can be displayed on the interface of at least one client device (such as interface 108). For example, the updated value for global likes can be displayed on the interface below the "like" button.

[0071] Figure 8 The illustration shows computing devices that can be used in various aspects, such as Figure 1 The services, networks, modules, and / or devices described herein. About Figure 1 In the example architecture, the message service, interface service, processing service, content service, cloud network, and client can all be provided by... Figure 8 This is achieved through one or more instances of the computing device 800. Figure 8 The computer architecture shown illustrates conventional server computers, workstations, desktop computers, laptop computers, tablet computers, network devices, PDAs, e-readers, digital cellular phones, or other computing nodes, and can be used to perform any aspect of the computer described herein, such as for implementing the methods described herein.

[0072] The computing device 800 may include a substrate or “motherboard”, which is a printed circuit board to which multiple components or devices may be connected via a system bus or other electrical communication paths. One or more central processing units (CPUs) 804 may operate in conjunction with a chipset 806. The CPUs (multiple) 804 may be standard programmable processors necessary for performing arithmetic and logical operations required to operate the computing device 800.

[0073] Multiple CPUs (804) can perform necessary operations by manipulating switching elements that distinguish and change these states, transitioning from one discrete physical state to the next. Switching elements typically include electronic circuitry (such as flip-flops) that holds one of two binary states, and electronic circuitry (such as logic gates) that provides an output state based on a logical combination of the states of one or more other switching elements. These basic switching elements can be combined to create more complex logic circuits, including registers, adder-subtractor units, arithmetic logic units, floating-point units, etc.

[0074] The CPU 804 can be expanded or replaced with other processing units such as GPUs. The GPUs may include processing units dedicated to, but not necessarily limited to, highly parallel computing (such as graphics and other visualization-related processing).

[0075] Chipset 806 may provide an interface between CPU(s) 804 and the remaining components and devices on the substrate. Chipset 806 may provide an interface to random access memory (RAM) 808, which serves as the main memory in computing device 800. Chipset 806 may further provide an interface to computer-readable storage media, such as read-only memory (ROM) 820 or non-volatile RAM (NVRAM) (not shown), for storing basic routines that can help start computing device 800 and transfer information between various components and devices. ROM 820 or NVRAM may also store other software components necessary for the operation of computing device 800 according to the aspects described herein.

[0076] Computing device 800 can operate in a networked environment using logical connections to remote computing nodes and computer systems via a local area network (LAN). Chipset 806 may include functionality for providing network connectivity via a network interface controller (NIC) 822 (such as a Gigabit Ethernet adapter). NIC 822 may be able to connect computing device 800 to other computing nodes via network 816. It should be understood that multiple NICs 822 may be present in computing device 800 to connect the computing device to other types of networks and remote computer systems.

[0077] Computing device 800 can be connected to mass storage device 828, which provides non-volatile storage for the computer. Mass storage device 828 can store system programs, application programs, other program modules, and data, which have been described in more detail herein. Mass storage device 828 can be connected to computing device 800 via storage controller 824, which is connected to chipset 806. Mass storage device 828 can consist of one or more physical storage units. Mass storage device 828 may include management unit 810. Storage controller 824 can interface with physical storage units via a Serial Attached SCSI (SAS) interface, a Serial Advanced Technology Attachment (SATA) interface, a Fibre Channel (FC) interface, or other types of interfaces used for physical connection and data transfer between the computer and physical storage units.

[0078] The computing device 800 can store data on the mass storage device 828 by transforming the physical state of the physical storage units to reflect the stored information. The specific transformation of the physical state may depend on various factors and the different implementations described herein. Examples of such factors may include, but are not limited to, the technology used to implement the physical storage units and whether the mass storage device 828 is characterized as main memory or secondary memory.

[0079] For example, computing device 800 can store information in mass storage device 828 by issuing instructions via storage controller 824 to change the magnetic properties of a specific location within a disk drive unit, the reflection or refraction properties of a specific location in an optical storage unit, or the electrical properties of a specific capacitor, transistor, or other discrete component in a solid-state storage unit. Other transformations of the physical medium are possible without departing from the scope and spirit of this description; the foregoing examples are provided merely for the purpose of illustration. Computing device 800 can further read information from mass storage device 828 by detecting the physical state or characteristics of one or more specific locations within the physical storage unit.

[0080] In addition to the mass storage device 828 described above, the computing device 800 can access other computer-readable storage media to store and retrieve information, such as program modules, data structures, or other data. Those skilled in the art will understand that a computer-readable storage medium can be any available medium that provides storage for non-transitory data and can be accessed by the computing device 800.

[0081] By way of example and not limitation, computer-readable storage media may include volatile and non-volatile media, transient computer-readable storage media and non-transient computer-readable storage media implemented in any method or technology, as well as removable and non-removable media. Computer-readable storage media include, but are not limited to, RAM, ROM, erasable programmable ROM (“EPROM”), electrically erasable programmable ROM (“EEPROM”), flash memory or other solid-state memory technologies, optical disc ROM (“CD-ROM”), digital versatile disc (“DVD”), high-definition DVD (“HD-DVD”), BLU-RAY or other optical storage devices, magnetic tape cassettes, magnetic tape, disk storage devices, other magnetic storage devices, or any other medium that can be used to store desired information in a non-transitory manner.

[0082] Such as Figure 8 The mass storage device 828 depicted herein can store an operating system used to control the operation of the computing device 800. The operating system may include a version of the LINUX operating system. The operating system may include a version of the WINDOWS server operating system from Microsoft Corporation. According to a further aspect, the operating system may include a version of the UNIX operating system. Various mobile phone operating systems, such as iOS and Android, may also be used. It should be understood that other operating systems may also be utilized. The mass storage device 828 may store other systems or applications and data used by the computing device 800.

[0083] Mass storage device 828 or other computer-readable storage medium may also be encoded with computer-executable instructions that, when loaded into computing device 800, transform the computing device from a general-purpose computing system into a special-purpose computer capable of implementing the aspects described herein. As described above, these computer-executable instructions transform computing device 800 by specifying how CPU(s) 804(s) transition between states. Computing device 800 can access the computer-readable storage medium storing the computer-executable instructions, which, when executed by computing device 800, can perform the methods described herein.

[0084] Computing devices (such as Figure 8 The computing device 800 depicted may also include an input / output controller 832 for receiving and processing input from multiple input devices, such as a keyboard, mouse, touchpad, touchscreen, electronic pen, or other types of input devices. Similarly, the input / output controller 832 may provide output to a display, such as a computer monitor, flat panel display, digital projector, printer, plotter, or other types of output device. It is understood that the computing device 800 may not include... Figure 8 All components shown may include Figure 8 Other components not explicitly shown in the document, or those that can be utilized with... Figure 8 The architecture shown is completely different.

[0085] As described herein, a computing device can be a physical computing device, such as... Figure 8 The computing device 800. A computing node may also include virtual machine host processes and one or more virtual machine instances. Computer-executable instructions can be indirectly executed by the physical hardware of the computing device by interpreting and / or executing instructions stored and executed in the context of the virtual machine.

[0086] It should be understood that the method and system are not limited to a particular method, a particular component, or a particular implementation. It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting.

[0087] As used in the specification and appended claims, the singular forms “a,” “an,” and “the” include plural indicators unless the context explicitly specifies otherwise. A range herein may be expressed as from “about” a particular value, and / or to “about” another particular value. When expressing such a range, another embodiment includes from one particular value and / or to another particular value. Similarly, when a value is expressed as an approximation using the antecedent “about,” it should be understood that the particular value forms another embodiment. It should further be understood that each endpoint in the range is significant both relative to and independent of the other endpoint.

[0088] "Optional" or "optionally" means that the event or situation described below may or may not occur, and the description includes both the possibility that the event or situation will occur and the possibility that it will not occur.

[0089] Throughout the description and claims of this specification, the word “comprise” and variations thereof, such as “comprising” and “comprises”, mean “including, but not limited to” and are not intended to exclude, for example, other components, integers, or steps. “Example” means “one example” and is not intended to convey indications of preferred or ideal embodiments. “Like” is not used in a limiting sense but for interpretive purposes.

[0090] Components that can be used to perform the described methods and systems are described. When describing combinations, subsets, interactions, groups, etc., of these components, it should be understood that although specific references to each of the various individual and collective combinations and arrangements of these components may not be explicitly described, each is specifically contemplated and described herein for all methods and systems. This applies to all aspects of this application, including but not limited to operations in the described methods. Therefore, if multiple additional operations are available, it should be understood that each of these additional operations can be performed by any particular embodiment or combination of embodiments of the described methods.

[0091] The method and system can be more readily understood by referring to the following detailed description of preferred embodiments, the examples included therein, and the accompanying drawings and their descriptions.

[0092] As those skilled in the art will understand, the method and system may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the method and system may take the form of a computer program product on a computer-readable storage medium having computer-readable program instructions (e.g., computer software) implemented therein. More particularly, the method and system may take the form of network-implemented computer software. Any suitable computer-readable storage medium may be used, including hard disks, CD-ROMs, optical storage devices, or magnetic storage devices.

[0093] Embodiments of the methods and systems are described below with reference to block diagrams and flowcharts illustrating the methods, systems, apparatus, and computer program products. It should be understood that each block in the block diagrams and flowcharts, as well as combinations of blocks in the block diagrams and flowcharts, can be implemented accordingly by computer program instructions. These computer program instructions can be loaded onto a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute on the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart.

[0094] These computer program instructions may also be stored in a computer-readable storage medium that can instruct a computer or other programmable data processing apparatus to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of writing comprising computer-readable instructions for implementing the functions specified in one or more boxes of the flowchart. The computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more boxes of the flowchart.

[0095] The various features and processes described above can be used independently of each other or in combination in various ways. All possible combinations and sub-combinations are intended to fall within the scope of this disclosure. Furthermore, in some implementations, certain method or process blocks may be omitted. The methods and processes described herein are not limited to any particular order, and the blocks or states associated with them may be executed in other suitable orders. For example, the described blocks or states may be executed in a different order than specifically described, or multiple blocks or states may be combined in a single block or state. Example blocks or states may be executed serially, in parallel, or in some other manner. Blocks or states may be added to or removed from the described example embodiments. The example systems and components described herein may be configured differently from those described. For example, elements may be added, removed, or rearranged compared to the described example embodiments.

[0096] It should also be understood that various items are shown to be stored in memory or on storage devices when in use, and these items, or portions thereof, may be transferred between memory and other storage devices for memory management and data integrity purposes. Alternatively, in other embodiments, some or all of the software modules and / or systems may be executed in memory on another device and communicate with the illustrated computing system via inter-computer communication. Furthermore, in some embodiments, some or all of the systems and / or modules may be implemented or provided in other ways, such as at least partially in firmware and / or hardware, including, but not limited to, one or more application-specific integrated circuits (“ASICs”), standard integrated circuits, controllers (e.g., by executing appropriate instructions, and including microcontrollers and / or embedded controllers), field-programmable gate arrays (“FPGAs”), complex programmable logic devices (“CPLDs”), etc. Some or all of the modules, systems, and data structures may also be stored (e.g., as software instructions or structured data) on computer-readable media (such as hard disks, memory, networks, or portable media products) for retrieval by appropriate devices or via appropriate connections. Systems, modules, and data structures can also be transmitted as generated data signals (e.g., as part of a carrier or other analog or digital propagation signal) over a variety of computer-readable transmission media, including wireless and wired / cable-based media, and can take many forms (e.g., as part of a single or multiplexed analog signal, or as multiple discrete digital packets or frames). In other embodiments, such computer program products can also take other forms. Therefore, the invention can be practiced with other computer system configurations.

[0097] Although methods and systems have been described in conjunction with preferred embodiments and specific examples, this is not intended to limit the scope to the specific embodiments illustrated, as the embodiments herein are illustrative rather than restrictive in all respects.

[0098] Unless otherwise expressly stated, this document does not imply that any method described herein requires its operations to be performed in a particular order. Therefore, no order is intended to be inferred in any way where the method claims do not actually state the order in which their operations are followed, or where the claims or specification do not otherwise specifically state that the operations are limited to a particular order. This applies to any possible non-express basis of interpretation, including: logical questions concerning the arrangement of steps or the flow of operations; simple meanings derived from grammatical organization or punctuation; and the number or type of embodiments described in the specification.

[0099] It will be apparent to those skilled in the art that various modifications and variations can be made without departing from the scope or spirit of this disclosure. Other embodiments will be apparent to those skilled in the art in light of the description and practice presented herein. The description and example drawings are to be considered illustrative only, and the true scope and spirit are indicated by the appended claims.

Claims

1. A system for content processing, comprising: At least one database; as well as At least one computing device communicating with the at least one database, the at least one computing device being configured to perform an operation including: The at least one database stores content associated with a unique stationary landmark in the real-world environment, the content being received from a first plurality of user devices, the first plurality of user devices providing information indicating the position of at least one of the first plurality of user devices relative to the unique stationary landmark in the real-world environment, wherein the camera of the first plurality of user devices is pointed at the unique stationary landmark in the real-world environment; Receive information from a second plurality of user devices instructing the second plurality of user devices to approach a unique immovable landmark in the real-world environment, wherein the second plurality of user devices connect to a server in response to approaching the unique immovable landmark; Sending the content and information indicating the position of the content relative to the unique stationary landmark to the second plurality of user devices for displaying the content as an overlay map of the camera view of the unique stationary landmark by each of the second plurality of user devices; and Additional content associated with unique, immovable landmarks in the real-world environment is stored in at least one database. This additional content is received from a plurality of user devices, which provide information indicating the association of the additional content with the unique, immovable landmarks in the real-world environment. The system includes a server associated with a unique, immovable landmark in the real-world environment, the server being configured to broadcast the content and additional content in real-time or near real-time to all user devices connected to the server.

2. The system of claim 1, wherein receiving information from the second plurality of user equipment indicating that the second plurality of user equipment is approaching a unique stationary landmark in the real-world environment comprises: Receive information from the second plurality of user equipment indicating that the second plurality of user equipment is approaching a real-world environment including a unique stationary landmark in the real-world environment or an image including a unique stationary landmark in the real-world environment.

3. The system of claim 1, wherein the content associated with a unique stationary landmark in the real-world environment includes at least one of text, graphics, or information instructing user feedback.

4. A method for content processing, applied to a server associated with a unique immovable landmark in a real-world environment, the method comprising: Content associated with a unique stationary landmark in the real-world environment is stored in at least one database. The content is received from a first plurality of user devices, which provide information indicating the position of at least one of the first plurality of user devices relative to the unique stationary landmark in the real-world environment, wherein the camera of the first plurality of user devices is pointed at the unique stationary landmark in the real-world environment. Receive information from a second plurality of user devices instructing the second plurality of user devices to approach a unique stationary landmark in the real-world environment, wherein the second plurality of user devices connect to the server in response to approaching the unique stationary landmark; Send the content and information indicating the position of the content relative to the unique stationary landmark to the second plurality of user devices for displaying the content as an overlay map of the camera view of the unique stationary landmark by each of the second plurality of user devices; as well as Additional content associated with a unique immovable landmark in the real-world environment is stored in at least one database. This additional content is received from a plurality of user devices, which provide information indicating the association of the additional content with the unique immovable landmark. The server is configured to broadcast the content and the additional content to all user devices connected to the server in real time or near real time.

5. The method of claim 4, wherein receiving information from the second plurality of user equipment indicating that the second plurality of user equipment is approaching a unique stationary landmark in the real-world environment comprises: Receive information from the second plurality of user equipment indicating that the second plurality of user equipment is approaching a real-world environment including a unique stationary landmark in the real-world environment or an image including a unique stationary landmark in the real-world environment.

6. The method of claim 4, wherein the content associated with a unique immovable landmark in the real-world environment includes at least one of text, images, or information instructing user feedback.

7. A computing device, comprising: At least one processor; as well as At least one memory communicatively coupled to the at least one processor and storing instructions, which, when executed by the at least one processor, cause the computing device to: Identify the only stationary landmark in the real-world environment near the computing device; In response to determining that the computing device is close to the unique immovable landmark, a connection is established with a server, wherein the server is associated with the unique immovable landmark; Send information instructing the computing device to approach a unique, immovable landmark in the real-world environment; Content associated with a unique stationary landmark in the real-world environment and information indicating the position of the content relative to the unique stationary landmark are received from at least one database. The content is sent from a first plurality of user devices to the at least one database. The first plurality of user devices provide information indicating the position of at least one of the first plurality of user devices relative to the unique stationary landmark in the real-world environment, wherein the camera of the first plurality of user devices is pointed at the unique stationary landmark in the real-world environment. On the interface of the computing device, the content associated with a unique stationary landmark in the real-world environment is displayed as an overlay on the camera view of the unique stationary landmark; as well as Send additional content associated with a unique immovable landmark in the real-world environment to the at least one database, and provide information indicating that the additional content is associated with the unique immovable landmark; The content therein and other content are broadcast in real time or near real time to all user devices connected to the server.

8. The computing device of claim 7, wherein sending information indicating that the computing device is approaching a unique, immovable landmark in the real-world environment comprises: Send information instructing the computing device to approach a real-world environment that includes a unique stationary landmark in the real-world environment, or to approach an image that includes a unique stationary landmark in the real-world environment.

9. The computing device of claim 7, wherein the content associated with a unique stationary landmark in the real-world environment includes at least one of text, graphics, or information instructing user feedback.

10. The computing device of claim 7, wherein the at least one memory further stores instructions that, when executed by the at least one processor, cause the computing device to: The system sends to the at least one database content associated with a second unique fixed landmark in the real-world environment, along with information about the location of the computing device relative to the second unique fixed landmark in the real-world environment, wherein the camera of the computing device is pointed at the second unique fixed landmark in the real-world environment. The second unique immovable landmark in the real-world environment is different from the unique immovable landmark in the real-world environment.

11. A method for content processing, comprising: Identify the unique stationary landmark in the real-world environment near the computing device; In response to determining that the computing device is close to the unique immovable landmark, a connection is established with a server, wherein the server is associated with the unique immovable landmark; Send a message instructing the computing device to approach a unique, immovable landmark in the real-world environment; Content associated with a unique stationary landmark in the real-world environment and information indicating the position of the content relative to the unique stationary landmark are received from at least one database. The content is sent from a first plurality of user devices to the at least one database. The first plurality of user devices provide information indicating the position of at least one of the first plurality of user devices relative to the unique stationary landmark in the real-world environment, wherein the camera of the first plurality of user devices is pointed at the unique stationary landmark in the real-world environment. On the interface of the computing device, the content associated with a unique stationary landmark in the real-world environment is displayed as an overlay on the camera view of the unique stationary landmark; as well as Send additional content associated with a unique immovable landmark in the real-world environment to the at least one database, and provide information indicating that the additional content is associated with the unique immovable landmark; The content therein and other content are broadcast in real time or near real time to all user devices connected to the server.

12. The method of claim 11, wherein sending information indicating that the computing device is approaching a unique, immovable landmark in the real-world environment comprises: Send information instructing the computing device to approach a real-world environment that includes a unique stationary landmark in the real-world environment, or to approach an image that includes a unique stationary landmark in the real-world environment.

13. The method of claim 11, wherein the content associated with a unique immovable landmark in the real-world environment includes at least one of text, graphics, or information instructing user feedback.

14. The method of claim 11, further comprising: The system sends to the at least one database content associated with a second unique fixed landmark in the real-world environment, along with information about the location of the computing device relative to the second unique fixed landmark in the real-world environment, wherein the camera of the computing device is pointed at the second unique fixed landmark in the real-world environment. The second unique immovable landmark in the real-world environment is different from the unique immovable landmark in the real-world environment.

Images