Automatic image formation system
By automatically classifying pictures in the list in the GUI of the network site and using machine learning models to determine the image classification, the problems of result accuracy and computing resource consumption during complex queries are solved, and user navigation efficiency and query speed are improved.
Patent Information
- Application Number
- JP2024176835
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-10-10
- Filing Date
- 2024-10-09
- Publication Date
- 2025-04-22
- Estimated Expiration
- 2044-10-09
AI Technical Summary
Existing network sites find it difficult to return accurate results in a timely manner when processing complex queries, resulting in excessive consumption of computing resources and complex and time-consuming user navigation.
By automatically or semi-automatically classifying images in the list in the graphical user interface (GUI) of a network site, users can place images in corresponding areas based on the classification and analyze images using machine learning models to determine their classification.
It improves the navigation efficiency of users on network sites, reduces the number of times users jump back and forth between multiple pages, reduces the consumption of computing resources, and ensures the accuracy of query results.
Smart Images

Figure 2025066085000001_ABST
Abstract
Description
[Technical field]
[0001] [CROSS-REFERENCE TO RELATED APPLICATIONS] This application is a non-provisional application and claims priority to U.S. Provisional Application No. 63 / 543,341, filed October 10, 2023, which is incorporated by reference herein in its entirety.
[0002] [Technical field]
[0003] The present disclosure relates generally to special purpose machines that manage data processing and improvements to such variations, and to techniques by which such special purpose machines are improved over other special purpose machines for generating reservation listings on real estate property listing sites. [Background technology]
[0004] Network site users can create content for viewing and interaction (e.g., booking, registering, subscribing, viewing listings) by other network site users. Posted content can be updated, created, or deleted, and it can be computationally challenging for a network site to return valid search results to a network site user searching for content (e.g., booking listings) with specified parameters (e.g., date, category, price, quantity). For example, when a large number of users post and update content and a large number of users submit complex searches of posted content, computational delays due to the complexity of the queries can cause inaccurate results to be returned and can result in significant computational resource consumption (e.g., processing, memory, network overhead). [Brief description of the drawings]
[0005] Various figures of the accompanying drawings depict only exemplary embodiments of the present disclosure and are not to be considered as limiting its scope. [Figure 1]FIG. 1 is a block diagram illustrating a search and display system implemented in a networked environment, according to some examples. [Diagram 2] 1 illustrates an example of a functional engine of a search and display system, according to some examples. [Diagram 3] 1 illustrates a listings network site user interface generated by a listings network platform and a search and display system (or listings search system), according to some examples. [Figure 4] 1 illustrates an exemplary configuration and user interface of a listing search system, according to some examples. [Diagram 5] 1 illustrates an exemplary configuration and user interface of a listing search system, according to some examples. [Figure 6] 1 illustrates an exemplary configuration and user interface of a listing search system, according to some examples. [Figure 7] 1 illustrates an exemplary configuration and user interface of a listing search system, according to some examples. [Figure 8] 1 illustrates a flow diagram of various processes and methods for generating listings for a listing network site, according to some examples. [Figure 9] FIG. 2 is a block diagram illustrating a software architecture used to implement the disclosed system, according to some examples. [Figure 10] A machine is illustrated as an exemplary computer system having instructions for causing the machine to implement the disclosed system, according to some examples. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0006] The following description includes systems, methods, techniques, instruction sequences, and computing machine program products embodying exemplary examples of the present disclosure. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide an understanding of various embodiments of the inventive subject matter. However, it will be apparent to those skilled in the art that examples of the inventive subject matter may be practiced without these specific details. In general, well-known instruction instances, protocols, structures, and techniques are not necessarily shown in detail.
[0007] As discussed above, returning up-to-date results for complex queries on content posted on network sites can be difficult. In addition, navigating the vast amount of content available on network sites can be very complex and time consuming. Such navigation can involve browsing multiple pages of information to find an appropriate result or action. To this end, the search and display system can be configured to efficiently search complex queries, enable browsing of network sites, return accurate results, and perform requested actions with low computational resource usage. In the following description, exemplary posted content is accommodation listings (e.g., booking listings) posted on a network site for search and interaction with other end users, although other types of network site content posted by end users and searched for by other end users may be implemented in the search and display system processes and methods as well, such as transportation, experiences, and / or events.
[0008] Generally, the listings platform may be searched for result listings that are available for a specified date range, price range, and / or other attributes such as amenities, cancellation policies, etc., that may be specified in a given query (e.g., text fields, drop-down menus, checkbox filters). To search and browse the listings platform, a user may access the listings platform on a particular user interface channel, for example, by telephone or through an application associated with the listings platform.
[0009] Sometimes, these systems present the user with results that include various images of different aspects of the corresponding listing. The images may be grouped or arranged in a random order or with no particular order at all. In either case, trying to ascertain which part of the physical accommodation is represented by a particular image can be difficult. For example, an image may include a depiction of a bed and the listing may include multiple bedrooms. Such images provide little information about which bedroom is represented by the image, confusing the user. The user may need to go back and forth between multiple images to try to determine which bedroom is represented by a particular image. This repetitive manual process can be prohibitively time consuming and very frustrating for the end user. The result can be missed opportunities and wasted computing resources.
[0010] To address these technical problems, the disclosed technology provides a network site that automatically or semi-automatically categorizes images of a particular listing in an efficient manner. That is, the network site presents an option in a graphical user interface (GUI) for arranging the images according to the classifications associated with the images. In response to receiving an input selecting the option, the network site animates a subset of the images in the GUI to be shuffled into one or more of the classifications. After, before, and / or during the animation of the subset of the images over a specified threshold period, the network site presents a template in the GUI that includes a plurality of regions, each region associated with a different classification, and populates a region of the plurality of regions associated with a different classification with each image corresponding to that classification.
[0011] Such classification of images can help users easily identify and select listings that may be of interest by accessing a GUI that presents images associated with the listings according to the classification associated with the images. In this manner, the network site can perform additional search queries, refine the initial search query, and prevent the user from navigating multiple pages of information. This, in effect, reduces the amount of computational resources that a given search end user needs to possess and consume, freeing up such resources to fulfill other tasks and other search requests.
[0012] In some aspects, techniques described herein relate to a method that includes receiving, by a network site of a listing network platform, an input including a plurality of images associated with individual listings; presenting within a GUI an option for arranging the plurality of images according to classifications associated with the plurality of images; in response to receiving the input selecting the option, animating within the GUI a subset of the plurality of images to be shuffled into one or more of the classifications; after the subset of the plurality of images has been animated for a specified threshold period, presenting within the GUI a template including a plurality of regions, each associated with a different classification; and populating a first region of the plurality of regions associated with a first classification with a first image corresponding to the first classification and populating a second region of the plurality of regions associated with a second classification with a second image corresponding to the second classification.
[0013] In some aspects, the techniques described herein relate to a method further including analyzing the plurality of images with a machine learning model to determine a respective classification associated with each of the plurality of images.
[0014] In some aspects, the techniques described herein relate to a method, in which a machine learning model analyzes a plurality of images in response to receiving an input selecting an option to arrange the plurality of images according to a classification associated with the plurality of images. The machine learning model can include a plurality of machine learning models. A first machine learning model can be used to assign an individual image a classification score indicating the likelihood that the image corresponds to a particular classification. A second machine learning model can be applied to the images to cluster the images based on similarity. This can be used when a particular listing has multiple of the same classification (e.g., multiple bedrooms). The second machine learning model can generate embeddings for the images and can cluster images with similar embeddings into the same bedroom, such that a first group of images are clustered and associated with the first bedroom and a second group of images are clustered and associated with the second bedroom.
[0015] In some aspects, the techniques described herein relate to a method, in which a machine learning model analyzes a plurality of images as each respective image of a plurality of images is received by an input, before presenting an option for pre-arranging the plurality of images within the GUI.
[0016] In some aspects, the techniques described herein relate to methods that further include receiving additional input including an additional image associated with the listing, and in response to receiving the additional input, re-analyzing the plurality of images previously classified by the machine learning model with the additional image to generate or update a classification for each of the plurality of images including the additional image.
[0017] In some aspects, techniques described herein relate to a method, wherein the first classification includes a first room type and the second classification includes a second room type.
[0018] In some aspects, the techniques described herein relate to a method that further includes allowing a user to modify a classification associated with the plurality of images by interacting with a template presented in the GUI.
[0019] In some aspects, the techniques described herein relate to a method, in which an order in which a number of areas are presented in a GUI is predefined, the order including: living area, full kitchen, kitchenette, dining area, bedroom, full bathroom, half bathroom, office, dedicated workspace, backyard, patio, balcony, front yard, deck, porch, courtyard, garden, terrace, rooftop, laundry area, garage, gym, exterior, pool, hot tub, theme room, children's playroom, bowling alley, movie theater, art studio, music studio, workshop, photography studio, darkroom, woodworking room, event space, library, game room, sunroom, and wine cellar.
[0020] In some aspects, techniques described herein relate to a method further including selecting one or more images from the plurality of images based on one or more criteria, and presenting the selected one or more images in a graphical element that includes options for arranging the plurality of images according to a classification.
[0021] In some aspects, the techniques described herein relate to a method, wherein the one or more criteria include at least one of an order in which the images are presented in a listing, or a respective classification of the images.
[0022] In some aspects, techniques described herein relate to a method, wherein the subset of the plurality of images includes one or more selected images, the method further including presenting a graphical element on the template presenting an animation of the subset of the plurality of images, each of a plurality of regions of the template including a placeholder for a respective image.
[0023] In some aspects, the techniques described herein relate to a method, wherein a placeholder comprises a grey box.
[0024] In some aspects, the techniques described herein relate to a method that further includes presenting an animation that includes a celebratory graphic over a template that includes a first region having a first image and a second region having a second image.
[0025] In some aspects, the techniques described herein relate to a method, wherein the celebratory graphic includes at least one of confetti, balloons, or a logo associated with a network site.
[0026] In some aspects, the techniques described herein relate to a method further including determining that an individual classification of the plurality of classifications is not assigned to any of the plurality of images, identifying an individual region of the plurality of regions associated with the individual classifications, and in response to determining that the individual classification is not assigned to any of the plurality of images, populating the individual region within the GUI with a three-dimensional (3D) graphical element representing the individual classification.
[0027] In some aspects, the techniques described herein relate to a method further including storing a database associating a plurality of classifications with respective 3D graphical elements representing the classifications, and retrieving the 3D graphical elements from the database by searching the database based on the respective classifications.
[0028] In some aspects, the techniques described herein relate to a method further including obtaining a list of amenities associated with each listing; detecting a physical space in each image of the plurality of images; selecting a respective classification from the plurality of classifications based on the list of amenities associated with each listing and the detected physical space; and associating the respective classification with the respective image.
[0029] In some aspects, techniques described herein relate to methods that further include determining that the list of amenities includes a first amenity and excludes a second amenity, selecting the first classification as an individual classification in response to determining that the list of amenities includes the first amenity and excludes the second amenity, and selecting the second classification as an individual classification in response to determining that the list of amenities excludes the first amenity and excludes the second amenity.
[0030] In some aspects, the techniques described herein relate to a system including one or more processors of a machine and a memory storing instructions that, when executed by the one or more processors, cause the machine to perform operations including receiving an input including a plurality of images associated with individual listings by a network site of a listing network platform; presenting an option within a GUI for arranging the plurality of images according to classifications associated with the plurality of images; in response to receiving an input selecting the option, animating a subset of the plurality of images within the GUI to be shuffled into one or more of the classifications; after the subset of the plurality of images has been animated for a specified threshold period, presenting a template within the GUI including a plurality of regions, each associated with a different classification; and populating a first region of the plurality of regions associated with a first classification with a first image corresponding to the first classification and populating a second region of the plurality of regions associated with a second classification with a second image corresponding to the second classification.
[0031] In some aspects, the techniques described herein relate to a machine-readable storage device embodying instructions that, when executed by the machine, cause the machine to perform operations including receiving, by a network site of a listing network platform, an input including a plurality of images associated with individual listings; presenting within a GUI an option for arranging the plurality of images according to classifications associated with the plurality of images; in response to receiving an input selecting the option, animating within the GUI a subset of the plurality of images to be shuffled into one or more of the classifications; after the subset of the plurality of images has been animated for a specified threshold period, presenting within the GUI a template including a plurality of regions, each associated with a different classification; and populating a first region of the plurality of regions associated with a first classification with a first image corresponding to the first classification and populating a second region of the plurality of regions associated with a second classification with a second image corresponding to the second classification.
[0032] 1, an example of a general client-server based network architecture 100 is shown. A networked system 102, in an exemplary form of a network-based listings service system, provides server-side functionality to one or more client devices 110 via a network 104 (e.g., the Internet or a wide area network (WAN)). In some implementations, a user (e.g., a user 106) interacts with the networked system 102 using the client device 110.
[0033] 1, for example, illustrates a web client 112 (e.g., a browser), client application(s) 114, and programmatic client 116 executing on a client device 110. The client device 110 includes the web client 112, the client application(s) 114, and / or the programmatic client 116, singly, together, or in any suitable combination. Although one client device 110 is illustrated in FIG. 1, in other implementations, the network architecture 100 includes multiple client devices.
[0034] In various implementations, the client device 110 may include a computing device including at least a display and a communication capability that provides access to the networked system 102 via the network 104. The client device 110 may include, but is not limited to, a remote device, a workstation, a computer, a general purpose computer, an Internet appliance, a handheld device, a wireless device, a portable device, a wearable computer, a cellular or mobile phone, a personal digital assistant (PDA), a smartphone, a tablet, an ultrabook, a netbook, a laptop, a desktop, a multiprocessor system, a microprocessor-based or programmable consumer electronics, a game console, a set-top box (STB), a networked personal computer (PC), a minicomputer, and the like. In one example, the client device 110 includes one or more of a touch screen, an accelerometer, a gyroscope, a biometric sensor, a camera, a microphone, a global positioning system (GPS) device, and the like.
[0035] The client device 110 communicates with the network 104 via a wired or wireless connection. For example, one or more portions of the network 104 may include an ad-hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a WAN, a wireless WAN (WWAN), a metropolitan area network (MAN), a portion of the Internet, a portion of the public switched telephone network (PSTN), a cellular network, a wireless network, a Wireless Fidelity (Wi-Fi® network, a Worldwide Interoperability for Microwave Access (WiMax) network, another type of network, or any suitable combination thereof.
[0036] In some examples, the client device 110 includes one or more of the following applications (also referred to as "apps"): a web browser, a book reader app (operable to read e-books), a media app (operable to present various forms of media including audio and video), a fitness app, a biometric monitoring app, a messaging app, an electronic mail (email) app, an e-commerce site app (also referred to as a "marketplace app"), and a booking application for temporary stays or experiences at hotels, motels, or residences managed by other end users (e.g., posting end users who own their homes and rent out entire homes or private rooms, referred to as "hosts").
[0037] In some implementations, the client application(s) 114 include various components operable to present information to a user and communicate with the networked system 102. In some examples, if an e-commerce site application is included on the client device 110, the application is configured to provide at least some of the user interface and functionality locally, with the application configured to communicate with the networked system 102 as necessary for data or processing capabilities not available locally (e.g., access to a database of items available for sale, authentication of a user, verification of a payment method). Conversely, if an e-commerce site application is not included on the client device 110, the client device 110 can use its web browser to access an e-commerce site (or a variation thereof) hosted on the networked system 102.
[0038] Web clients 112 access the various systems of networked system 102 through a web interface supported by a web server 122. Similarly, programmatic clients 116 and client application(s) 114 access the various services and functions provided by networked system 102 through a programmatic interface provided by an application program interface (API) server 120.
[0039] A user (e.g., user 106) may include a person, a machine, or other means of interacting with client device 110. In some examples, user 106 is not part of network architecture 100 but interacts with network architecture 100 through client device 110 or another means. For example, user 106 provides input (e.g., touch screen input or alphanumeric input) to client device 110, which is communicated to networked system 102 over network 104 by a second user interaction channel. In this example, networked system 102 communicates information to be presented to user 106 to client device 110 over network 104 in response to receiving input from user 106. In this manner, user 106 can interact with networked system 102 using client device 110. As another example, user 106 provides input (e.g., voice input) to client device 110, which is communicated to networked system 102 over network 104 in the form of audio packets or audio data.
[0040] The API server 120 and the web server 122 are coupled to one or more application server(s) 140, providing a programmatic interface and a web interface, respectively, to the one or more application server(s) 140. The application server(s) 140 may host a listings network platform 142 and a listings search system 150, each of which includes one or more modules or applications, each of which may be embodied as hardware, software, firmware, or any combination thereof. The application server(s) 140 are, in turn, shown coupled to one or more database server(s) 124 that facilitate access to one or more information storage repositories or database(s) 126. In one example, the database(s) 126 are storage devices that store information (e.g., inventory, image data, catalog data, 3D and / or 2D representations of room / space classifications) that are posted to the listings network platform 142. The database(s) 126 also store digital product information, according to some examples.
[0041] Additionally, the social network platform 131 is shown as executing on the third party server(s) 130. Further, the social network platform 131 can programmatically access the networked system 102 through programmatic interfaces provided by the API server 120. The social network platform 131 may include a social network website, a messaging platform, and one or more APIs. In some examples, the electronic messages described below are messages (e.g., social media chat messages, posts, pings ("hello" notifications), etc.) sent to a given user via the messaging system of the social network platform 131.
[0042] The listings network platform 142 provides a number of publishing functions and listing services to users accessing the networked system 102. Although the listings network platform 142 is shown in FIG. 1 as forming part of the networked system 102, it will be appreciated that in alternative examples, the listings network platform 142 is separate from the networked system 102 and it may form part of a separate and distinct web service.
[0043] 1 employs a client-server architecture, the disclosed techniques are not limited to such an architecture and can equally be implemented in, for example, a distributed or peer-to-peer architecture system. Various systems of the application server(s) 140 (e.g., listings network platform 142 and listings search system 150) may also be implemented as stand-alone software programs that do not necessarily have networking capabilities.
[0044] The listing network platform 142 may be hosted on dedicated or shared server machines that are communicatively coupled to enable communication between the server machines. The components themselves are communicatively coupled (e.g., via appropriate interfaces) to each other and to various data sources to enable information to be passed between applications or to enable applications to share and access common data. Additionally, the components access one or more database(s) 126 via a database server 124. The listing network platform 142 provides several publishing and listing mechanisms whereby sellers (also referred to as "first users", posting users, hosts) can list (or publish information about) goods or services for sale or barter, and buyers (also referred to as "second users", search users, guests) can express an interest in or indicate a desire to purchase or barter such goods or services, and can complete transactions (e.g., trades) related to the goods or services.
[0045] 2 illustrates an exemplary function engine of listings search system 150, according to some examples. As shown, listings search system 150 includes an image input component 220, a classifier component 240, and a listings tour GUI component 260. In some examples, a user may wish to perform some action on listings network platform 142. For example, a user, such as an accommodation host, may wish to generate a listing for a reservation, modify one or more listings for a reservation maintained by listings network platform 142, and / or perform any other available function on listings network platform 142.
[0046] In some examples, the GUI allows a user to access reservation listings associated with the user, for example, if the user is the host (e.g., owner, manager) of a listing. The GUI receives input from the user selecting an individual listing. In response, the GUI presents various information about the listing, including one or more images, amenities, availability dates, descriptive information, location information, and / or price or reservation details. The GUI can receive input from the user requesting to modify, edit, or adjust any of the information about the presented listing. Once the information has been modified and optionally approved by a moderator, the listing is updated and made available to other users of the network site looking for accommodations.
[0047] In some examples, the GUI allows users to improve the appearance and functionality of a listing, for example, by automatically assigning images to their respective classifications. For example, the network site assigns a classification to each image in the listing and then classifies the images by space type based on the classification. That is, a first machine learning model generates and associates classifications for the images. A second machine learning model generates embeddings for the images to cluster similar images and associate such similar images with the same classification (e.g., room type).
[0048] The network site generates a tour GUI that includes images organized according to space type (also called classification or category). The tour GUI presents one or more images for one or more of a plurality of classifications. This allows the host and one or more end users (e.g., guests) to quickly and easily determine what is shown or represented by the different images. Input from the GUI allows selection of individual classifications and causes a subset of images associated with each classification to be presented within the new GUI.
[0049] To assign a classification to each image, the image input component 220 processes the images as they are input to generate and assign a classification or category to the image. That is, a collection of images or a plurality of images may be initially received from a user and associated with a particular listing. The plurality of images are provided to a classifier component 240 to generate a classification for the images. To reduce the processing time and resources required to perform such classification on the fly in real time, this operation may be performed prior to receiving input from a host for generating a photo tour.
[0050] In one example, the classifier component 240 generates a prediction score indicating how likely an image is to be associated with a particular room type. A room type is selected for an image based on a threshold prediction score. For example, if an image has a prediction score of 0.8 to be associated with a bedroom and a prediction score of 0.4 to be associated with a living room, a bedroom room type would be selected by the system for the image. The classifier component 240 implements one or more machine learning models (e.g., artificial neural networks and / or convolutional neural networks) trained to analyze features of a collection of images to predict a classification of each of the images. The classifier component 240 can include a second machine learning model for clustering images based on embeddings, as described below.
[0051] Specifically, the classifier component 240 may be trained by processing training data including different image groups and respective ground truth classifications of the images. The classifier component 240 may access each individual image group and extract features from the image group. The classifier component 240 predicts or estimates a classification for each of the images in the image group based on the extracted features. Each image may be assigned to only one classification. The classifier component 240 may then obtain a ground truth classification associated with the image group. The classifier component 240 may compare the classification associated with each image in the estimated image group to the corresponding ground truth classification. The classifier component 240 may calculate a deviation based on the comparison and may then update parameters of the classifier component 240 based on the deviation. The classifier component 240 may repeat this training process for additional image groups until a stopping criterion is reached.
[0052] In some examples, the image input component 220 receives an input to add a new or additional image in association with a listing where a collection of previous images have already been classified. In response, the image input component 220 provides the new image to the classifier component 240 along with instructions to classify the new image. The classifier component 240 retrieves the previously classified images along with their classifications. The classifier component 240 then reanalyzes all images, including the new image as well as the images already classified. The classifier component 240 can generate or estimate a new classification for the new image and update one or more classifications associated with the previously classified images. In some cases, only the new image is classified independently from other images, but a second machine learning model is applied collectively to all images and their classifications to group or cluster similar images based on their generated embeddings.
[0053] For example, the classifier component 240 determines that a first image in a collection of previously classified images was associated with a first classification (e.g., a first bedroom classification). The classifier component 240 processes a new image together with the first image to derive new information about the first image and the new image. In particular, the new image may represent a bedroom that includes similar features as the first image. In such a case, the classifier component 240 determines that the first image and the new image belong to the same classification. In addition, the classifier component 240 may determine that the new image corresponds to a second classification (e.g., a second bedroom classification). In such a case, the classifier component 240 updates the classification associated with the first image from the first classification to the second classification in response to processing the new image (not yet classified) together with the previously classified images.
[0054] In some examples, the classifier component 240 generates a predicted score for each image classification that indicates the likelihood that the image belongs to a particular classification or room. The classifier component 240 takes all images and their respective scores and determines, for each classification, whether the score of the respective image exceeds a threshold. The classifier component 240 assigns a classification to images that have a score of a classification that exceeds the threshold. For example, if an image has a first score for a first classification and a second score for a second classification, the classifier component 240 compares the first score to a first threshold and the second score to a second threshold. If the first score does not exceed the threshold but the second score does exceed the threshold, the classifier component 240 assigns the image to the second classification instead of the first classification.
[0055] After obtaining or generating a classification and / or score for each of the images, the classifier component 240 processes the classified images with an additional machine learning model (e.g., an additional artificial neural network). The additional machine learning model generates an embedding for each image (before and / or after the image is classified by the first machine learning model). The embeddings are compared to each other for each of the images to cluster and / or group images with similar embeddings. For example, the classifier component 240 obtains the embedding generated for the first image and the embedding generated for the second image and each additional image. The classifier component 240 calculates a similarity score (e.g., by calculating a cosine similarity) between the first image embedding and the embedding of each additional image, including the second image embedding. The classifier component 240 determines that the embedding of the first image is similar to the embedding of the second image by more than a specified threshold. In such a case, the classifier component 240 groups the first image with the second image to generate a first cluster of images. Similar operations are performed to cluster all of the other images.
[0056] The classifier component 240 provides the classified images to the listings tour GUI component 260. The listings tour GUI component 260 presents a GUI that allows a user to request automatic classification of images associated with a listing. In some cases, the classifier component 240 performs the classification in response to an input received from the GUI requesting that the classification be performed. In some cases, the classifier component 240 performs the classification before the GUI presents an option for the user to request that the classification be performed. After the images are classified, the classification of the images is presented to the end user on the listings network platform 142 in a GUI or arrangement. For example, as described below, a GUI can be presented in which different regions of the GUI correspond to different classifications and a single representative image is presented in each region.
[0057] FIG. 3 illustrates a listings network site user interface 300 (e.g., a mobile application user interface, a web browser user interface) generated by the listings network platform 142 and the listings search system 150, according to some examples. The user interface 300 can be presented by a program client 116 implemented on a client device 110. As shown, the user interface 300 includes a search field 310, a filter menu element 315 (e.g., location type, amenities), and a search button 320. A user enters a listing query into the search field 310, such as searching for temporary residences in San Francisco, and enters a category restriction from the filter menu element 315 for "Entire Place" (e.g., the user is looking to rent the entire residence on the dates in question, rather than renting a single room in another person's residence). The user can directly customize the query using the terms entered into the search field 310 or the filters listed via the selection of the filter menu element 315. Additionally, the user can select dates using the date drop-down element 317 to select a specific date range for the temporary stay. For example, a user may select the date drop-down element 317 and a pop-up calendar (not shown in FIG. 3 ) to specify that a stay in San Francisco is specifically from July 16, 2021 to July 18, 2021. In some cases, a user may use the filter menu element 315 to provide one or more attributes of a search query, such as specifying the ages of children and minimum / maximum duration of stay.
[0058] Submitting a query (e.g., via selection of search button 320 or automatically via selection of composite listings element 313 (split stay option) or date drop-down element 317) sends a communication from program client 116 to listings search system 150. Listings search system 150 generates output including a result display of listings (e.g., graphical objects) that match the query and sends this output to program client 116.
[0059] The results are then displayed in the listings results area 305 using respective graphical objects (also referred to as indicators of the listings). The user can then select the listing's graphical object or navigate to additional pages via page navigation elements 325. In some examples, the user interface 300 includes a set of combined listings 323 along with the individual listings displayed in the results area 305. The combined listing 323 may be located in a dedicated area in the display, above an individual listing, between two individual listings, and / or below an individual listing. In some examples, the combined listing 323 is provided in response to receiving an input selecting the combined listing element 313. In some examples, the combined listing 323 is presented automatically without receiving an input selecting the combined listing element 313.
[0060] In some examples, the composite listing 323 is displayed in a different slot or portion of the display compared to other individual listings based on the type of client device being used to access the system. For example, on a mobile device, on the first page, the composite listing 323 may be placed in slots 3, 6, 9, and 12, while on a desktop computer, the same composite listing 323 may be presented in slots 5, 9, 14, and 20 for better visual balance. As referred to herein, the term "slot" refers to an area of the display where a category is presented. In some cases, the composite listing 323 is excluded from presentation for last minute stays, such as when the travel date is within 48 hours of check-in or trip start. In some examples, the composite listing 323 includes individual listings for destinations or stays that are at least a two hour drive apart, but not more than a ten hour drive apart. In some examples, the composite listing 323 excludes repeated pairs of the same individual listings. In some examples, the composite listing 323 relates to pairs of individual listings from different neighborhoods and locations. In such cases, the neighbors and listings may be repeated across pairs of composite listings 323 .
[0061] To take a specific example, a listing request may be received for a one-month stay on the island of Kauai, specifying a beachfront home. In this example, the user identifies only three individual listings that meet these search parameters, but qualifies 10 additional stays as composite listings 323. Composite listings 323 may represent an opportunity to experience different sides of the island throughout the trip, and may include options such as splitting time between two cities (e.g., Koloa and Hanalei), splitting time between two other cities (e.g., Koloa and Lihue), splitting time between two listings that are one mile apart in Koloa, splitting time in related cities (e.g., Poipu and Princeville) and / or other related cities (e.g., Lihue and Princeville), etc. The result of selecting the composite listing element 313, or the user's automatic identification of a composite listing that does not have a composite listing element 313, may provide 40% more unique inventory for the user to choose from that would be excluded if only individual listings were presented.
[0062] In some examples, the presentation of the composite listing 323 may vary based on the type of category selected. For example, if the national park category is selected, the composite listing 323 may include points of interest for two different national parks that are at least two hours drive away and at most ten hours apart. In some cases, if the national parks are very far apart, they may still be paired even if their distance exceeds these thresholds. The graphical representation of the composite listing 323 may visually identify two national park listings that form part of the same composite listing without specifying the region / city name. As another example, if the surfing category is selected, two different surfing destinations that are at least two hours drive away and at most ten hours apart may be selected. The graphical representation of the composite listing 323 may visually identify two surfing destinations that form part of the same composite listing using the region / city name.
[0063] The listings tour GUI component 260 can instruct the user interface 300 to present images of a particular listing selected from the results area 305. The images can be presented in groups. For example, a first image or a first collection of images associated with a first classification can be presented in a first area of the GUI dedicated to presenting information / images for the first classification. A second image or a second collection of images associated with a second classification can be presented in a second area of the GUI dedicated to presenting information / images for the second classification. Different areas with different collections of images can be presented simultaneously within the same GUI.
[0064] 2, the listings tour GUI component 260 allows a host to automatically or semi-automatically generate or define the taxonomy that is provided to an end user in a listings tour. For example, the listings tour GUI component 260 presents a GUI, such as the exemplary GUI 400 shown in FIG. 4. Specifically, the GUI 400 is accessed in response to receiving input from a user requesting to edit or modify a listing for booking. Among the multiple fields of the listing that can be modified, the GUI 400 includes a photo editor field (or area) to allow the host to modify the placement of images associated with the listing and / or add new images.
[0065] The GUI 400 includes a graphical element 410 (e.g., a card) within a photo editor field. The graphical element 410 includes an information panel that describes the host's ability to instantly and automatically generate a classification of previously uploaded images. The listings tour GUI component 260 accesses a list of previously uploaded images. The listings tour GUI component 260 identifies an order associated with each of the uploaded images. The order indicates the order in which the images are presented in the GUI to an end user searching or accessing a listing. The listings tour GUI component 260 can select a subset of images that are at the top of the order, such as the first three images in a sequence or order of images. The listings tour GUI component 260 can generate thumbnails of the first three images and present those thumbnails 412 as part of the graphical element 410.
[0066] In some examples, the listings tour GUI component 260 accesses a classification associated with a previously uploaded image, as described above. In some cases, the listings tour GUI component 260 accesses a group of similar images generated by a second machine learning model based on the embeddings and / or directly accesses the classification generated by the first machine learning model. The listings tour GUI component 260 selects a first image associated with the first classification (group or cluster), a second image associated with the second classification (group or cluster), a third image associated with a third classification (group or cluster), and / or any number of additional images up to a specified threshold. The listings tour GUI component 260 generates thumbnails of the first, second, and third images and presents those thumbnails 412 as part of the graphical element 410.
[0067] In some examples, the listings tour GUI component 260 presents an option 414 for triggering the generation of a photo tour based on classifications associated with previously uploaded images. In response to receiving input selecting option 414, the listings tour GUI component 260 presents a sequence of user interfaces, such as, for example, the exemplary user interfaces 500, 501, and 502 shown in FIG. 5. This sequence of user interfaces 500, 501, and 502 indicates to the user the progress in classifying previously uploaded images into respective classifications or categories for generating a photo tour.
[0068] For example, a first user interface 500 may be presented that includes a graphical card 510. The graphical card 510 includes the same or different collection of images 512 presented in the graphical element 410 as thumbnails 412. The graphical card 510 includes a message indicating that the previously uploaded images are sorted into their respective classifications (or rooms or spaces). In some examples, the listing tour GUI component 260 retrieves a photo tour template. The photo tour template includes dedicated regions organized according to specified criteria. Each region is associated with and represents a different classification, room, or space. Each region includes an identifier of the corresponding classification. Before the classification of the images is finished (or before the visual animation / simulation of the classification is completed, where the images have already been classified and clustered by the first and second machine learning models before the input is received to generate the photo tour), the photo tour template includes a placeholder (e.g., a gray box) in each of the different regions. This photo tour template with placeholders may be presented on a background 520 along with the graphical card 510. In some cases, as each image is classified, a corresponding area of the photo tour template is populated with the classified image.
[0069] In some cases, the same or different collections of images 512 are animated over a threshold period (e.g., 5 seconds or 10 seconds) to represent to the user the progress in classifying the images and associating the images with regions of the template. For example, as shown in the second user interface 501, a first animation 530 is presented in which the same or different collections of images 512 are randomly scattered in different directions. Then, the second user interface 501 is followed by a third user interface 502. In the sequence of user interfaces 502, the images that were randomly scattered in different directions are randomly arranged in a grid 540 (e.g., a 4×4 grid), with a single image presented in each quadrant of the grid. During a first time interval, the first and second images presented along a first dimension 542 (e.g., a first row) of the grid 540 are swapped such that the position of the first image takes the position of the second image and the position of the second image takes the position of the first image. This is represented by an animation showing the first image sliding towards the position of the second image, and then the position of the second image sliding towards the position of the first image.
[0070] During the second time interval, the second and third images presented along the second dimension 544 (e.g., the first column) of the grid 540 are swapped such that the position of the second image takes the position of the third image and the position of the third image takes the position of the second image. This is represented by an animation showing the second image sliding towards the position of the third image and then the position of the third image sliding towards the position of the second image. The process of swapping images alternately along the first and second dimensions 542 and 544 is repeated for a threshold period (e.g., 5 seconds). After the threshold period has elapsed, the listings tour GUI component 260 presents an option 550 for accessing a preview of the photo tour. The listings tour GUI component 260 receives an input selecting the option 550 and, in response, navigates the user to a sequence of user interfaces, such as the exemplary user interfaces 600 and 601 shown in FIG. 6.
[0071] Specifically, in response to receiving an input selecting option 550, listing tour GUI component 260 animates card 610 showing grid 540 to slide down the screen while presenting a celebration animation. The celebration animation can include many different types of graphical elements, such as confetti 620, balloons, and / or listing site logo 622 sliding down the screen. The photo tour template continues to be presented in the background, for example, in dark contrast to card 610, which is presented in lighter contrast. Once card 610 has completely disappeared from the screen, the contrast of the photo tour template increases while continuing to present the celebration animation. Specifically, as shown in user interface 601, photo tour template 630 currently being populated with images is presented in lighter contrast than the photo tour template shown in user interface 600. The sequence of user interface 601 also presents the celebration animation for a specified threshold period.
[0072] After the threshold period has elapsed, the listing tour GUI component 260 may present a sequence of user interfaces, such as user interfaces 700 and 701 shown in FIG. 7. The sequence of user interfaces 700 and 701 no longer includes the celebration animation. As shown in user interface 700, a plurality of regions associated with different classifications are presented. For example, a first region associated with a first classification (group or cluster) is presented in the upper left corner. Along with the first region, a description of the first classification (e.g., bedroom) and the amount of images associated with the first classification are presented. The first region includes a random or designated image of the set of images associated with the first classification. In response to receiving an input selecting the first region or the designated image, all of the corresponding images in the set of images associated with the first classification are presented.
[0073] A second region associated with the second classification is presented in the upper right corner next to the first region. Along with the second region, a description of the second classification (e.g., shared living room) and the amount of images associated with the second classification are presented. The second region includes a random or designated image of the set of images associated with the second classification. In response to receiving an input selecting the second region or the designated image, all of the corresponding images in the set of images associated with the second classification are presented.
[0074] In some cases, a prompt 710 is presented that instructs the user or host to review the classification by browsing the populated photo tour template. The listings tour GUI component 260 can receive input to navigate the photo tour template to show each of the regions and their corresponding assigned images and descriptions. The listings tour GUI component 260 can receive input to move an image from one region to another and change the classification associated with the image.
[0075] In some examples, the listings tour GUI component 260 determines that the photo tour template includes individual classifications 720 that are not associated with any of the images. That is, the listings tour GUI component 260 compares the classifications associated with all of the images and determines that none of the classifications match the individual classifications 720. In this case, the listings tour GUI component 260 includes individual regions corresponding to the individual classifications 720 for display with other regions in the photo tour template. The listings tour GUI component 260 can search a database of 3D or 2D graphical elements to find or identify 3D or 2D graphical elements corresponding to the individual classifications 720. The database can associate each 3D or 2D graphical element with a corresponding classification. Each 2D or 3D graphical element includes visual attributes that represent the corresponding classification. The listings tour GUI component 260 retrieves the 3D or 2D graphical element from the database associated with the individual classifications 720 and presents the retrieved 3D or 2D graphical element 730 as part of the individual region of the photo tour template. The listing tour GUI component 260 can receive an input selecting a 3D or 2D graphical element 730. In response, the listing tour GUI component 260 allows the user to submit or upload a new image or photo corresponding to the individual classification 720 to replace the display of the 3D or 2D graphical element 730 in the individual area. In this manner, the listing tour GUI component 260 can identify a particular key room that is empty in that there is no corresponding image or photo for that key room or classification. Based on this identification, the listing tour GUI component 260 can recommend to the user (host) to submit or upload or add additional images corresponding to the key room or classification. This helps to build a full loop between the completion of the photo tour and the publication of the photo tour for viewing by guest users.
[0076] The ordering of the areas in the photo tour template may be static and unchangeable by the user. In some cases, the user may change the ordering or organization of the areas of the photo tour template. In some cases, only users with advanced status or subscriptions may be permitted to change the ordering or organization of the areas of the photo tour template. In some cases, the ordering of the areas of the photo tour template may be: living area, full kitchen, kitchenette, dining area, bedroom, full bathroom, half bathroom, office, dedicated workspace, backyard, patio, balcony, front yard, deck, porch, courtyard, garden, terrace, rooftop, laundry area, garage, gym, exterior, pool, hot tub, theme room, children's playroom, bowling alley, movie theater, art studio, music studio, workshop, photo studio, darkroom, woodworking room, event space, library, game room, sunroom, and wine cellar. Some of these may be omitted in certain situations.
[0077] In some examples, after the images are classified and / or clustered, the classification can be further refined based on amenities associated with the listings. In such cases, the listings tour GUI component 260 obtains a list of amenities associated with each listing and detects physical spaces within each image of the multiple images. For example, the listings tour GUI component 260 applies one or more trained machine learning models to extract features from the individual images and estimate amenities or physical spaces characterized within the individual images. The listings tour GUI component 260 selects an individual classification from the multiple classifications based on the list of amenities associated with each listing and the detected physical spaces, and associates the individual classification with the individual image. The listings tour GUI component 260 can determine that the list of amenities includes the first amenity and excludes the second amenity. In response to determining that the list of amenities includes the first amenity and excludes the second amenity, the listings tour GUI component 260 selects the first classification as the individual classification. The listings tour GUI component 260 selects the second classification as an individual classification in response to determining that the list of amenities excludes the first amenity and excludes the second amenity.
[0078] For example, the listings tour GUI component 260 can assign images based on existing amenities specified in the listing. Specifically, the listings tour GUI component 260 detects outdoor spaces (patios, balconies, backyards, porches, etc.) in each image. If the listings tour GUI component 260 determines that the listing has a backyard amenity and no patio / balcony amenity, the listings tour GUI component 260 assigns the image to a backyard space (or classification). If the listings tour GUI component 260 determines that the listing has a patio / balcony amenity and no backyard amenity, the listings tour GUI component 260 assigns the image to a balcony classification if the listing corresponds to an apartment or condominium, and assigns the image to a patio classification if the listing does not correspond to an apartment or condominium. If the listings tour GUI component 260 determines that the listing has both backyard and patio / balcony amenities, the listings tour GUI component 260 does not assign the image to any space or classification. The listing tour GUI component 260 may still create backyard and patio rooms or classifications, but may prevent the assignment of images to those rooms or classifications. If the listing tour GUI component 260 determines that the listing excludes backyard and patio / balcony amenities, the listing tour GUI component 260 assigns images to an exterior space or classification.
[0079] As another example, if a pool is detected in the image, the listing tour GUI component 260 determines whether the listing includes a pool amenity and excludes a backyard amenity, in which case the listing tour GUI component 260 assigns the image to a pool space or classification if the listing corresponds to an apartment or condominium, and assigns the image to a backyard amenity if the listing does not correspond to an apartment or condominium.
[0080] In some examples, the listings tour GUI component 260 groups similar images, for example, by applying a second machine learning model to the classification generated by the first machine learning model and / or by processing amenities or other attributes associated with the listing. That is, if a listing specifies more than one bedroom / bathroom, the listings tour GUI component 260 can apply a second machine learning model to the images and their classification to generate clusters or groups of images. For example, the listings tour GUI component 260 can cluster images with a bedroom type classification with full bathroom and half bathroom type classifications of images. If a listing has only one bedroom, the listings tour GUI component 260 assigns images with a bedroom to the bedroom classification without applying a second machine learning model to generate clusters of images based on similarity. If a listing has only one bathroom, the listings tour GUI component 260 assigns images with a bathroom to the bathroom classification. If the listing includes multiple bedrooms / bathrooms, the listings tour GUI component 260 applies a second machine learning model to the image classification to assign or group images into bedroom numbers in order of largest to smallest group size based on the similarity scores corresponding to the embeddings. If the listings tour GUI component 260 determines that the listing does not include a bedroom, the listings tour GUI component 260 can create a sleeping area object or classification and associate any images that have a bedroom with the sleeping area object or classification.
[0081] For example, the listings tour GUI component 260 can conditionally apply a second machine learning model to classify images based on whether the listing includes multiple amenities of the same type (e.g., whether the listing includes multiple bedrooms). In such a case, the listings tour GUI component 260 can group a first cluster of similar images (having similar embeddings) with a first bedroom determined by the second machine learning, and can group a second cluster of similar images (having similar embeddings) with a second bedroom determined by the second machine learning model embeddings.
[0082] 8 illustrates a flow diagram of various processes and methods 800 for generating listings on a listing network site, according to some examples. These processes and methods 800 may be performed in any order or sequence by any of the components described above or below, such as the listing search system 150.
[0083] At operation 805, the listings search system 150 receives input including a plurality of images associated with individual listings by a network site of a listings network platform, as described above.
[0084] At operation 810, the listings search system 150 presents an option in the GUI to arrange the images according to the classifications associated with the images, as described above.
[0085] At operation 815, the listing search system 150 animates the subset of the plurality of images in the GUI to be shuffled into one or more of the classifications in response to receiving input selecting an option, as described above.
[0086] In operation 820, after the subset of the multiple images has been animated for a specified threshold period, the listing search system 150 presents a template in the GUI that includes multiple regions, each associated with a different classification, as described above.
[0087] In operation 825, the listing search system 150 populates a first region of the plurality of regions associated with the first classification with a first image corresponding to the first classification, and populates a second region of the plurality of regions associated with the second classification with a second image corresponding to the second classification, as described above. Working Example
[0088] [Example 1] A method comprising: receiving, by a network site of a listing network platform, an input including a plurality of images associated with individual listings; presenting within a graphical user interface (GUI) an option for arranging the plurality of images according to classifications associated with the plurality of images; in response to receiving input selecting an option, animating within the GUI a subset of the plurality of images to be shuffled into one or more of the classifications; after the subset of the plurality of images has been animated for a specified threshold period, presenting within the GUI a template including a plurality of regions, each of which is associated with a different classification; and populating a first region of the plurality of regions associated with a first classification with a first image corresponding to the first classification and populating a second region of the plurality of regions associated with a second classification with a second image corresponding to the second classification.
[0089] [Example 2] The method described in Example 1, further comprising a step of analyzing the plurality of images with a machine learning model to determine a respective classification associated with each of the plurality of images.
[0090] [Example 3] The method described in Example 2, wherein the machine learning model analyzes a plurality of images in response to receiving an input selecting an option.
[0091] [Example 4] The method described in Example 2 or 3, wherein the machine learning model analyzes the plurality of images before presenting an option for prearranging the plurality of images in the GUI when each respective image of the plurality of images is received by input.
[0092] [Example 5] The method described in Example 4, further comprising the steps of receiving additional input including additional images associated with the listing, and in response to receiving the additional input, reanalyzing the plurality of images previously classified by the machine learning model together with the additional images to generate or update a classification for each of the plurality of images including the additional images.
[0093] [Example 6] The method according to any one of Examples 1 to 5, wherein the first classification includes a first room type and the second classification includes a second room type.
[0094] [Example 7] A method as described in any one of Examples 1 to 6, further comprising a step of allowing a user to modify classifications associated with a plurality of images by interacting with templates presented in the GUI.
[0095] [Example 8] The method of any one of Examples 1 to 7, wherein the order in which the multiple areas are presented in the GUI is predefined, and the order includes: living area, full kitchen, kitchenette, dining area, bedroom, full bathroom, half bathroom, office, dedicated workspace, backyard, patio, balcony, front yard, deck, porch, courtyard, garden, terrace, rooftop, laundry area, garage, gym, exterior, pool, hot tub, theme room, children's playroom, bowling alley, movie theater, art studio, music studio, workshop, photography studio, darkroom, woodworking room, event space, library, game room, sunroom, and wine cellar.
[0096] [Example 9] A method described in any one of Examples 1 to 8, further comprising the steps of selecting one or more images from the plurality of images based on one or more criteria, and presenting the selected one or more images within a graphical element including options for arranging the plurality of images according to classification.
[0097] [Example 10] The method described in Example 9, wherein the one or more criteria include at least one of the order in which the images are presented in the listing, or the respective classification of the images.
[0098] [Example 11] A method as described in Example 9 or 10, wherein the subset of the plurality of images includes one or more selected images, and the method further includes a step of presenting a graphical element on the template presenting an animation of the subset of the plurality of images, and each of the plurality of regions of the template includes a placeholder for a respective image.
[0099] [Example 12] The method described in Example 11, wherein the placeholder includes a gray box.
[0100] [Example 13] A method described in any one of Examples 1 to 12, further comprising a step of presenting an animation including a celebratory graphic on a template including a first area having a first image and a second area having a second image.
[0101] [Example 14] The method of example 13, wherein the celebration graphic includes at least one of confetti, balloons, or a logo associated with the network site.
[0102] [Example 15] A method as described in any one of Examples 1 to 14, further comprising the steps of determining that an individual classification of the plurality of classifications is not assigned to any of the plurality of images, identifying individual regions of the plurality of regions associated with the individual classifications, and in response to determining that the individual classification is not assigned to any of the plurality of images, populating the individual regions within the GUI with three-dimensional graphical elements representing the individual classifications.
[0103] [Example 16] The method described in Example 15, further comprising the steps of storing a database associating a plurality of classifications with respective three-dimensional graphical elements representing the classifications, and retrieving the three-dimensional graphical elements from the database by searching the database based on the individual classifications.
[0104] [Example 17] A method described in any one of Examples 1 to 16, further comprising the steps of obtaining a list of amenities associated with each listing, detecting a physical space in each image of the plurality of images, selecting an individual classification from the plurality of classifications based on the list of amenities associated with each listing and the detected physical space, and associating the individual classification with each image.
[0105] [Example 18] The method described in Example 17, further comprising the steps of determining that the list of amenities includes the first amenity and excludes the second amenity, selecting the first classification as an individual classification in response to determining that the list of amenities includes the first amenity and excludes the second amenity, and selecting the second classification as an individual classification in response to determining that the list of amenities excludes the first amenity and excludes the second amenity.
[0106] [Example 19] A system comprising one or more processors of a machine and a memory for storing instructions, which, when executed by the one or more processors, cause the machine to perform operations including: receiving an input including a plurality of images associated with individual listings by a network site of a listing network platform; presenting an option within a graphical user interface (GUI) for arranging the plurality of images according to classifications associated with the plurality of images; in response to receiving an input selecting an option, animating a subset of the plurality of images within the GUI to be shuffled into one or more of the classifications; after the subset of the plurality of images has been animated for a specified threshold period, presenting a template within the GUI including a plurality of regions, each of which is associated with a different classification; and populating a first region of the plurality of regions associated with a first classification with a first image corresponding to the first classification and populating a second region of the plurality of regions associated with a second classification with a second image corresponding to the second classification.
[0107] [Example 20] A machine-readable storage device having instructions which, when executed by the machine, cause the machine to perform operations including receiving input by a network site of a listing network platform including a plurality of images associated with individual listings; presenting an option within a graphical user interface (GUI) for arranging the plurality of images according to classifications associated with the plurality of images; in response to receiving input selecting an option, animating a subset of the plurality of images within the GUI to be shuffled into one or more of the classifications; after the subset of the plurality of images has been animated for a specified threshold period, presenting a template within the GUI including a plurality of regions, each associated with a different classification; and populating a first region of the plurality of regions associated with a first classification with a first image corresponding to the first classification and populating a second region of the plurality of regions associated with a second classification with a second image corresponding to the second classification.
[0108] FIG. 9 is a block diagram 900 illustrating an architecture of software 902 that may be installed on any one or more of the devices described above. It will be appreciated that FIG. 9 is merely a non-limiting example of a software architecture and that many other architectures may be implemented to facilitate the functionality described herein. In various embodiments, the software 902 is implemented by hardware, such as machine 1000 of FIG. 10, which includes a processor 1010, memory 1030, and input / output (I / O) components 1050. In this exemplary architecture, the software 902 may be conceptualized as a stack of layers, where each layer may provide a particular function. For example, the software 902 includes layers such as an operating system 904, libraries 906, frameworks 908, and applications 910. In operation, the applications 910 invoke API calls 912 through the software stack and receive messages 914 in response to the API calls 912, consistent with some embodiments.
[0109] In various implementations, the operating system 904 manages hardware resources and provides common services. The operating system 904 includes, for example, a kernel 920, services 922, and drivers 924. The kernel 920 serves as an abstraction layer between the hardware and other software layers, consistent with some embodiments. For example, the kernel 920 provides memory management, processor management (e.g., scheduling), component management, networking, and security configuration, among other functions. The services 922 can provide other common services for the other software layers. The drivers 924 are responsible for controlling or interfacing with the underlying hardware, according to some embodiments. For example, the drivers 924 can include a display driver, a camera driver, a BLUETOOTH® or BLUETOOTH® Low Energy driver, a flash memory driver, a serial communication driver (e.g., a Universal Serial Bus (USB) driver), a WI-FI® driver, an audio driver, a power management driver, and the like.
[0110] In some embodiments, libraries 906 provide a low-level common infrastructure utilized by applications 910. Libraries 906 can include system libraries 930 (e.g., C standard libraries) that can provide functions such as memory allocation functions, string manipulation functions, mathematical functions, etc. Additionally, libraries 906 can include API libraries 932 such as a media library (e.g., a library for supporting the presentation and manipulation of various media formats such as Moving Picture Experts Group-4 (MPEG4), Advanced Video Coding (H.264 or AVC), Moving Picture Experts Group Layer-3 (MP3), Advanced Audio Coding (AAC), Adaptive Multi-Rate AMR) audio codec, Joint Photographic Experts Group (JPEG or JPG), or Portable Network Graphics (PNG), a graphics library (e.g., an OpenGL framework used for rendering in two dimensions (2D) and 3D in graphical content on a display), a database library (e.g., SQLite for providing various relational database functions), a web library (e.g., WebKit for providing web browsing functions), etc. The libraries 906 can also include a wide variety of other libraries 934 to provide many other APIs to the applications 910 .
[0111] Framework 908, according to some embodiments, provides a high-level, common infrastructure that can be utilized by applications 910. For example, framework 908 provides various graphic user interface (GUI) functionality, high-level resource management, high-level location services, etc. Framework 908 can provide a wide range of other APIs that can be utilized by applications 910, some of which may be specific to a particular operating system or platform.
[0112] In an exemplary embodiment, the applications 910 include a wide variety of other applications, such as a home application 950, a contacts application 952, a browser application 954, a book reader application 956, a location application 958, a media application 960, a messaging application 962, a games application 964, and a third party application 966. According to some embodiments, the applications 910 are programs that perform functions defined in the program. Various programming languages, such as object-oriented programming languages (e.g., Objective-C, Java, or C++) or procedural programming languages (e.g., C or assembly language), may be employed to create one or more of the applications 910, which may be structured in various ways. In a particular example, the third party application 966 (e.g., an application developed using the ANDROID or IOS Software Development Kit (SDK) by an entity other than the particular platform vendor) may be mobile software that runs on a mobile operating system, such as IOS, ANDROID, WINDOWS Phone, or another mobile operating system. In this example, the third party application 966 can invoke API calls 912 provided by the operating system 904 to facilitate the functionality described herein.
[0113] FIG. 10 shows a schematic diagram of a machine 1000 in the form of a computer system on which a set of instructions may be executed to cause the machine to perform any one or more of the methods described herein, according to an exemplary embodiment. Specifically, FIG. 10 shows a schematic diagram of a machine 1000 in the exemplary form of a computer system on which instructions 1016 (e.g., software, programs, applications, applets, apps, or other executable code) may be executed to cause the machine 1000 to perform any one or more of the methods described herein. The instructions 1016 transform the general, unprogrammed machine 1000 into a specific machine 1000 programmed to perform the functions described and illustrated in the manner described. In alternative embodiments, the machine 1000 may operate as a standalone device or be coupled (e.g., networked) to other machines. In a networked deployment, the machine 1000 may operate in the capacity of a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine 1000 may include, without limitation, a server computer, a client computer, a PC, a tablet computer, a laptop computer, a netbook, a STB, a PDA, an entertainment media system, a cell phone, a smartphone, a mobile device, a wearable device (e.g., a smart watch), a smart home device (e.g., a smart appliance), other smart devices, a web appliance, a network router, a network switch, a network bridge, or any machine capable of executing instructions 1016, sequentially or otherwise, that specify actions to be taken by the machine 1000. Additionally, although only a single machine 1000 is shown, the term "machine" should also be construed to include a collection of machines 1000 that individually or together execute instructions 1016 to perform any one or more of the methods described herein.
[0114] The machine 1000 may include a processor 1010, a memory 1030, and I / O components 1050, which may be configured to communicate with each other via a bus 1002 or the like. In an exemplary embodiment, the processor 1010 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an ASIC, a radio frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, a processor 1012 and a processor 1014, which may execute instructions 1016. The term "processor" is intended to include a multi-core processor, which may include two or more independent processors (sometimes referred to as "cores") that may execute instructions simultaneously. Although FIG. 10 illustrates multiple processors 1010, the machine 1000 may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiple cores, or any combination thereof.
[0115] The memory 1030 may include a main memory 1032, a static memory 1034, and a storage unit 1036, all of which are accessible to the processor 1010, for example, via the bus 1002. The main memory 1032, the static memory 1034, and the storage unit 1036 store instructions 1016 that embody any one or more of the methods or functions described herein. The instructions 1016 may also reside, completely or partially, within the main memory 1032, within the static memory 1034, within the storage unit 1036, within at least one of the processors 1010 (e.g., within a cache memory of the processor), or any suitable combination thereof, during its execution by the machine 1000.
[0116] The I / O components 1050 may include a wide variety of components for receiving input, providing output, generating output, transmitting information, exchanging information, capturing measurements, and the like. The particular I / O components 1050 included in a particular machine will depend on the type of machine. For example, a portable machine such as a cell phone will likely include a touch input device or other such input mechanism, while a headless server machine will likely not include such a touch input device. It will be appreciated that the I / O components 1050 may include many other components not shown in FIG. 10. The I / O components 1050 are grouped according to function merely to simplify the following description, and the grouping is in no way limiting. In various exemplary embodiments, the I / O components 1050 may include output components 1052 and input components 1054. Output components 1052 may include visual components (e.g., a display such as a plasma display panel (PDP), a light emitting diode (LED) display, a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)), audio-acoustic components (e.g., speakers), haptic components (e.g., vibration motors, resistive mechanisms), other signal generators, etc. Input components 1054 may include alphanumeric input components (e.g., a keyboard, a touch screen configured to receive alphanumeric input, a photo-optical keyboard, or other alphanumeric input component), point-based input components (e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, or another pointing instrument), haptic input components (e.g., a physical button, a touch screen that provides the location and / or force of a touch or touch gesture, or other haptic input component), audio input components (e.g., a microphone), etc.
[0117] In further exemplary embodiments, the I / O components 1050 may include biometric components 1056, motion components 1058, environmental components 1060, or positional components 1062, among other rich components. For example, the biometric components 1056 may include components for detecting expressions (e.g., hand expressions, facial expressions, vocal expressions, body gestures, or eye tracking), measuring biosignals (e.g., blood pressure, heart rate, body temperature, sweat, or brain waves), identifying people (e.g., voice identification, retina identification, face identification, fingerprint identification, or brainwave-based identification), etc. The motion components 1058 may include acceleration sensor components (e.g., accelerometers), gravity sensor components, rotational sensor components (e.g., gyroscopes), etc. The environmental components 1060 may include, for example, an illuminance sensor component (e.g., a light meter), a temperature sensor component (e.g., one or more thermometers to detect ambient temperature), a humidity sensor component, a pressure sensor component (e.g., a barometer), an acoustic sensor component (e.g., one or more microphones to detect background noise), a proximity sensor component (e.g., an infrared sensor to detect nearby objects), a gas sensor (e.g., a gas detection sensor to detect concentrations of harmful gases for safety or to measure pollutants in the air), or other components that may provide indications, measurements, or signals corresponding to the surrounding physical environment. The position components 1062 may include a location sensor component (e.g., a GPS receiver component), an altitude sensor component (e.g., an altimeter or barometer to detect air pressure from which altitude may be derived), an orientation sensor component (e.g., a magnetometer), etc.
[0118] Communications may be implemented using a wide variety of technologies. The I / O component 1050 may include a communication component 1064 operable to couple the machine 1000 to the network 1080 or the device 1070 via coupling 1082 and coupling 1072, respectively. For example, the communication component 1064 may include a network interface component or another suitable device for interfacing with the network 1080. In further examples, the communication component 1064 includes a wired communication component, a wireless communication component, a cellular communication component, a near field communication (NFC) component, a Bluetooth component (e.g., Bluetooth low energy), a Wi-Fi component, and other communication components for providing communication via other modalities. The device 1070 may be either another machine or a wide variety of peripheral devices (e.g., a peripheral device coupled via USB).
[0119] Further, the communication component 1064 may detect the identifier or may include a component operable to detect the identifier. For example, the communication component 1064 may include a radio frequency identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (e.g., an optical sensor for detecting one-dimensional barcodes such as Universal Product Code (UPC) barcodes, multi-dimensional barcodes such as Quick Response (QR) codes, Aztec codes, Data Matrix, Dataglyph, MaxiCode, PDF417, UltraCode, UCC RSS-2D barcodes, and other optical codes), or an acoustic detection component (e.g., a microphone for identifying tagged audio signals). In addition, various information such as location via Internet Protocol (IP) geolocation, location via Wi-Fi signal triangulation, location via detection of NFC beacon signals that may indicate a particular location, may be derived via the communication component 1064.
[0120] The various memories (i.e., 1030, 1032, 1034, and / or memory of the processor(s) 1010) and / or storage unit 1036 may store one or more sets of instructions and data structures (e.g., software) that embody or are utilized by any one or more of the methods or functions described herein. These instructions (e.g., instructions 1016), when executed by the processor(s) 1010, cause various operations to implement the disclosed embodiments.
[0121] As used herein, the terms "machine storage medium", "device storage medium", and "computer storage medium" mean the same thing and may be used interchangeably in this disclosure. These terms refer to a single or multiple storage devices and / or media (e.g., centralized or distributed databases, and / or associated caches and servers) that store executable instructions and / or data. Thus, these terms should be interpreted to include, but are not limited to, solid-state memory, including memory internal or external to a processor, as well as optical and magnetic media. Specific examples of machine storage media, computer storage media, and / or device storage media include, by way of example, non-volatile memory including semiconductor memory devices, e.g., erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), FPGA, and flash memory devices, magnetic disks such as internal hard disks and removable disks, magneto-optical disks, and CD-ROM and DVD-ROM disks. The terms "mechanical storage medium," "computer storage medium," and "device storage medium" specifically exclude carrier waves, modulated data signals, and other such media, at least some of which are covered by the term "signal media" below.
[0122] In various exemplary embodiments, one or more portions of network 1080 may be an ad-hoc network, an intranet, an extranet, a VPN, a LAN, a WLAN, a WAN, a WWAN, a MAN, the Internet, a portion of the Internet, a portion of the PSTN, a plain old telephone service (POTS) network, a cellular telephone network, a wireless network, a Wi-Fi network, another type of network, or a combination of two or more such networks. For example, network 1080 or a portion of network 1080 may include a wireless or cellular network, and coupling 1082 may be a code division multiple access (CDMA) connection, a Global System for Mobile Communications (GSM) connection, or another type of cellular or wireless coupling. In this example, the coupling 1082 may implement any of various types of data transfer technologies, such as single carrier radio transmission technology (1xRTT), Evolution Data Optimized (EVDO) technology, General Packet Radio Service (GPRS) technology, Enhanced Data Rates for GSM Evolution (EDGE) technology, Third Generation Partnership Project (3GPP®) including 3G, Fourth Generation Wireless (4G) networks, Universal Mobile Telecommunications System (UMTS), High Speed Packet Access (HSPA), Worldwide Interoperability for Microwave Access (WiMAX), Long Term Evolution (LTE) standards, others defined by various standards setting organizations, other long-range protocols, or other data transfer technologies.
[0123] The instructions 1016 may be transmitted or received over the network 1080 using a transmission medium via a network interface device (e.g., a network interface component included in the communication component 1064) and utilizing any one of several well-known transfer protocols (e.g., Hypertext Transfer Protocol (HTTP)). Similarly, the instructions 1016 may be transmitted or received using a transmission medium via a coupling 1072 (e.g., a peer-to-peer coupling) to the device 1070. The terms "transmission medium" and "signal medium" mean the same thing and may be used interchangeably in this disclosure. The terms "transmission medium" and "signal medium" shall be interpreted to include any intangible medium capable of storing, encoding, or carrying the instructions 1016 for execution by the machine 1000, including digital or analog communication signals or other intangible media for facilitating communication of such software. Thus, the terms "transmission medium" and "signal medium" shall be interpreted to include any form of modulated data signal, carrier wave, or the like. The term "modulated data signal" means a signal having one or more of its characteristics set or changed in such a manner as to encode information within the signal.
[0124] The terms "machine-readable medium," "computer-readable medium," and "device-readable medium" mean the same thing and may be used interchangeably in this disclosure. These terms are defined to include both mechanical storage media and transmission media. Thus, these terms include both storage devices / media and carrier waves / modulated data signals.
[0125] Although some examples, such as those illustrated in the figures, include a particular sequence of operations, the sequence may be modified without departing from the scope of the present disclosure. For example, some of the illustrated operations may be performed in parallel or in a different order without substantially affecting the functionality described in the examples. In other examples, different components of an example device or system implementing an example method may perform functions substantially simultaneously or in a particular sequence.
[0126] The various features, steps, and processes described herein may be used independently of one another or may be combined in various ways. All possible combinations and subcombinations are intended to fall within the scope of the present disclosure. In addition, in some implementations, certain method or process blocks may be omitted.
Claims
1. 1. A method comprising: receiving, by a network site of the listing network platform, an input including a plurality of images associated with each listing; presenting an option within a graphical user interface (GUI) for arranging the plurality of images according to classifications associated with the plurality of images; in response to receiving an input selecting the option, animating within the GUI a subset of the plurality of images to be shuffled into one or more of the classifications; presenting a template in the GUI after the subset of the plurality of images has been animated for a specified threshold period, the template including a plurality of regions, each region associated with a different classification; populating a first region of the plurality of regions associated with a first classification with a first image corresponding to the first classification and populating a second region of the plurality of regions associated with a second classification with a second image corresponding to the second classification; The method includes:
2. Analyzing the plurality of images with a machine learning model to determine a respective classification associated with each of the plurality of images. The method of claim 1 further comprising:
3. The method of claim 2 , wherein the machine learning model analyzes the plurality of images in response to receiving the input selecting the option.
4. 3. The method of claim 2, wherein the machine learning model analyzes the plurality of images as each respective image of the plurality of images is received by the input before presenting the option for prearranging the plurality of images within the GUI.
5. receiving additional input including an additional image associated with the listing; and in response to receiving the additional input, re-analyzing the plurality of images previously classified by the machine learning model together with the additional image to generate or update a classification for each of the plurality of images including the additional image. The method of claim 4 further comprising:
6. the first classification includes a first room type; the second classification includes a second room type; The method of claim 1.
7. enabling a user to modify the classification associated with the plurality of images by interacting with the templates presented within the GUI. The method of claim 1 further comprising:
8. 2. The method of claim 1, wherein an order in which the plurality of areas are presented within the GUI is predefined, the order including: living area, full kitchen, kitchenette, dining area, bedroom, full bathroom, half bathroom, office, dedicated workspace, backyard, patio, balcony, front yard, deck, porch, courtyard, garden, terrace, rooftop, laundry area, garage, gym, exterior, pool, hot tub, theme room, children's playroom, bowling alley, movie theater, art studio, music studio, workshop, photography studio, darkroom, woodworking room, event space, library, game room, sunroom, and wine cellar.
9. selecting one or more images from the plurality of images based on one or more criteria; and presenting the selected one or more images in a graphical element including the option for arranging the plurality of images according to a classification. The method of claim 1 further comprising:
10. The method of claim 9 , wherein the one or more criteria include at least one of an order in which the images are presented in the listing, or a classification of each of the images.
11. 10. The method of claim 9, wherein the subset of the plurality of images includes the selected one or more images, the method further comprising presenting a graphical element on the template presenting the animation of the subset of the plurality of images, each of the plurality of regions of the template including a placeholder for a respective image.
12. The method of claim 11 , wherein the placeholder comprises a grey box.
13. presenting an animation including a congratulatory graphic on the template including the first region having the first image and the second region having the second image. The method of claim 1 further comprising:
14. The method of claim 13 , wherein the congratulatory graphic comprises at least one of confetti, balloons, or a logo associated with the network site.
15. determining that each of a plurality of classifications is not assigned to any of the plurality of images; identifying individual regions of the plurality of regions associated with the individual classifications; and in response to determining that the individual classifications have not been assigned to any of the plurality of images, populating the individual regions within the GUI with three-dimensional graphical elements representing the individual classifications. The method of claim 1 further comprising:
16. storing a database associating a plurality of categories with respective three-dimensional graphical elements representing said categories; and retrieving said three-dimensional graphical elements from said database by searching said database based on said respective categories. The method of claim 15 further comprising:
17. obtaining a list of amenities associated with each of the listings; and detecting physical spaces within each of the plurality of images; selecting a respective classification from a plurality of classifications based on the list of amenities associated with the respective listing and the detected physical space; and associating the respective classification with the respective image. The method of claim 1 further comprising:
18. determining that the list of amenities includes a first amenity and excludes a second amenity; in response to determining that the list of amenities includes the first amenity and excludes the second amenity, selecting a first classification as the individual classification; in response to determining that the list of amenities excludes the first amenity and excludes the second amenity, selecting a second classification as the individual classification; 20. The method of claim 17, further comprising:
19. 1. A system comprising: one or more processors of the machine; A memory for storing instructions; the instructions, when executed by the one or more processors, cause the machine to: receiving, by a network site of the listing network platform, an input including a plurality of images associated with each listing; presenting an option within a graphical user interface (GUI) for arranging the plurality of images according to classifications associated with the plurality of images; in response to receiving an input selecting the option, animating within the GUI a subset of the plurality of images to be shuffled into one or more of the classifications; presenting a template in the GUI after the subset of the plurality of images has been animated for a specified threshold period, the template including a plurality of regions, each region associated with a different classification; populating a first region of the plurality of regions associated with a first classification with a first image corresponding to the first classification and populating a second region of the plurality of regions associated with a second classification with a second image corresponding to the second classification; A system that causes a system to perform an operation including:
20. A machine-readable storage device having instructions embodied thereon that, when executed by a machine, cause the machine to: receiving, by a network site of the listing network platform, an input including a plurality of images associated with each listing; presenting an option within a graphical user interface (GUI) for arranging the plurality of images according to classifications associated with the plurality of images; in response to receiving an input selecting the option, animating within the GUI a subset of the plurality of images to be shuffled into one or more of the classifications; presenting a template in the GUI after the subset of the plurality of images has been animated for a specified threshold period, the template including a plurality of regions, each region associated with a different classification; populating a first region of the plurality of regions associated with a first classification with a first image corresponding to the first classification and populating a second region of the plurality of regions associated with a second classification with a second image corresponding to the second classification; A machine-readable storage device that causes operations including:
Citation Information
Patent Citations
Electronic apparatus, display controlling method, and program
JP2012138750A
Gallery application for displaying content
JP2013512508A
Classification label allocation device, classification label allocation method, and program
JP2017162025A
Apparatus, method and graphical user interface for messaging
JP2019514083A
Information processing device, information processing method, and information processing program
JP2023073148A