Automatic Image Editing System
By using machine learning to categorize and animate images within a GUI, the network site addresses computational delays and resource consumption issues, enhancing search efficiency and user experience in navigating accommodation listings.
Patent Information
- Application Number
- JP2024176835
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2023-10-10
- Filing Date
- 2024-10-09
- Publication Date
- 2025-11-26
- Estimated Expiration
- 2044-10-09
AI Technical Summary
Network sites face challenges in returning accurate and efficient search results for complex queries due to computational delays and resource consumption, particularly when users navigate vast amounts of accommodation listing content, and images often lack clear classification, making it difficult for users to identify relevant information quickly.
A network site employs machine learning models to categorize images automatically or semi-automatically, animating and arranging them into classifications within a graphical user interface (GUI), allowing users to efficiently browse and select listings based on image classifications, reducing computational resource usage and enhancing user experience.
This approach improves the efficiency of search results by minimizing computational resources needed and enabling users to easily identify and select relevant listings, thus reducing time consumption and improving user satisfaction.
Smart Images

Figure 0007776600000001 
Figure 0007776600000002 
Figure 0007776600000003
Abstract
Description
[Technical Field]
[0001] [CROSS-REFERENCE TO RELATED APPLICATIONS] This application is a non-provisional application of and claims priority to U.S. Provisional Application No. 63 / 543,341, filed October 10, 2023, which is incorporated herein by reference in its entirety.
[0002] [Technical field]
[0003] The present disclosure generally relates to dedicated machines that manage data processing and improvements to such variations, and to techniques by which such dedicated machines are improved over other dedicated machines for generating reservation listings on real estate property listing sites. [Background technology]
[0004] Network site users can create content for viewing and interaction (e.g., reservations, registrations, subscriptions, viewing listings) by other network site users. Posted content can be updated, created, or deleted, and it can be computationally challenging for a network site to return useful search results to a network site user searching for content (e.g., reservation listings) with specified parameters (e.g., date, category, price, quantity). For example, when a large number of users post and update content and a large number of users submit complex searches of the posted content, computational delays due to query complexity can cause inaccurate results to be returned and can incur significant computational resource consumption (e.g., processing, memory, network overhead). [Brief explanation of the drawings]
[0005] The various figures of the accompanying drawings depict only exemplary embodiments of the present disclosure and are not to be considered as limiting its scope. [Figure 1]1 is a block diagram illustrating a search and display system implemented in a networked environment, according to some examples. [Figure 2] 1 illustrates an example of a functional engine for a search and display system, according to some examples. [Figure 3] 1 illustrates a listings network site user interface generated by a listings network platform and search and display system (or listings search system), according to some examples. [Figure 4] 1 illustrates an exemplary configuration and user interface of a listing search system, according to some examples. [Figure 5] 1 illustrates an exemplary configuration and user interface of a listing search system, according to some examples. [Figure 6] 1 illustrates an exemplary configuration and user interface of a listing search system, according to some examples. [Figure 7] 1 illustrates an exemplary configuration and user interface of a listing search system, according to some examples. [Figure 8] 1 illustrates a flow diagram of various processes and methods for generating listings for a listing network site, according to some examples. [Figure 9] FIG. 1 is a block diagram illustrating a software architecture used to implement the disclosed system, according to some examples. [Figure 10] A machine is depicted as an exemplary computer system having instructions for causing the machine to implement the disclosed system, according to some examples. DETAILED DESCRIPTION OF THE INVENTION
[0006] The following description includes systems, methods, techniques, instruction sequences, and computing machine program products that embody illustrative examples of the present disclosure. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide an understanding of various embodiments of the inventive subject matter. However, it will be apparent to those skilled in the art that examples of the inventive subject matter may be practiced without these specific details. In general, well-known instruction instances, protocols, structures, and techniques are not necessarily shown in detail.
[0007] As described above, returning up-to-date results for complex queries on content posted on network sites can be difficult. In addition, navigating the vast amount of content available on network sites can be very complex and time-consuming. Such navigation may involve browsing multiple pages of information to find an appropriate result or action. To this end, a search and display system may be configured to efficiently search complex queries, enable browsing of network sites, return accurate results, and perform requested actions with low computational resource usage. In the following description, exemplary posted content is accommodation listings (e.g., reservation listings) posted on a network site for search and interaction with other end users, although other types of network site content posted by end users and searched for by other end users may likewise be implemented in the search and display system processes and methods, such as transportation, experiences, and / or events.
[0008] Generally, the listing platform may be searched for result listings available within a specified date range, price range, and / or other attributes such as amenities, cancellation policies, etc., which may be specified in a given query (e.g., text fields, drop-down menus, checkbox filters). To search and browse the listing platform, a user may access the listing platform on a particular user interface channel, for example, by telephone or through an application associated with the listing platform.
[0009] Sometimes, these systems present users with results that include various images of different aspects of the corresponding listing. The images may be grouped, arranged in a random order, or with no particular order at all. In either case, trying to ascertain which part of the physical accommodation is represented by a particular image can be difficult. For example, an image may include a depiction of a bed, and the listing may include multiple bedrooms. Such images provide little information about which bedroom is represented by the image, confusing the user. The user may need to navigate between multiple images to try to determine which bedroom is represented by a particular image. This repetitive, manual process can be prohibitively time-consuming and extremely frustrating for the end user, resulting in missed opportunities and wasted computing resources.
[0010] To address these technical problems, the disclosed technology provides a network site that automatically or semi-automatically categorizes (categorizes) images of a particular listing in an efficient manner. That is, the network site presents options within a graphical user interface (GUI) for arranging a plurality of images according to classifications associated with the plurality of images. In response to receiving input selecting an option, the network site animates a subset of the plurality of images within the GUI to be shuffled into one or more of the classifications. After, before, and / or during the animation of the subset of the plurality of images over a specified threshold period, the network site presents a template within the GUI including a plurality of regions, each associated with a different classification, and populates a region of the plurality of regions associated with a different classification with each image corresponding to that classification.
[0011] Such image classifications can help users easily identify and select listings of potential interest by accessing a GUI that presents images associated with the listings according to the classifications associated with the images. In this way, network sites can perform additional search queries, refine their initial search query, and prevent users from navigating multiple pages of information. This effectively reduces the amount of computational resources a given search end user needs to possess and consume, freeing up such resources to fulfill other tasks and other search requests.
[0012] In some aspects, techniques described herein relate to a method that includes receiving, by a network site of a listing network platform, input including a plurality of images associated with individual listings; presenting within a GUI options for arranging the plurality of images according to classifications associated with the plurality of images; in response to receiving input selecting an option, animating within the GUI a subset of the plurality of images to be shuffled into one or more of the classifications; after the subset of the plurality of images has been animated for a specified threshold period, presenting within the GUI a template including a plurality of regions, each associated with a different classification; and populating a first region of the plurality of regions associated with a first classification with a first image corresponding to the first classification and populating a second region of the plurality of regions associated with a second classification with a second image corresponding to the second classification.
[0013] In some aspects, the techniques described herein relate to methods that further include analyzing the plurality of images with a machine learning model to determine a respective classification associated with each of the plurality of images.
[0014] In some aspects, the techniques described herein relate to a method, in which a machine learning model analyzes a plurality of images in response to receiving an input selecting an option for arranging the plurality of images according to a classification associated with the plurality of images. The machine learning model can include multiple machine learning models. A first machine learning model can be used to assign a classification score to each image indicating the likelihood that the image corresponds to a particular classification. A second machine learning model can be applied to the images to cluster the images based on similarity. This can be used when a particular listing is associated with multiple of the same classification (e.g., multiple bedrooms). The second machine learning model can generate embeddings for the images and can cluster images with similar embeddings into the same bedroom, such that a first group of images are clustered and associated with the first bedroom and a second group of images are clustered and associated with the second bedroom.
[0015] In some aspects, the techniques described herein relate to a method, in which a machine learning model analyzes a plurality of images as each respective image of the plurality of images is received by an input, before presenting an option for pre-arranging the plurality of images within the GUI.
[0016] In some aspects, the techniques described herein relate to methods that further include receiving additional input including additional images associated with the listing, and in response to receiving the additional input, re-analyzing the plurality of images previously classified by the machine learning model along with the additional images to generate or update a classification for each of the plurality of images that includes the additional images.
[0017] In some aspects, the techniques described herein relate to a method, wherein the first classification includes a first room type and the second classification includes a second room type.
[0018] In some aspects, the techniques described herein relate to a method further including allowing a user to modify classifications associated with the plurality of images by interacting with templates presented in the GUI.
[0019] In some aspects, the techniques described herein relate to a method, wherein an order in which a plurality of areas are presented in a GUI is predefined, the order including: living area, full kitchen, kitchenette, dining area, bedroom, full bathroom, half bathroom, office, dedicated workspace, backyard, patio, balcony, front yard, deck, porch, courtyard, garden, terrace, rooftop, laundry area, garage, gym, exterior, pool, hot tub, theme room, children's playroom, bowling alley, movie theater, art studio, music studio, workshop, photography studio, darkroom, woodworking room, event space, library, game room, sunroom, and wine cellar.
[0020] In some aspects, the techniques described herein relate to a method further including selecting one or more images from the plurality of images based on one or more criteria, and presenting the selected one or more images in a graphical element that includes options for arranging the plurality of images according to classification.
[0021] In some aspects, the techniques described herein relate to a method, wherein the one or more criteria include at least one of an order in which images are presented in a listing or a respective classification of the images.
[0022] In some aspects, the techniques described herein relate to a method, wherein the subset of the plurality of images includes one or more selected images, the method further including presenting a graphical element on a template presenting an animation of the subset of the plurality of images, wherein each of a plurality of regions of the template includes a placeholder for a respective image.
[0023] In some aspects, the techniques described herein relate to a method, wherein a placeholder comprises a gray box.
[0024] In some aspects, the techniques described herein relate to a method further including presenting an animation including a celebratory graphic over a template including a first region having a first image and a second region having a second image.
[0025] In some aspects, the techniques described herein relate to a method, wherein the celebratory graphic includes at least one of confetti, balloons, or a logo associated with a network site.
[0026] In some aspects, the techniques described herein relate to a method further including determining that an individual classification of the plurality of classifications is not assigned to any of the plurality of images, identifying an individual region of the plurality of regions associated with the individual classification, and, in response to determining that the individual classification is not assigned to any of the plurality of images, populating the individual region within the GUI with a three-dimensional (3D) graphical element representing the individual classification.
[0027] In some aspects, the techniques described herein relate to methods that further include storing a database that associates a plurality of classifications with respective 3D graphical elements that represent the classifications, and retrieving the 3D graphical elements from the database by searching the database based on the respective classifications.
[0028] In some aspects, the techniques described herein relate to a method further including obtaining a list of amenities associated with each listing; detecting a physical space in each image of the plurality of images; selecting a respective classification from the plurality of classifications based on the list of amenities associated with each listing and the detected physical space; and associating the respective classification with the respective image.
[0029] In some aspects, the techniques described herein relate to methods further including determining that the list of amenities includes a first amenity and excludes a second amenity; selecting the first classification as an individual classification in response to determining that the list of amenities includes the first amenity and excludes the second amenity; and selecting the second classification as an individual classification in response to determining that the list of amenities excludes the first amenity and excludes the second amenity.
[0030] In some aspects, the techniques described herein relate to a system including one or more processors of a machine and a memory storing instructions that, when executed by the one or more processors, cause the machine to perform operations including receiving, by a network site of a listing network platform, input including a plurality of images associated with individual listings; presenting within a GUI options for arranging the plurality of images according to classifications associated with the plurality of images; and, in response to receiving input selecting an option, animating within the GUI a subset of the plurality of images to be shuffled into one or more of the classifications; and, after the subset of the plurality of images has been animated for a specified threshold period, presenting within the GUI a template including a plurality of regions each associated with a different classification; and populating a first region of the plurality of regions associated with a first classification with a first image corresponding to the first classification and populating a second region of the plurality of regions associated with a second classification with a second image corresponding to the second classification.
[0031] In some aspects, the techniques described herein relate to a machine-readable storage device embodying instructions that, when executed by the machine, cause the machine to perform operations including receiving, by a network site of a listing network platform, input including a plurality of images associated with individual listings; presenting within a GUI options for arranging the plurality of images according to classifications associated with the plurality of images; and, in response to receiving input selecting the option, animating within the GUI a subset of the plurality of images to be shuffled into one or more of the classifications; and, after the subset of the plurality of images has been animated for a specified threshold period, presenting within the GUI a template including a plurality of regions, each associated with a different classification; and populating a first region of the plurality of regions associated with a first classification with a first image corresponding to the first classification and populating a second region of the plurality of regions associated with a second classification with a second image corresponding to the second classification.
[0032] 1, an example of a general client-server based network architecture 100 is shown. A networked system 102, in the illustrative form of a network-based listings service system, provides server-side functionality to one or more client devices 110 via a network 104 (e.g., the Internet or a wide area network (WAN)). In some implementations, a user (e.g., a user 106) interacts with the networked system 102 using the client device 110.
[0033] 1, for example, illustrates a web client 112 (e.g., a browser), client application(s) 114, and programmatic clients 116 executing on a client device 110. The client device 110 includes the web client 112, client application(s) 114, and / or programmatic clients 116, singly, together, or in any suitable combination. Although one client device 110 is shown in FIG. 1, in other implementations, the network architecture 100 includes multiple client devices.
[0034] In various implementations, client device 110 may include a computing device including at least a display and communications capabilities that provide access to networked system 102 via network 104. Client device 110 includes, but is not limited to, a remote device, a workstation, a computer, a general-purpose computer, an Internet appliance, a handheld device, a wireless device, a portable device, a wearable computer, a cellular or mobile phone, a personal digital assistant (PDA), a smartphone, a tablet, an ultrabook, a netbook, a laptop, a desktop, a multiprocessor system, a microprocessor-based or programmable consumer electronics, a game console, a set-top box (STB), a networked personal computer (PC), a minicomputer, etc. In one example, client device 110 includes one or more of a touchscreen, an accelerometer, a gyroscope, a biometric sensor, a camera, a microphone, a global positioning system (GPS) device, etc.
[0035] The client device 110 communicates with the network 104 via a wired or wireless connection. For example, one or more portions of the network 104 may include an ad-hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a WAN, a wireless WAN (WWAN), a metropolitan area network (MAN), a portion of the Internet, a portion of the public switched telephone network (PSTN), a cellular network, a wireless network, a Wireless Fidelity (Wi-Fi®) network, a Worldwide Interoperability for Microwave Access (WiMax) network, another type of network, or any suitable combination thereof.
[0036] In some examples, the client device 110 includes one or more applications (also referred to as "apps") such as, but not limited to, a web browser, a book reader app (operable to read e-books), a media app (operable to present various media forms including audio and video), a fitness app, a biometric monitoring app, a messaging app, an electronic mail (email) app, an e-commerce site app (also referred to as a "marketplace app"), and a booking application for temporary stays or experiences at hotels, motels, or residences managed by other end users (e.g., posting end users, referred to as "hosts," who own their homes and rent out entire homes or private rooms).
[0037] In some implementations, client application(s) 114 include various components operable to present information to a user and communicate with networked system 102. In some examples, if an e-commerce site application is included on client device 110, the application is configured to provide at least some of the user interface and functionality locally, with the application configured to communicate with networked system 102 as needed for data or processing capabilities that are not available locally (e.g., access to a database of items available for sale, authentication of a user, verification of a payment method). Conversely, if an e-commerce site application is not included on client device 110, client device 110 can use its web browser to access an e-commerce site (or a variant thereof) hosted on networked system 102.
[0038] Web clients 112 access various systems of networked system 102 through a web interface supported by web server 122. Similarly, programmatic clients 116 and client application(s) 114 access various services and functions offered by networked system 102 through a programmatic interface provided by application program interface (API) server 120.
[0039] A user (e.g., user 106) may include a person, a machine, or other means of interacting with client device 110. In some examples, user 106 is not part of network architecture 100 but interacts with network architecture 100 through client device 110 or another means. For example, user 106 provides input (e.g., touchscreen input or alphanumeric input) to client device 110, which is communicated to networked system 102 over network 104 by a second user interaction channel. In this example, networked system 102, in response to receiving input from user 106, communicates information to be presented to user 106 to client device 110 over network 104. In this manner, user 106 can interact with networked system 102 using client device 110. As another example, user 106 provides input (e.g., voice input) to client device 110, which is communicated to networked system 102 over network 104 in the form of audio packets or audio data.
[0040] The API server 120 and the web server 122 are coupled to one or more application server(s) 140, providing programmatic and web interfaces, respectively, to the one or more application server(s) 140. The application server(s) 140 may host a listings network platform 142 and a listings search system 150, each of which includes one or more modules or applications, each of which may be embodied as hardware, software, firmware, or any combination thereof. The application server(s) 140, in turn, are shown coupled to one or more database server(s) 124 that facilitate access to one or more information storage repositories or database(s) 126. In one example, the database(s) 126 are storage devices that store information (e.g., inventory, image data, catalog data, 3D and / or 2D representations of room / space classifications) posted to the listings network platform 142. The database(s) 126 also store digital product information, according to some examples.
[0041] Additionally, the social network platform 131 is shown as executing on third-party server(s) 130. Furthermore, the social network platform 131 can programmatically access the networked system 102 through a programmatic interface provided by the API server 120. The social network platform 131 may include a social network website, a messaging platform, and one or more APIs. In some examples, the electronic messages described below are messages (e.g., social media chat messages, posts, pings ("hello" notifications), etc.) sent to a given user via the messaging system of the social network platform 131.
[0042] The listings network platform 142 provides several publishing functions and listing services to users accessing the networked system 102. Although the listings network platform 142 is shown in FIG. 1 as forming part of the networked system 102, it will be appreciated that in alternative examples, the listings network platform 142 is separate from the networked system 102 and it may form part of a separate and distinct web service.
[0043] 1 employs a client-server architecture, the disclosed techniques are not limited to such an architecture and can equally be implemented in, for example, a distributed or peer-to-peer architecture system. Various systems of application server(s) 140 (e.g., listings network platform 142 and listings search system 150) may also be implemented as standalone software programs that do not necessarily have networking capabilities.
[0044] The listing network platform 142 can be hosted on dedicated or shared server machines communicatively coupled to enable communication between the server machines. The components themselves are communicatively coupled to each other and to various data sources (e.g., via appropriate interfaces) to allow information to be passed between applications or to allow applications to share and access common data. Additionally, the components access one or more database(s) 126 via a database server 124. The listing network platform 142 provides several publishing and listing mechanisms whereby sellers (also referred to as “first users,” posting users, or hosts) can list (or publish information about) goods or services for sale or barter, and buyers (also referred to as “second users,” search users, or guests) can express interest in or indicate a desire to purchase or barter such goods or services, and complete transactions (e.g., trades) related to the goods or services.
[0045] 2 shows an exemplary function engine of listings search system 150, according to some examples. As shown, listings search system 150 includes an image input component 220, a classifier component 240, and a listings tour GUI component 260. In some examples, a user may wish to perform some action on listings network platform 142. For example, a user, such as an accommodation host, may wish to create a listing for a reservation, modify one or more listings for a reservation maintained by listings network platform 142, and / or perform any other available function on listings network platform 142.
[0046] In some examples, the GUI allows a user to access reservation listings associated with the user, for example, if the user is the host (e.g., owner, manager) of a listing. The GUI receives input from the user selecting an individual listing. In response, the GUI presents various information about the listing, including one or more images, amenities, availability dates, description information, location information, and / or price or reservation details. The GUI can receive input from the user requesting to modify, edit, or adjust any of the presented information about the listing. Once the information is modified and optionally approved by a moderator, the listing is updated and made available to other users of the network site seeking accommodations.
[0047] In some examples, the GUI allows users to improve the appearance and functionality of listings, for example, by automatically assigning images to respective classifications. For example, the network site assigns a classification to each image in a listing and then categorizes the images by space type based on the classification. That is, a first machine learning model generates and associates classifications for the images. A second machine learning model generates image embeddings to cluster similar images and associate such similar images with the same classification (e.g., room type).
[0048] The network site generates a tour GUI containing images organized according to space type (also called classification or category). The tour GUI presents one or more images for one or more of the classifications. This allows the host and one or more end users (e.g., guests) to quickly and easily determine what is shown or represented by the different images. Input from the GUI allows selection of individual classifications, causing a subset of images associated with each classification to be presented within the new GUI.
[0049] To assign a classification to each image, the image input component 220 processes the images as they are input to generate and assign classifications or categories to the images. That is, a collection or plurality of images may be initially received from a user and associated with a particular listing. The plurality of images are provided to the classifier component 240 to generate classifications for the images. To reduce the processing time and resources required to perform such classifications on the fly in real time, this operation may be performed before receiving input from the host for generating the photo tour.
[0050] In one example, the classifier component 240 generates a prediction score indicating how likely an image is to be associated with a particular room type. A room type is selected for an image based on a threshold prediction score. For example, if an image has a prediction score of 0.8 for being associated with a bedroom and a prediction score of 0.4 for being associated with a living room, a bedroom room type would be selected by the system for that image. The classifier component 240 implements one or more machine learning models (e.g., artificial neural networks and / or convolutional neural networks) trained to analyze features of a collection of images to predict the classification of each of the images. The classifier component 240 can include a second machine learning model for clustering images based on embeddings, as described below.
[0051] Specifically, the classifier component 240 may be trained by processing training data including different image groups and their respective ground truth classifications. The classifier component 240 may access each image group and extract features from the image group. The classifier component 240 may predict or estimate a classification for each of the images in the image group based on the extracted features. Each image may be assigned to only one classification. The classifier component 240 may then obtain a ground truth classification associated with the image group. The classifier component 240 may compare the classification associated with each image in the estimated image group with the corresponding ground truth classification. The classifier component 240 may calculate a deviation based on the comparison and then update the parameters of the classifier component 240 based on the deviation. The classifier component 240 may repeat this training process for additional image groups until a stopping criterion is reached.
[0052] In some examples, the image input component 220 receives input adding a new or additional image in association with a listing where a set of previous images has already been classified. In response, the image input component 220 provides the new image to the classifier component 240 along with instructions for classifying the new image. The classifier component 240 retrieves the previously classified images along with their classifications. The classifier component 240 then reanalyzes all images, including the new image and the previously classified images. The classifier component 240 can generate or estimate a new classification for the new image and update one or more classifications associated with the previously classified images. In some cases, only the new image is classified independently from other images, but a second machine learning model is applied collectively to all images and their classifications to group or cluster similar images based on their generated embeddings.
[0053] For example, the classifier component 240 may determine that a first image in a collection of previously classified images was associated with a first classification (e.g., a first bedroom classification). The classifier component 240 processes a new image together with the first image to derive new information about the first image and the new image. In particular, the new image may represent a bedroom that contains similar features as the first image. In such a case, the classifier component 240 may determine that the first image and the new image belong to the same classification. In addition, the classifier component 240 may determine that the new image corresponds to a second classification (e.g., a second bedroom classification). In such a case, the classifier component 240 may update the classification associated with the first image from the first classification to the second classification in response to processing the new image (not yet classified) together with the previously classified image.
[0054] In some examples, the classifier component 240 generates a predicted score for each image classification that indicates the likelihood that the image belongs to a particular classification or room. The classifier component 240 retrieves all images and their respective scores and, for each classification, determines whether the score of each image exceeds a threshold. The classifier component 240 assigns a classification to images that have a classification score that exceeds the threshold. For example, if an image has a first score for a first classification and a second score for a second classification, the classifier component 240 compares the first score to a first threshold and the second score to a second threshold. If the first score does not exceed the threshold but the second score does exceed the threshold, the classifier component 240 assigns the image to the second classification instead of the first classification.
[0055] After obtaining or generating a classification and / or score for each of the images, the classifier component 240 processes the classified images using an additional machine learning model (e.g., an additional artificial neural network). The additional machine learning model generates an embedding for each image (before and / or after the image is classified by the first machine learning model). The embeddings are compared to each other for each of the images to cluster and / or group images with similar embeddings. For example, the classifier component 240 obtains the embedding generated for the first image and the embedding generated for the second image and each additional image. The classifier component 240 calculates a similarity score (e.g., by calculating cosine similarity) between the first image embedding and the embedding of each additional image, including the second image embedding. The classifier component 240 determines that the embedding of the first image is similar to the embedding of the second image by more than a specified threshold. In such a case, the classifier component 240 groups the first image with the second image to generate a first cluster of images. Similar operations are performed to cluster all of the other images.
[0056] The classifier component 240 provides the classified images to the listings tour GUI component 260. The listings tour GUI component 260 presents a GUI that allows a user to request automatic classification of images associated with a listing. In some cases, the classifier component 240 performs the classification in response to input received from the GUI requesting that the classification be performed. In some cases, the classifier component 240 performs the classification before the GUI presents an option for the user to request that the classification be performed. After the images are classified, the image classifications are presented to the end user on the listings network platform 142 in a GUI or organized manner. For example, as described below, a GUI can be presented in which different regions of the GUI correspond to different classifications and a single representative image is presented in each region.
[0057] FIG. 3 illustrates a listings network site user interface 300 (e.g., a mobile application user interface, a web browser user interface) generated by the listings network platform 142 and the listings search system 150, according to some examples. The user interface 300 can be presented by a program client 116 implemented on a client device 110. As shown, the user interface 300 includes a search field 310, filter menu elements 315 (e.g., location type, amenities), and a search button 320. A user enters a listing query, such as a search for temporary housing in San Francisco, into the search field 310 and enters a category restriction from the filter menu element 315 for "Entire Place" (e.g., the user is seeking to rent the entire residence on a given date, rather than renting a single room in another person's residence). The user can customize the query directly using terms entered in the search field 310 or filters listed via selection of the filter menu element 315. Additionally, the user can select dates using a date drop-down element 317 to select a specific date range for the temporary stay. For example, a user may select the date drop-down element 317 and a pop-up calendar (not shown in FIG. 3 ) to specify that their stay in San Francisco is specifically from July 16, 2021 to July 18, 2021. In some cases, a user may use the filter menu element 315 to provide one or more attributes of the search query, such as specifying the ages of children and minimum / maximum duration of stay.
[0058] Submitting a query (e.g., via selection of search button 320 or automatically by selection of composite listings element 313 (split stay option) or date drop-down element 317) sends a communication from program client 116 to listings retrieval system 150. Listings retrieval system 150 generates output including a result display of listings (e.g., graphical objects) that match the query and sends this output to program client 116.
[0059] The results are then displayed in the listings results area 305 using respective graphical objects (also called listing indicators). The user can then select the listing's graphical object or navigate to additional pages via page navigation elements 325. In some examples, the user interface 300 includes a set of combined listings 323 along with the individual listings displayed in the results area 305. The composite listing 323 may be located in a dedicated area within the display, above an individual listing, between two individual listings, and / or below an individual listing. In some examples, the composite listing 323 is provided in response to receiving input selecting the composite listing element 313. In some examples, the composite listing 323 is presented automatically without receiving input selecting the composite listing element 313.
[0060] In some examples, the composite listing 323 is displayed in a different slot or portion of the display compared to other individual listings based on the type of client device being used to access the system. For example, on a mobile device, on the first page, the composite listing 323 may be placed in slots 3, 6, 9, and 12, while on a desktop computer, the same composite listing 323 may be presented in slots 5, 9, 14, and 20 for better visual balance. As referred to herein, the term "slot" refers to an area of the display in which categories are presented. In some cases, the composite listing 323 is excluded from presentation for last-minute stays, such as when the travel date is within 48 hours of check-in or travel start. In some examples, the composite listing 323 includes individual listings for destinations or stays that are at least a two-hour drive apart, but not more than a ten-hour drive apart. In some examples, the composite listing 323 excludes repeated pairs of the same individual listings. In some examples, the composite listing 323 relates to pairs of individual listings from different neighborhoods and locations. In such cases, the neighbors and listings may be repeated across pairs of composite listings 323 .
[0061] To take a specific example, a listing request may be received for a one-month stay on the island of Kauai, specifying a beachfront home. In this example, the user identifies only three individual listings that meet these search parameters, but 10 additional stays qualify as composite listings 323. Composite listings 323 may represent an opportunity to experience different sides of the island throughout the trip and may include options such as splitting time between two cities (e.g., Koloa and Hanalei), splitting time between two other cities (e.g., Koloa and Lihue), splitting time between two listings within Koloa that are one mile apart, splitting time between related cities (e.g., Poipu and Princeville) and / or other related cities (e.g., Lihue and Princeville). Selecting composite listing element 313, or the user's automatic identification of composite listings without composite listing element 313, may provide 40% more unique inventory for the user to choose from that would be excluded if only individual listings were presented.
[0062] In some examples, the presentation of the composite listing 323 may vary based on the type of category selected. For example, if the national parks category is selected, the composite listing 323 may include points of interest in two different national parks that are at least two hours' drive away and at most ten hours' drive away. In some cases, if the national parks are very far apart, they may still be paired even if their distance exceeds these thresholds. The graphical representation of the composite listing 323 may visually identify two national park listings that form part of the same composite listing without specifying the region / city name. As another example, if the surfing category is selected, two different surfing destinations that are at least two hours' drive away and at most ten hours' drive away may be selected. The graphical representation of the composite listing 323 may visually identify two surfing destinations that form part of the same composite listing using the region / city name.
[0063] The listing tour GUI component 260 can instruct the user interface 300 to present images of a particular listing selected from the results area 305. The images can be presented in groups. For example, a first image or a first collection of images associated with a first classification can be presented in a first area of the GUI dedicated to presenting information / images for the first classification. A second image or a second collection of images associated with a second classification can be presented in a second area of the GUI dedicated to presenting information / images for the second classification. Different areas with different collections of images can be presented simultaneously within the same GUI.
[0064] 2, the listings tour GUI component 260 allows a host to automatically or semi-automatically generate or define the classifications that are provided to end users in a listings tour. For example, the listings tour GUI component 260 presents a GUI, such as the exemplary GUI 400 shown in FIG. 4. Specifically, the GUI 400 is accessed in response to receiving input from a user requesting to edit or modify a listing for booking. Among the multiple fields of the listing that can be modified, the GUI 400 includes a photo editor field (or area) that allows the host to modify the placement of images associated with the listing and / or add new images.
[0065] The GUI 400 includes a graphical element 410 (e.g., a card) within a photo editor field. The graphical element 410 includes an information panel that describes the host's ability to instantly and automatically generate a classification of previously uploaded images. The listings tour GUI component 260 accesses a list of previously uploaded images. The listings tour GUI component 260 identifies an order associated with each uploaded image. The order indicates the order in which the images are presented in the GUI to an end user searching for or accessing a listing. The listings tour GUI component 260 can select a subset of images that are highest in the order, such as the first three images in the sequence or order of images. The listings tour GUI component 260 can generate thumbnails of the first three images and present those thumbnails 412 as part of the graphical element 410.
[0066] In some examples, the listings tour GUI component 260 accesses classifications associated with previously uploaded images, as described above. In some cases, the listings tour GUI component 260 accesses groups of similar images generated by a second machine learning model based on the embeddings and / or directly accesses the classifications generated by the first machine learning model. The listings tour GUI component 260 selects a first image associated with a first classification (group or cluster), a second image associated with a second classification (group or cluster), a third image associated with a third classification (group or cluster), and / or any number of additional images up to a specified threshold. The listings tour GUI component 260 generates thumbnails 412 of the first, second, and third images and presents those thumbnails 412 as part of the graphical element 410.
[0067] In some examples, the listings tour GUI component 260 presents an option 414 for triggering the generation of a photo tour based on classifications associated with previously uploaded images. In response to receiving input selecting option 414, the listings tour GUI component 260 presents a sequence of user interfaces, such as, for example, exemplary user interfaces 500, 501, and 502 shown in FIG. 5. This sequence of user interfaces 500, 501, and 502 indicates to the user the progress in classifying previously uploaded images into respective classifications or categories for generating a photo tour.
[0068] For example, a first user interface 500 can be presented that includes a graphical card 510. The graphical card 510 includes the same or a different collection of images 512 presented in the graphical element 410 as thumbnails 412. The graphical card 510 includes a message indicating that previously uploaded images have been sorted into their respective classifications (or rooms or spaces). In some examples, the listing tour GUI component 260 retrieves a photo tour template. The photo tour template includes dedicated regions organized according to specified criteria. Each region is associated with and represents a different classification, room, or space. Each region includes an identifier for the corresponding classification. Before the classification of the images is finished (or before the visual animation / simulation of the classification is completed, where the images have already been classified and clustered by the first and second machine learning models before input is received to generate the photo tour), the photo tour template includes placeholders (e.g., gray boxes) in each of the different regions. This photo tour template with placeholders can be presented on a background 520 along with the graphical card 510. In some cases, as each image is classified, the corresponding area of the photo tour template is populated with the classified image.
[0069] In some cases, the same or different collections of images 512 are animated over a threshold period (e.g., 5 or 10 seconds) to represent to the user their progress in classifying the images and associating them with regions of the template. For example, as shown in second user interface 501, a first animation 530 is presented in which the same or different collections of images 512 are randomly scattered in different directions. Then, after second user interface 501, a third user interface 502 is presented. In the sequence of user interfaces 502, the images that were randomly scattered in different directions are randomly arranged in a grid 540 (e.g., a 4x4 grid), with a single image presented in each quadrant of the grid. During a first time interval, the first and second images presented along a first dimension 542 (e.g., a first row) of grid 540 are swapped so that the position of the first image takes the position of the second image and the position of the second image takes the position of the first image. This is represented by an animation showing the first image sliding towards the position of the second image, and then the position of the second image sliding towards the position of the first image.
[0070] During a second time interval, the second and third images presented along the second dimension 544 (e.g., the first column) of the grid 540 are swapped such that the second image takes the position of the third image and the third image takes the position of the second image. This is represented by an animation showing the second image sliding toward the position of the third image, and then the position of the third image sliding toward the position of the second image. The process of swapping images along the first and second dimensions 542 and 544 is repeated for a threshold period (e.g., 5 seconds). After the threshold period has elapsed, the listings tour GUI component 260 presents an option 550 for accessing a photo tour preview. The listings tour GUI component 260 receives input selecting option 550 and, in response, navigates the user through a sequence of user interfaces, such as the exemplary user interfaces 600 and 601 shown in FIG. 6 .
[0071] Specifically, in response to receiving input selecting option 550, listing tour GUI component 260 animates card 610 showing grid 540 sliding down the screen while presenting a celebratory animation. The celebratory animation can include many different types of graphical elements, such as confetti 620, balloons, and / or the listing site's logo 622 sliding down the screen. The photo tour template continues to be presented in the background, for example, in dark contrast with card 610, which is presented in lighter contrast. Once card 610 completely disappears from the screen, the contrast of the photo tour template increases while continuing to present the celebratory animation. Specifically, as shown in user interface 601, photo tour template 630, which is currently populated with images, is presented in lighter contrast than the photo tour template shown in user interface 600. The sequence in user interface 601 also presents the celebratory animation for a specified threshold period.
[0072] After the threshold period has elapsed, the listing tour GUI component 260 may present a sequence of user interfaces, such as user interfaces 700 and 701 shown in FIG. 7. The sequence of user interfaces 700 and 701 no longer includes the celebratory animation. As shown in user interface 700, multiple regions associated with different classifications are presented. For example, a first region associated with a first classification (group or cluster) is presented in the upper left corner. Along with the first region, a description of the first classification (e.g., bedroom) and the amount of images associated with the first classification are presented. The first region includes a random or designated image from the set of images associated with the first classification. In response to receiving an input selecting the first region or the designated image, all of the corresponding images in the set of images associated with the first classification are presented.
[0073] A second region associated with a second classification is presented in the upper right corner next to the first region. A description of the second classification (e.g., "shared living room") and the quantity of images associated with the second classification are presented along with the second region. The second region contains a random or designated image from the set of images associated with the second classification. In response to receiving an input selecting the second region or the designated image, all of the corresponding images in the set of images associated with the second classification are presented.
[0074] In some cases, a prompt 710 is presented instructing the user or host to review the classification by browsing the populated photo tour template. The listings tour GUI component 260 can receive input to navigate the photo tour template to show each of the regions and their corresponding assigned images and descriptions. The listings tour GUI component 260 can receive input to move an image from one region to another and change the classification associated with the image.
[0075] In some examples, the listings tour GUI component 260 determines that the photo tour template includes individual classifications 720 that are not associated with any of the images. That is, the listings tour GUI component 260 compares the classifications associated with all of the images and determines that none of the classifications match the individual classifications 720. In this case, the listings tour GUI component 260 includes an individual region corresponding to the individual classification 720 for display with the other regions in the photo tour template. The listings tour GUI component 260 can search a database of 3D or 2D graphical elements to find or identify the 3D or 2D graphical element corresponding to the individual classification 720. The database can associate each 3D or 2D graphical element with a corresponding classification. Each 2D or 3D graphical element includes visual attributes that represent the corresponding classification. The listings tour GUI component 260 retrieves the 3D or 2D graphical element from the database associated with the individual classification 720 and presents the retrieved 3D or 2D graphical element 730 as part of the individual region of the photo tour template. The listings tour GUI component 260 can receive input selecting a 3D or 2D graphical element 730. In response, the listings tour GUI component 260 allows the user to submit or upload new images or photos corresponding to individual classifications 720 to replace the display of the 3D or 2D graphical element 730 in the individual areas. In this manner, the listings tour GUI component 260 can identify particular key rooms that are empty in that no corresponding images or photos exist for that key room or classification. Based on this identification, the listings tour GUI component 260 can recommend to the user (host) to submit, upload, or add additional images corresponding to the key room or classification. This helps to create a complete loop between the completion of a photo tour and the publication of the photo tour for guest users to view.
[0076] The ordering of regions within a photo tour template may be static and unchangeable by users. In some cases, users may change the ordering or organization of regions in a photo tour template. In some cases, only users with advanced status or subscriptions may be permitted to change the ordering or organization of regions in a photo tour template. In some cases, the ordering of regions in a photo tour template may be: living area, full kitchen, kitchenette, dining area, bedroom, full bathroom, half bathroom, office, dedicated workspace, backyard, patio, balcony, front yard, deck, porch, courtyard, garden, terrace, rooftop, laundry area, garage, gym, exterior, pool, hot tub, theme room, children's playroom, bowling alley, movie theater, art studio, music studio, workshop, photography studio, darkroom, woodworking room, event space, library, game room, sunroom, and wine cellar. Some of these may be omitted in certain situations.
[0077] In some examples, after the images are classified and / or clustered, the classification can be further refined based on amenities associated with the listings. In such cases, the listings tour GUI component 260 obtains a list of amenities associated with each listing and detects physical spaces within each image of the multiple images. For example, the listings tour GUI component 260 applies one or more trained machine learning models to extract features from each image and estimate amenities or physical spaces characterized within each image. The listings tour GUI component 260 selects an individual classification from the multiple classifications based on the list of amenities associated with each listing and the detected physical spaces, and associates the individual classification with each image. The listings tour GUI component 260 can determine that the list of amenities includes a first amenity and excludes a second amenity. In response to determining that the list of amenities includes the first amenity and excludes the second amenity, the listings tour GUI component 260 selects the first classification as the individual classification. The listing tour GUI component 260 selects the second classification as the individual classification in response to determining that the list of amenities excludes the first amenity and excludes the second amenity.
[0078] For example, the listing tour GUI component 260 can assign images based on existing amenities specified in the listing. Specifically, the listing tour GUI component 260 detects outdoor spaces (patios, balconies, backyards, porches, etc.) in each image. If the listing tour GUI component 260 determines that a listing has a backyard amenity but not a patio / balcony amenity, the listing tour GUI component 260 assigns the image to the backyard space (or classification). If the listing tour GUI component 260 determines that a listing has a patio / balcony amenity but not a backyard amenity, the listing tour GUI component 260 assigns the image to the balcony classification if the listing corresponds to an apartment or condominium, and assigns the image to the patio classification if the listing does not correspond to an apartment or condominium. If the listing tour GUI component 260 determines that a listing has both backyard and patio / balcony amenities, the listing tour GUI component 260 does not assign the image to any space or classification. The listing tour GUI component 260 may still create backyard and patio rooms or categories, but may prevent the assignment of images to those rooms or categories. If the listing tour GUI component 260 determines that the listing excludes backyard and patio / balcony amenities, the listing tour GUI component 260 assigns images to exterior spaces or categories.
[0079] As another example, if a pool is detected in the image, the listing tour GUI component 260 determines whether the listing includes a pool amenity and excludes a backyard amenity. In such a case, the listing tour GUI component 260 assigns the image to the pool space or classification if the listing corresponds to an apartment or condominium, or assigns the image to the backyard amenity category if the listing does not correspond to an apartment or condominium.
[0080] In some examples, the listings tour GUI component 260 groups similar images, for example, by applying a second machine learning model to the classification generated by the first machine learning model and / or by processing amenities or other attributes associated with the listing. That is, if a listing specifies more than one bedroom / bathroom, the listings tour GUI component 260 can apply a second machine learning model to the images and their classification to generate clusters or groups of images. For example, the listings tour GUI component 260 can cluster images with a bedroom type classification with full bathroom and half bathroom type classifications of images. If a listing has only one bedroom, the listings tour GUI component 260 assigns images with a bedroom to the bedroom classification without applying a second machine learning model to generate clusters of images based on similarity. If a listing has only one bathroom, the listings tour GUI component 260 assigns images with a bathroom to the bathroom classification. If the listing includes multiple bedrooms / bathrooms, the listing tour GUI component 260 applies a second machine learning model to image classification to assign or group images into bedroom numbers in order of largest to smallest group size based on the similarity scores corresponding to the embeddings. If the listing tour GUI component 260 determines that the listing does not include a bedroom, the listing tour GUI component 260 can create a sleeping area object or classification and associate any images that have a bedroom with the sleeping area object or classification.
[0081] For example, the listing tour GUI component 260 can conditionally apply a second machine learning model to classify images based on whether the listing includes multiple amenities of the same type (e.g., whether the listing includes multiple bedrooms). In such a case, the listing tour GUI component 260 can group a first cluster of similar images (having similar embeddings) with a first bedroom determined by the second machine learning model embeddings, and can group a second cluster of similar images (having similar embeddings) with a second bedroom determined by the second machine learning model embeddings.
[0082] 8 illustrates a flow diagram of various processes and methods 800 for generating listings on a listing network site, according to some examples. These processes and methods 800 may be performed in any order or sequence by any of the components described above or below, such as listing search system 150.
[0083] In operation 805, the listing search system 150 receives input including a plurality of images associated with individual listings by a network site of a listing network platform, as described above.
[0084] In operation 810, the listings search system 150 presents options in the GUI for arranging the images according to the classifications associated with the images, as described above.
[0085] At operation 815, the listing search system 150 animates the subset of the plurality of images in the GUI as being shuffled into one or more of the categories in response to receiving input selecting an option, as described above.
[0086] In operation 820, after the subset of the multiple images has been animated for a specified threshold period, the listing search system 150 presents a template in the GUI that includes multiple regions, each associated with a different classification, as described above.
[0087] In operation 825, the listing search system 150 populates a first region of the plurality of regions associated with the first classification with a first image corresponding to the first classification, and populates a second region of the plurality of regions associated with the second classification with a second image corresponding to the second classification, as described above. Example
[0088] [Example 1] A method comprising: receiving, by a network site of a listing network platform, input including a plurality of images associated with individual listings; presenting within a graphical user interface (GUI) options for arranging the plurality of images according to classifications associated with the plurality of images; in response to receiving input selecting an option, animating within the GUI a subset of the plurality of images to be shuffled into one or more of the classifications; after the subset of the plurality of images has been animated for a specified threshold period, presenting within the GUI a template including a plurality of regions, each associated with a different classification; and populating a first region of the plurality of regions associated with a first classification with a first image corresponding to the first classification and populating a second region of the plurality of regions associated with a second classification with a second image corresponding to the second classification.
[0089] [Example 2] The method of Example 1, further comprising analyzing the plurality of images with a machine learning model to determine a respective classification associated with each of the plurality of images.
[0090] [Example 3] The method of example 2, wherein the machine learning model analyzes the plurality of images in response to receiving an input selecting an option.
[0091] [Example 4] The method of Example 2 or 3, wherein the machine learning model analyzes the plurality of images before presenting an option for pre-arranging the plurality of images within the GUI as each respective image of the plurality of images is received by input.
[0092] [Example 5] The method described in Example 4, further comprising the steps of receiving additional input including additional images associated with the listing, and in response to receiving the additional input, reanalyzing the plurality of images previously classified by the machine learning model together with the additional images to generate or update a classification for each of the plurality of images including the additional images.
[0093] [Example 6] The method according to any one of Examples 1 to 5, wherein the first classification includes a first room type and the second classification includes a second room type.
[0094] [Example 7] A method as described in any one of Examples 1 to 6, further comprising a step of allowing a user to modify classifications associated with multiple images by interacting with templates presented in the GUI.
[0095] Example 8: The method of any one of Examples 1 to 7, wherein the order in which the multiple areas are presented in the GUI is predefined, and the order includes: living area, full kitchen, kitchenette, dining area, bedroom, full bathroom, half bathroom, office, dedicated workspace, backyard, patio, balcony, front yard, deck, porch, courtyard, garden, terrace, rooftop, laundry area, garage, gym, exterior, pool, hot tub, theme room, children's playroom, bowling alley, movie theater, art studio, music studio, workshop, photography studio, darkroom, woodworking room, event space, library, game room, sunroom, and wine cellar.
[0096] [Example 9] A method as described in any one of Examples 1 to 8, further comprising the steps of selecting one or more images from the plurality of images based on one or more criteria, and presenting the selected one or more images in a graphical element including options for arranging the plurality of images according to classification.
[0097] [Example 10] The method of Example 9, wherein the one or more criteria include at least one of the order in which the images are presented in the listing or the respective classification of the images.
[0098] [Example 11] The method described in Example 9 or 10, wherein the subset of the plurality of images includes one or more selected images, and the method further includes a step of presenting a graphical element on the template that presents an animation of the subset of the plurality of images, and each of the plurality of regions of the template includes a placeholder for a respective image.
[0099] [Example 12] The method described in Example 11, wherein the placeholder includes a gray box.
[0100] [Example 13] A method described in any one of Examples 1 to 12, further comprising a step of presenting an animation including a celebratory graphic on a template including a first area having a first image and a second area having a second image.
[0101] [Example 14] The method described in Example 13, wherein the celebration graphic includes at least one of confetti, balloons, or a logo associated with the network site.
[0102] [Example 15] A method as described in any one of Examples 1 to 14, further comprising the steps of determining that an individual classification of the plurality of classifications is not assigned to any of the plurality of images, identifying an individual region of the plurality of regions associated with the individual classification, and in response to determining that the individual classification is not assigned to any of the plurality of images, populating the individual region within the GUI with a three-dimensional graphical element representing the individual classification.
[0103] [Example 16] The method described in Example 15, further comprising the steps of storing a database associating a plurality of classifications with respective three-dimensional graphical elements representing the classifications, and retrieving the three-dimensional graphical elements from the database by searching the database based on the individual classifications.
[0104] [Example 17] A method described in any one of Examples 1 to 16, further comprising the steps of obtaining a list of amenities associated with each listing, detecting physical spaces within each image of the plurality of images, selecting an individual classification from the plurality of classifications based on the list of amenities associated with each listing and the detected physical space, and associating the individual classification with each image.
[0105] [Example 18] The method described in Example 17, further comprising the steps of determining that the list of amenities includes a first amenity and excludes a second amenity, selecting the first classification as an individual classification in response to determining that the list of amenities includes the first amenity and excludes the second amenity, and selecting the second classification as an individual classification in response to determining that the list of amenities excludes the first amenity and the second amenity.
[0106] [Example 19] A system comprising one or more processors of a machine and a memory storing instructions, which, when executed by the one or more processors, cause the machine to perform operations including receiving input including a plurality of images associated with individual listings by a network site of a listing network platform; presenting options within a graphical user interface (GUI) for arranging the plurality of images according to classifications associated with the plurality of images; in response to receiving input selecting an option, animating a subset of the plurality of images within the GUI to be shuffled into one or more of the classifications; after the subset of the plurality of images has been animated for a specified threshold period, presenting a template within the GUI including a plurality of regions each associated with a different classification; and populating a first region of the plurality of regions associated with a first classification with a first image corresponding to the first classification and populating a second region of the plurality of regions associated with a second classification with a second image corresponding to the second classification.
[0107] [Example 20] A machine-readable storage device embodying instructions that, when executed by the machine, cause the machine to perform operations including receiving, by a network site of a listing network platform, input including a plurality of images associated with individual listings; presenting within a graphical user interface (GUI) options for arranging the plurality of images according to classifications associated with the plurality of images; in response to receiving input selecting an option, animating within the GUI a subset of the plurality of images to be shuffled into one or more of the classifications; after the subset of the plurality of images has been animated for a specified threshold period, presenting within the GUI a template including a plurality of regions each associated with a different classification; and populating a first region of the plurality of regions associated with a first classification with a first image corresponding to the first classification and populating a second region of the plurality of regions associated with a second classification with a second image corresponding to the second classification.
[0108] FIG. 9 is a block diagram 900 illustrating the architecture of software 902 that may be installed on any one or more of the devices described above. It will be understood that FIG. 9 is merely a non-limiting example of a software architecture and that many other architectures may be implemented to facilitate the functionality described herein. In various embodiments, the software 902 is implemented by hardware, such as machine 1000 of FIG. 10, which includes a processor 1010, memory 1030, and input / output (I / O) components 1050. In this exemplary architecture, the software 902 may be conceptualized as a stack of layers, with each layer providing a specific function. For example, the software 902 includes layers such as an operating system 904, libraries 906, frameworks 908, and applications 910. In operation, the applications 910 invoke API calls 912 through the software stack and receive messages 914 in response to the API calls 912, consistent with some embodiments.
[0109] In various implementations, the operating system 904 manages hardware resources and provides common services. The operating system 904 includes, for example, a kernel 920, services 922, and drivers 924. The kernel 920, consistent with some embodiments, serves as an abstraction layer between the hardware and other software layers. For example, the kernel 920 provides memory management, processor management (e.g., scheduling), component management, networking, and security configuration, among other functions. The services 922 may provide other common services for the other software layers. The drivers 924, according to some embodiments, are responsible for controlling or interfacing with the underlying hardware. For example, the drivers 924 may include a display driver, a camera driver, a BLUETOOTH or BLUETOOTH Low Energy driver, a flash memory driver, a serial communications driver (e.g., a Universal Serial Bus (USB) driver), a Wi-Fi driver, an audio driver, a power management driver, etc.
[0110] In some embodiments, libraries 906 provide low-level common infrastructure utilized by applications 910. Libraries 906 may include system libraries 930 (e.g., the C standard library) that may provide functions such as memory allocation functions, string manipulation functions, mathematical functions, etc. Additionally, libraries 906 may include API libraries 932 such as a media library (e.g., a library for supporting the presentation and manipulation of various media formats such as Moving Picture Experts Group-4 (MPEG4), Advanced Video Coding (H.264 or AVC), Moving Picture Experts Group Layer-3 (MP3), Advanced Audio Coding (AAC), Adaptive Multi-Rate AMR) audio codec, Joint Photographic Experts Group (JPEG or JPG), or Portable Network Graphics (PNG)), a graphics library (e.g., an OpenGL framework used for rendering graphical content on a display in two dimensions (2D) and 3D), a database library (e.g., SQLite for providing various relational database functions), a web library (e.g., WebKit for providing web browsing functions), etc. The libraries 906 may also include a wide variety of other libraries 934 to provide many other APIs to the applications 910 .
[0111] Framework 908, according to some embodiments, provides a high-level common infrastructure that can be utilized by applications 910. For example, framework 908 provides various graphic user interface (GUI) functionality, high-level resource management, high-level location services, etc. Framework 908 can provide a wide range of other APIs that can be utilized by applications 910, some of which may be specific to a particular operating system or platform.
[0112] In an exemplary embodiment, applications 910 include a wide variety of other applications, such as a home application 950, a contacts application 952, a browser application 954, a book reader application 956, a location application 958, a media application 960, a messaging application 962, a game application 964, and third-party applications 966. According to some embodiments, applications 910 are programs that perform programmatically defined functions. Various programming languages, such as object-oriented programming languages (e.g., Objective-C, Java, or C++) or procedural programming languages (e.g., C or assembly language), may be employed to create one or more of applications 910, which may be structured in various ways. In a particular example, third-party applications 966 (e.g., applications developed using the ANDROID or IOS Software Development Kit (SDK) by an entity other than the particular platform vendor) may be mobile software running on a mobile operating system, such as IOS, ANDROID, WINDOWS Phone, or another mobile operating system. In this example, the third-party application 966 can invoke API calls 912 provided by the operating system 904 to facilitate the functionality described herein.
[0113] FIG. 10 shows a schematic diagram of a machine 1000 in the form of a computer system on which a set of instructions may be executed to cause the machine to perform any one or more of the methodologies described herein, according to an exemplary embodiment. Specifically, FIG. 10 shows a schematic diagram of a machine 1000 in the exemplary form of a computer system on which instructions 1016 (e.g., software, programs, applications, applets, apps, or other executable code) may be executed to cause the machine 1000 to perform any one or more of the methodologies described herein. The instructions 1016 transform the general, unprogrammed machine 1000 into a specific machine 1000 programmed to perform the functions described and illustrated in the manner described. In alternative embodiments, the machine 1000 may operate as a standalone device or may be coupled (e.g., networked) to other machines. In a networked deployment, the machine 1000 may operate in the capacity of a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine 1000 may include, but is not limited to, a server computer, a client computer, a PC, a tablet computer, a laptop computer, a netbook, an STB, a PDA, an entertainment media system, a mobile phone, a smartphone, a mobile device, a wearable device (e.g., a smart watch), a smart home device (e.g., a smart appliance), other smart devices, a web appliance, a network router, a network switch, a network bridge, or any machine capable of executing, sequentially or otherwise, instructions 1016 that specify actions to be taken by the machine 1000. Furthermore, although only a single machine 1000 is shown, the term "machine" should also be taken to include a collection of machines 1000 that individually or together execute instructions 1016 to perform any one or more of the methods described herein.
[0114] Machine 1000 may include processor 1010, memory 1030, and I / O components 1050, which may be configured to communicate with each other via a bus 1002 or the like. In an exemplary embodiment, processor 1010 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an ASIC, a radio frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, processor 1012 and processor 1014, which may execute instructions 1016. The term "processor" is intended to include multi-core processors, which may include two or more independent processors (sometimes referred to as "cores") that may execute instructions simultaneously. While FIG. 10 shows multiple processors 1010, machine 1000 may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiple cores, or any combination thereof.
[0115] The memory 1030 may include a main memory 1032, a static memory 1034, and a storage unit 1036, all of which are accessible to the processor 1010, for example, via the bus 1002. The main memory 1032, the static memory 1034, and the storage unit 1036 store instructions 1016 that embody any one or more of the methods or functions described herein. The instructions 1016 may also reside, completely or partially, within the main memory 1032, the static memory 1034, the storage unit 1036, within at least one of the processors 1010 (e.g., within a processor's cache memory), or any suitable combination thereof during their execution by the machine 1000.
[0116] I / O components 1050 may include a wide variety of components for receiving input, providing output, generating output, transmitting information, exchanging information, capturing measurements, etc. The specific I / O components 1050 included in a particular machine will depend on the type of machine. For example, a portable machine such as a cell phone will likely include a touch input device or other such input mechanism, while a headless server machine will likely not include such a touch input device. It will be understood that I / O components 1050 may include many other components not shown in FIG. 10 . I / O components 1050 are grouped according to function solely to simplify the following description, and the grouping is in no way limiting. In various exemplary embodiments, I / O components 1050 may include output components 1052 and input components 1054. Output components 1052 may include visual components (e.g., a display such as a plasma display panel (PDP), a light-emitting diode (LED) display, a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)), audio-acoustic components (e.g., speakers), tactile components (e.g., vibration motors, resistive mechanisms), other signal generators, etc. Input components 1054 may include alphanumeric input components (e.g., a keyboard, a touchscreen configured to receive alphanumeric input, a photo-optical keyboard, or other alphanumeric input component), point-based input components (e.g., a mouse, touchpad, trackball, joystick, motion sensor, or another pointing instrument), tactile input components (e.g., physical buttons, a touchscreen that provides the location and / or force of a touch or touch gesture, or other tactile input component), audio input components (e.g., a microphone), etc.
[0117] In further exemplary embodiments, I / O components 1050 may include biometric components 1056, motion components 1058, environmental components 1060, or position components 1062, among other components. For example, biometric components 1056 may include components for detecting expressions (e.g., hand expressions, facial expressions, vocal expressions, body gestures, or eye tracking), measuring biosignals (e.g., blood pressure, heart rate, body temperature, sweat, or brain waves), identifying people (e.g., voice identification, retinal identification, face identification, fingerprint identification, or brainwave-based identification), etc. Motion components 1058 may include acceleration sensor components (e.g., accelerometers), gravity sensor components, rotation sensor components (e.g., gyroscopes), etc. The environmental components 1060 may include, for example, an illuminance sensor component (e.g., a photometer), a temperature sensor component (e.g., one or more thermometers that detect ambient temperature), a humidity sensor component, a pressure sensor component (e.g., a barometer), an acoustic sensor component (e.g., one or more microphones that detect background noise), a proximity sensor component (e.g., an infrared sensor that detects nearby objects), a gas sensor (e.g., a gas detection sensor for detecting concentrations of harmful gases for safety or for measuring pollutants in the air), or other components that may provide indications, measurements, or signals corresponding to the surrounding physical environment. The position components 1062 may include a location sensor component (e.g., a GPS receiver component), an altitude sensor component (e.g., an altimeter or barometer that detects air pressure from which altitude can be derived), an orientation sensor component (e.g., a magnetometer), etc.
[0118] Communications may be implemented using a wide variety of technologies. I / O component 1050 may include a communication component 1064 operable to couple machine 1000 to network 1080 or device 1070 via coupling 1082 and coupling 1072, respectively. For example, communication component 1064 may include a network interface component or another suitable device for interfacing with network 1080. In further examples, communication component 1064 includes a wired communication component, a wireless communication component, a cellular communication component, a near field communication (NFC) component, a Bluetooth® component (e.g., Bluetooth® low energy), a Wi-Fi® component, and other communication components for providing communication via other modalities. Device 1070 may be another machine or any of a wide variety of peripheral devices (e.g., a peripheral device coupled via USB).
[0119] Further, the communication component 1064 may detect an identifier or may include a component operable to detect an identifier. For example, the communication component 1064 may include a radio frequency identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (e.g., an optical sensor for detecting one-dimensional barcodes such as Universal Product Code (UPC) barcodes, multidimensional barcodes such as Quick Response (QR) Code, Aztec Code, Data Matrix, Dataglyph, MaxiCode, PDF417, UltraCode, UCC RSS-2D barcode, and other optical codes), or an acoustic detection component (e.g., a microphone for identifying tagged audio signals). Additionally, various information, such as location via Internet Protocol (IP) geolocation, location via Wi-Fi signal triangulation, location via detection of NFC beacon signals that may indicate a particular location, may be derived via the communication component 1064.
[0120] The various memories (i.e., 1030, 1032, 1034, and / or memory of the processor(s) 1010) and / or storage unit 1036 may store one or more sets of instructions and data structures (e.g., software) that embody or are utilized by any one or more of the methods or functions described herein. These instructions (e.g., instructions 1016), when executed by the processor(s) 1010, cause various operations to implement the disclosed embodiments.
[0121] As used herein, the terms “mechanical storage medium,” “device storage medium,” and “computer storage medium” mean the same thing and may be used interchangeably in this disclosure. These terms refer to a single or multiple storage devices and / or media (e.g., centralized or distributed databases, and / or associated caches and servers) that store executable instructions and / or data. Accordingly, these terms should be interpreted to include, but are not limited to, solid-state memory, including memory internal or external to a processor, as well as optical and magnetic media. Specific examples of mechanical storage media, computer storage media, and / or device storage media include, by way of example, semiconductor memory devices, non-volatile memory including, for example, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), FPGAs, and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The terms "mechanical storage media," "computer storage media," and "device storage media" specifically exclude carrier waves, modulated data signals, and other such media, at least some of which are covered by the term "signal media" below.
[0122] In various exemplary embodiments, one or more portions of network 1080 may be an ad-hoc network, an intranet, an extranet, a VPN, a LAN, a WLAN, a WAN, a WWAN, a MAN, the Internet, a portion of the Internet, a portion of the PSTN, a plain old telephone service (POTS) network, a cellular telephone network, a wireless network, a Wi-Fi network, another type of network, or a combination of two or more such networks. For example, network 1080 or a portion of network 1080 may include a wireless or cellular network, and coupling 1082 may be a code division multiple access (CDMA) connection, a Global System for Mobile Communications (GSM) connection, or another type of cellular or wireless coupling. In this example, the coupling 1082 may implement any of various types of data transfer technologies, such as single-carrier radio transmission technology (1xRTT), Evolution Data Optimized (EVDO) technology, General Packet Radio Service (GPRS) technology, Enhanced Data Rates for GSM Evolution (EDGE) technology, Third Generation Partnership Project (3GPP®) including 3G, Fourth Generation Wireless (4G) networks, Universal Mobile Telecommunications System (UMTS), High Speed Packet Access (HSPA), Worldwide Interoperability for Microwave Access (WiMAX), Long Term Evolution (LTE) standards, others defined by various standards-setting organizations, other long-range protocols, or other data transfer technologies.
[0123] The instructions 1016 may be transmitted or received over the network 1080 using a transmission medium via a network interface device (e.g., a network interface component included in the communications component 1064) and utilizing any one of several well-known transfer protocols (e.g., Hypertext Transfer Protocol (HTTP)). Similarly, the instructions 1016 may be transmitted or received using a transmission medium via a coupling 1072 (e.g., a peer-to-peer coupling) to the device 1070. The terms “transmission medium” and “signal medium” mean the same thing and may be used interchangeably in this disclosure. The terms “transmission medium” and “signal medium” shall be interpreted to include any intangible medium capable of storing, encoding, or carrying instructions 1016 for execution by the machine 1000, including digital or analog communications signals or other intangible media for facilitating communication of such software. Accordingly, the terms “transmission medium” and “signal medium” shall be interpreted to include any form of modulated data signal, carrier wave, etc. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information within the signal.
[0124] The terms "machine-readable medium," "computer-readable medium," and "device-readable medium" mean the same thing and may be used interchangeably in this disclosure. These terms are defined to include both mechanical storage media and transmission media. Thus, these terms include both storage devices / media and carrier wave / modulated data signals.
[0125] Although some examples, such as those illustrated in the figures, include a specific sequence of operations, the sequence may be changed without departing from the scope of the present disclosure. For example, some of the operations illustrated may be performed in parallel or in a different order without substantially affecting the functionality described in the examples. In other examples, different components of an example device or system implementing an example method may perform functions substantially simultaneously or in a specific sequence.
[0126] The various features, steps, and processes described herein may be used independently of one another or may be combined in various ways. All possible combinations and subcombinations are intended to fall within the scope of the present disclosure. Additionally, in some implementations, certain method or process blocks may be omitted.
Claims
1. 1. A method comprising: receiving, by a network site of a listing network platform, input including a plurality of images associated with individual listings; presenting options within a graphical user interface (GUI) for arranging the plurality of images according to classifications associated with the plurality of images; in response to receiving input selecting the option, animating within the GUI a subset of the plurality of images to be shuffled into one or more of the categories; presenting a template in the GUI, after the subset of the plurality of images has been animated for a specified threshold period, the template including multiple regions, each region associated with a different classification; populating a first region of the plurality of regions associated with a first classification with a first image corresponding to the first classification and populating a second region of the plurality of regions associated with a second classification with a second image corresponding to the second classification; determining that each classification of a plurality of classifications is not assigned to any of the plurality of images; identifying individual regions of the plurality of regions associated with the individual classifications; and, in response to determining that the individual classifications are not assigned to any of the plurality of images, populating the individual regions within the GUI with three-dimensional graphical elements representing the individual classifications. A method comprising:
2. analyzing the plurality of images with a machine learning model to determine a respective classification associated with each of the plurality of images. The method of claim 1 further comprising:
3. The method of claim 2 , wherein the machine learning model analyzes the plurality of images in response to receiving the input selecting the option.
4. 3. The method of claim 2, wherein the machine learning model analyzes the plurality of images as each respective image of the plurality of images is received by the input before presenting the option for pre-arranging the plurality of images within the GUI.
5. receiving additional input including an additional image associated with the listing; and, in response to receiving the additional input, re-analyzing the plurality of images previously classified by the machine learning model together with the additional image to generate or update a classification for each of the plurality of images including the additional image. The method of claim 4 further comprising:
6. the first classification includes a first room type; the second classification includes a second room type; The method of claim 1.
7. allowing a user to modify the classification associated with the plurality of images by interacting with the template presented within the GUI. The method of claim 1 further comprising:
8. 2. The method of claim 1, wherein an order in which the plurality of areas are presented in the GUI is predefined, the order including: living area, full kitchen, kitchenette, dining area, bedroom, full bathroom, half bathroom, office, dedicated workspace, backyard, patio, balcony, front yard, deck, porch, courtyard, garden, terrace, rooftop, laundry area, garage, gym, exterior, pool, hot tub, theme room, children's playroom, bowling alley, movie theater, art studio, music studio, workshop, photography studio, darkroom, woodworking room, event space, library, game room, sunroom, and wine cellar.
9. selecting one or more images from the plurality of images based on one or more criteria; and presenting the selected one or more images in a graphical element including the option for arranging the plurality of images according to classification; The method of claim 1 further comprising:
10. The method of claim 9 , wherein the one or more criteria include at least one of an order in which the images are presented in the listing or a classification of each of the images.
11. 10. The method of claim 9, wherein the subset of the plurality of images includes the selected one or more images, the method further comprising presenting a graphical element on the template that presents the animation of the subset of the plurality of images, and wherein each of the plurality of regions of the template includes a placeholder for a respective image.
12. The method of claim 11 , wherein the placeholder comprises a gray box.
13. presenting an animation including a celebratory graphic on the template including the first region having the first image and the second region having the second image. The method of claim 1 further comprising:
14. The method of claim 13 , wherein the celebratory graphic includes at least one of confetti, balloons, or a logo associated with the network site.
15. storing a database associating a plurality of categories with respective three-dimensional graphical elements representing said categories; and retrieving said three-dimensional graphical elements from said database by searching said database based on said respective categories; The method of claim 1 further comprising:
16. obtaining a list of amenities associated with each of the listings; and detecting physical spaces within each of the plurality of images. selecting an individual classification from a plurality of classifications based on the list of amenities associated with the individual listing and the detected physical space; and associating the individual classification with the individual image. The method of claim 1 further comprising:
17. determining that the list of amenities includes a first amenity and excludes a second amenity; responsive to determining that the list of amenities includes the first amenity and excludes the second amenity, selecting a first classification as the individual classification; selecting a second category as the individual category in response to determining that the list of amenities excludes the first amenity and excludes the second amenity; 17. The method of claim 16, further comprising:
18. 1. A system comprising: one or more processors of the machine; A memory for storing instructions the instructions, when executed by the one or more processors, cause the machine to: receiving, by a network site of a listing network platform, input including a plurality of images associated with each listing; presenting options within a graphical user interface (GUI) for arranging the plurality of images according to classifications associated with the plurality of images; In response to receiving input selecting the option, animating the subset of the plurality of images within the GUI to be shuffled into one or more of the categories; presenting a template in the GUI after the subset of the plurality of images has been animated for a specified threshold period, the template including a plurality of regions, each region associated with a different classification; populating a first region of the plurality of regions associated with a first classification with a first image corresponding to the first classification and populating a second region of the plurality of regions associated with a second classification with a second image corresponding to the second classification; determining that each classification of a plurality of classifications is not assigned to any of the plurality of images; identifying individual regions of the plurality of regions associated with the individual classifications; and in response to determining that the individual classifications are not assigned to any of the plurality of images, populating the individual regions within the GUI with three-dimensional graphical elements representing the individual classifications. A system that causes an operation including
19. A machine-readable storage device embodying instructions that, when executed by a machine, cause the machine to: receiving, by a network site of a listing network platform, input including a plurality of images associated with each listing; presenting options within a graphical user interface (GUI) for arranging the plurality of images according to classifications associated with the plurality of images; In response to receiving input selecting the option, animating the subset of the plurality of images within the GUI to be shuffled into one or more of the categories; presenting a template in the GUI after the subset of the plurality of images has been animated for a specified threshold period, the template including a plurality of regions, each region associated with a different classification; populating a first region of the plurality of regions associated with a first classification with a first image corresponding to the first classification and populating a second region of the plurality of regions associated with a second classification with a second image corresponding to the second classification; determining that each classification of a plurality of classifications is not assigned to any of the plurality of images; identifying individual regions of the plurality of regions associated with the individual classifications; and in response to determining that the individual classifications are not assigned to any of the plurality of images, populating the individual regions within the GUI with three-dimensional graphical elements representing the individual classifications. A machine-readable storage device that causes the device to perform operations including:
Citation Information
Patent Citations
Electronic apparatus, display controlling method, and program
JP2012138750A
Gallery application for displaying content
JP2013512508A
Classification label allocation device, classification label allocation method, and program
JP2017162025A
Apparatus, method and graphical user interface for messaging
JP2019514083A
Information processing device, information processing method, and information processing program
JP2023073148A