People-oriented building space assessment

Abstract

One embodiment of the invention sets forth a technique for evaluating a building space. The technique includes receiving a 2D input image of a building space and generating one or more cues. Each cue includes one of a plurality of human-centric criteria. The plurality of human-centric criteria can include terms such as "social," "isolated," "quiet," "distracting," "inspiring," or "boring." The technique also includes generating, via a trained machine learning model, an alignment score associated with each of the cues and the 2D input image, where the alignment score indicates a degree of alignment between the cue and the 2D input image. The technique further includes storing the 2D input image and the generated alignment scores for later retrieval and / or presentation to a user.

Description

[0001] Cross-references to related applications This application claims the benefit of U.S. Patent Application Serial No. 18 / 423,153, filed January 25, 2024, which is hereby incorporated herein by reference. Technical Field

[0002] The embodiments disclosed herein generally relate to machine learning and computer vision, and more specifically to techniques for evaluating architectural spaces. Background Technology

[0003] Users (such as architects, building planners, or interior designers) typically evaluate architectural spaces based on one or more human-centered assessment criteria. For example, an architect might consider whether a proposed entrance or lobby space in a building is "inspiring" or "boring." A building planner might want to choose "social" spaces for dining facilities rather than spaces that the building planner deems "isolated." For office spaces or other workspaces, a "quiet" environment may be more suitable than a "distracting" one. Such human-centered architectural space assessments can be useful when assigning various functions or uses to existing spaces within a building. Human-centered assessments can also be valuable when planning the renovation or alteration of existing spaces, ensuring that spaces best meet specific criteria related to their intended future use. Before construction, a human-centered assessment of 2D renderings based on the proposed floor plans can provide insights into whether various spaces are suitable for their designated purpose.

[0004] Existing machine learning techniques for human-centered building space assessment rely on training data derived from human assessments of building spaces, based on direct observation of existing building spaces or analysis of floor plans and other architectural drawings. A drawback of existing techniques is the lack of a sufficiently large training dataset that includes human assessments of building spaces. Human assessments are inherently subjective and depend on the assessor's expertise, experience, and personal preferences. This subjectivity can lead to inconsistent or conflicting assessments of one or more spaces by multiple assessors, or even to inconsistencies in a single assessor's assessment of a single space over time. Therefore, a sufficiently large amount of training data collected from multiple assessors or over a period of time may be internally inconsistent. This inconsistency complicates training machine learning models to perform human-centered building space assessments accurately and consistently.

[0005] As mentioned above, there is a need in the field for more effective techniques for performing human-centered assessments of built spaces. Summary of the Invention

[0006] In one embodiment of the invention, a method for evaluating a computer-based architectural space includes: generating a text prompt, wherein the text prompt includes human-centered evaluation criteria; and generating an alignment score based on the text prompt and a two-dimensional (2D) input image via a machine learning model, wherein the 2D input image is a visual representation of the architectural space, and the alignment score is a quantitative indicator of the accuracy of the text prompt in describing the 2D input image based on the human-centered evaluation criteria. The method further includes: assigning the human-centered evaluation criteria and the alignment score to the 2D input image; and displaying one or more of the 2D input image, the human-centered evaluation criteria, and the alignment score via a graphical user interface.

[0007] One technological advantage of the disclosed technique over existing technologies is that it provides consistent, quantifiable, and repeatable assessments of architectural spaces based on human-centered standards. The disclosed technique improves a pre-trained model by iteratively training the model based on a relatively small number of image-text pairs representing human assessments of architectural spaces. The disclosed technique can be applied to two-dimensional (2D) photographs of architectural spaces, 2D renderings of architectural spaces, or 2D still images representing individual frames in a 3D video walkthrough of one or more architectural spaces. These technological advantages provide one or more technological improvements over existing methods. Attached Figure Description

[0008] To gain a more detailed understanding of the features described above in the various embodiments, reference can be made to the various embodiments for a more specific description of the inventive concept briefly outlined above, some of which are illustrated in the accompanying drawings. However, it should be noted that the drawings only show typical embodiments of the inventive concept and should therefore not be construed as limiting the scope in any way, and that other equally effective embodiments exist.

[0009] Figure 1 A computing device configured to implement one or more aspects of various implementation schemes is shown.

[0010] Figure 2 It is based on some implementation plans. Figure 1 A more detailed illustration of the training engine.

[0011] Figure 3 It is based on some implementation plans. Figure 1 A more detailed illustration of the evaluation engine.

[0012] Figure 4 Example evaluations based on various implementation schemes are shown.

[0013] Figure 5It is a flowchart of the steps of a method for training a machine learning model to generate alignment scores for a 2D input image based on one or more human-centered evaluation criteria, according to various implementation schemes.

[0014] Figure 6 It is a flowchart of the methods and steps used to evaluate architectural space according to various implementation plans. Detailed Implementation

[0015] In the following description, numerous specific details are set forth to provide a more thorough understanding of various embodiments. However, it will be apparent to those skilled in the art that the inventive concept can be practiced without one or more of these specific details.

[0016] System Overview Figure 1 A computing device 100 is shown, configured to implement one or more aspects of various embodiments. In one embodiment, the computing device 100 includes a desktop computer, laptop computer, smartphone, personal digital assistant (PDA), tablet computer, or any other type of computing device configured to receive input, process data, and optionally display images, and is suitable for practicing one or more embodiments. The computing device 100 is configured to run a training engine 122 and an evaluation engine 124 residing in memory 116.

[0017] Note that the computing device described herein is illustrative, and any other technically feasible configuration falls within the scope of this disclosure. For example, multiple instances of training engine 122 and evaluation engine 124 may execute on a set of nodes in a distributed and / or cloud computing system to implement the functionality of computing device 100. In another example, training engine 122 and / or evaluation engine 124 may execute on various sets of hardware, different types of devices, or environments to adapt training engine 122 and / or evaluation engine 124 to different use cases or applications. In a third example, training engine 122 and evaluation engine 124 may execute on different computing devices and / or different groups of computing devices.

[0018] In one embodiment, computing device 100 includes, but is not limited to, interconnects (buses) 112 connecting one or more processors 102, input / output (I / O) device interfaces 104 coupled to one or more input / output (I / O) devices 108, memory 116, storage devices 114, and network interfaces 106. The one or more processors 102 can be any suitable processor implemented as a central processing unit (CPU), graphics processing unit (GPU), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), artificial intelligence (AI) accelerator, any other type of processing unit, or combination of different processing units, such as a CPU configured to combine GPU operation. Generally, the one or more processors 102 can be any technically feasible hardware unit capable of processing data and / or executing software applications. Furthermore, in the context of this disclosure, the computing elements shown in computing device 100 may correspond to a physical computing system (e.g., a system in a data center) or may be a virtual computing instance executing within a computing cloud.

[0019] I / O device 108 includes devices capable of providing input, such as a keyboard, mouse, touchscreen, etc., and devices capable of providing output, such as a display device. Additionally, I / O device 108 may include devices capable of both receiving input and providing output, such as a touchscreen, Universal Serial Bus (USB) port, etc. I / O device 108 can be configured to receive input from end users of computing device 100 (…). For example The computing device 100 (designer) receives various types of input and also provides various types of output to the end user of the computing device 100, such as displayed digital images or digital video or text. In some embodiments, one or more I / O devices in I / O device 108 are configured to couple computing device 100 to network 110.

[0020] Network 110 is any technically feasible type of communication network that allows the exchange of data between computing device 100 and external entities or devices, such as a web server or another networked computing device. For example, network 110 may include a wide area network (WAN), a local area network (LAN), a wireless (WiFi) network, and / or the Internet, etc.

[0021] Storage device 114 includes non-volatile storage for applications and data, and may include fixed or removable disk drives, flash memory devices, and CD-ROM, DVD-ROM, Blu-ray, HD-DVD, or other magnetic, optical, or solid-state storage devices. Training engine 122 and evaluation engine 124 may be stored in storage device 114 and loaded into memory 116 during execution.

[0022] Memory 116 includes random access memory (RAM) modules, flash memory cells, or any other type of memory cell or combinations thereof. Processor 102, I / O device interface 104, and network interface 106 are configured to read data from memory 116 and write data to memory. Memory 116 includes various software programs executable by processor 102 and application data associated with said software programs, including training engine 122 and evaluation engine 124.

[0023] In some implementations, training engine 122 trains one or more machine learning models to perform human-centered architectural space assessments. In a human-centered assessment, one or more machine learning models generate scores associated with a 2D input view based on one or more human-centered assessment criteria. Assessment engine 124 executes the machine learning models to generate one or more scores associated with the 2D input view.

[0024] More specifically, training engine 122 and evaluation engine 124 are configured to train and execute one or more machine learning models that perform human-centered architectural space evaluations based on 2D input views of the architectural space. The 2D input view may be a photograph, an architectural rendering of the space, or a 2D still image representing a single frame in a 3D video walkthrough of one or more architectural spaces.

[0025] Human-centered assessment Figure 2 It is based on some implementation plans. Figure 1 A more detailed illustration of the training engine 122 is provided. The training engine 122 generates a trained model 260 that produces alignment scores for 2D input images based on one or more human-centered evaluation metrics. As shown, the training engine 122 includes a machine learning model 200, a pre-trained model 210, training data 215, test data 220, a 2D input image 225, cues 230, alignment scores 240, and model accuracy 250. The training engine 122 trains the machine learning model 200 such that the scores generated by the machine learning model 200 accurately reflect the labels associated with the test data 220 within a predetermined accuracy threshold. After training, the training engine 122 generates the trained model 260.

[0026] The pre-trained model 210 includes a machine learning model that has been previously trained on a training dataset of labeled text-image pairs. In various implementations, the pre-trained model 210 is trained on a large dataset comprising tens or hundreds of millions of text-image pairs. For a given text input and a 2D input image, the pre-trained model 210 generates an alignment score associated with the text input and the 2D input image. The alignment score is a quantitative measure of how accurately the text input describes the content of the 2D input image. In various implementations, the generated alignment score can range from 0 to 1, where a score closer to 1 indicates a higher degree of alignment between the text input and the 2D input image, while a score closer to 0 indicates a lower degree of alignment. Before training the machine learning model 200, the training engine 122 assigns the trained parameter values ​​of the pre-trained model 210 to the machine learning model 200, such that the initial configuration of the machine learning model 200 is the same as that of the pre-trained model 210.

[0027] Training data 215 and test data 220 each include multiple text-image pairs. In various implementations, each text-image pair includes a 2D input image representing an architectural space. Architectural spaces may include, but are not limited to, spaces such as offices, entrances, meeting spaces, catering facilities, or corridors. For each text-image pair, the 2D input image may include, but is not limited to, a photograph of the architectural space, a 2D rendering of the architectural space, or a 2D still image of a single frame representing one or more architectural spaces in a 3D video walkthrough. The 2D input image may represent a specific view of the architectural space, defined by a specified location of a real or virtual camera, a specified orientation of the real or virtual camera, and a specified field of view of the real or virtual camera.

[0028] In various implementations, each text-image pair in training data 215 and test data 220 further includes a text entry associated with the 2D input image included in the text-image pair. The text entry may include one or more words or sentence fragments, or a complete sentence. The text entry associated with each text-image pair describes the 2D input image associated with the text-image pair. The text entry may include one or more human-centered evaluation criteria or attributes describing the associated 2D input image. In various implementations, the text entry may include descriptive terms such as “social,” “isolated,” “tranquil,” “distracting,” “inspiring,” or “boring.” In various implementations, correlation pairs of human-centered evaluation criteria may represent relative endpoint values ​​of a human-centered dimension. For example, a correlation pair of the descriptive terms “social” and “isolated” may represent relative endpoint values ​​of a human-centered dimension, just as a correlation pair of the terms “tranquil” and “distracting” or “inspiring” and “boring.” In various implementations, training data 215 and / or test data may include more than one instance of a particular 2D input image, wherein each instance of the 2D input image has a different associated text entry including a single descriptive term. For example, training data 215 may include two instances of a particular 2D image of an office space, one instance having an associated text entry including the phrase “This is a social space”, and a second instance of the 2D image having an associated text entry including the phrase “This is an inspiring space”.

[0029] Test data 220 includes image-text pairs not included in training data 215 and not previously used to train the pre-trained model 210. Training engine 122 trains machine learning model 200 based on training data 215 and test data 220, as described below. Training engine 122 further determines the accuracy of machine learning model 200 based on test data 220, as described below.

[0030] As discussed above, training engine 122 assigns the parameter values ​​of pre-trained model 210 to machine learning model 200, such that the initial configuration of machine learning model 200 is the same as that of pre-trained model 210. Training engine 122 retrieves a 2D input image 225 from test data 220 to be included as input to machine learning model 200. Training engine 122 further generates one or more cues 230 to be included as input to machine learning model 200. In various embodiments, training engine 122 may generate cues for each of one or more human-centered descriptive text entries included in test data 220. For example, training engine 122 may generate cues stating “This is a social space” or “This is an inspiring space.” Training engine 122 transmits the generated cues 230 and 2D input image 225 to machine learning model 200.

[0031] Machine learning model 200 generates an alignment score 240 based on cue 230 and 2D input image 225. As discussed above, in various implementations, the value of the generated alignment score 240 can be from 0 to 1, where a score closer to 1 indicates a higher degree of alignment between cue 230 and 2D input image 225, while a score closer to 0 indicates a lower degree of alignment between cue 230 and 2D input image.

[0032] Training engine 122 generates model accuracy 250 based on test data 220, cue 230, and alignment score 240. In various implementations, for text-image pairs included in test data 220, training engine 122 calculates an alignment score 240 between the descriptive terms included in the text-image pair and independent descriptive terms related to the associated human-centered dimension. For example, if the text-image pair includes the text entry “sociality,” training engine 122 uses cue 230 to calculate the alignment score 240 for the descriptive term “sociality” and the opposing descriptive term “isolation.” If the alignment score 240 of the descriptive terms included in the text-image pair is higher than the alignment score of the opposing descriptive terms, training engine 122 determines that machine learning model 200 has accurately predicted the alignment relationship between the 2D input image 225 and the cue 230. If the alignment score 240 of the descriptive terms included in the text-image pair is lower than the alignment score of the opposing descriptive terms, the training engine 122 determines that the machine learning model 200 has not accurately predicted the alignment relationship between the 2D input image 225 and the cue 230.

[0033] For each input image 225 included in the test data 220, the training engine 122 repeats the above scoring process for each generated cue 230 and records whether the machine learning model 200 accurately predicts the alignment between the cue 230 and the 2D input image 225. In various implementations, each generated cue 230 includes one of the human-centered descriptive terms “social,” “isolated,” “tranquil,” “distracting,” “inspiring,” and “boring.”

[0034] Training engine 122 analyzes each of the text-image pairs included in test data 220 and generates a set of cues 230 and associated alignment scores 240. Training engine 122 generates model accuracy 250 based on the percentage of cues 230, for which machine learning model 200 accurately predicts the alignment between cues 230 and associated 2D input images 225 according to test data 220.

[0035] If training engine 122 determines that model accuracy 250 is equal to or greater than a predetermined model accuracy threshold percentage, then training engine 122 transfers the trained parameters of machine learning model 200 to trained model 260. If training engine 122 determines that model accuracy 250 is less than a predetermined model accuracy threshold percentage, then training engine 122 trains machine learning model 200 by transferring a predetermined number of text-image pairs from training data 215 to machine learning model 200 and iteratively adjusting the parameters of machine learning model 200 based on 2D input image 225, cue 230, and alignment score 240 generated from machine learning model 200 in training data 215. After iteratively adjusting the parameters of machine learning model 200 based on the predetermined number of text-image pairs in training data 215, training engine 122 again processes test data 220 via machine learning model 200. Training engine 122 re-evaluates model accuracy 250 as described above. If training engine 122 determines that model accuracy 250 is equal to or greater than a predetermined model accuracy threshold percentage, then training engine 122 transfers the trained parameters of machine learning model 200 to trained model 260. If training engine 122 determines that model accuracy 250 is less than a predetermined model accuracy threshold percentage, then training engine 122 continues to train the machine learning model based on additional text-image pairs in training data 215, as described above.

[0036] Figure 3 It is based on some implementation plans. Figure 1A more detailed illustration of the evaluation engine 124 is provided. Evaluation engine 124 generates multiple alignment scores based on a 2D input image and multiple text prompts. As shown, training engine 122 includes a trained model 260, prompts 330, alignment scores 340, and evaluation data 350. Evaluation engine 124 receives a 2D input image 320 and generates evaluation data 350.

[0037] Evaluation engine 124 receives 2D input image 320 from a human user or an upstream software application. In various embodiments, 2D input image 320 may be a photograph of an architectural space, a 2D rendering of an architectural space, or a 2D still image representing a single frame included in a 3D video walkthrough of one or more architectural spaces.

[0038] Evaluation engine 124 generates one or more prompts 330. Each of the one or more prompts 330 includes one of a plurality of predetermined human-centered descriptive terms. In various embodiments, each generated prompt 330 includes one of the human-centered descriptive terms “social,” “isolated,” “tranquil,” “distracting,” “inspiring,” and “boring.” For example, prompt 330 may include the statement “This is an inspiring space” or “This is a social space.”

[0039] For each cue 330, the evaluation engine 124 transmits the cue 330 and the 2D input image 320 to a trained model 260. The trained model 260 calculates an alignment score 340 associated with the cue 330 and the 2D input image 320. The alignment score 340 represents the degree of alignment between the 2D input image 320 and the descriptive terms included in the associated cue 330. In various embodiments, the value of the generated alignment score 340 can be from 0 to 1, where a score closer to 1 indicates a higher degree of alignment between the cue 330 and the 2D input image 320, while a score closer to 0 indicates a lower degree of alignment between the cue 330 and the 2D input image 320.

[0040] The evaluation engine 124 transmits the 2D input image 320 and alignment scores 340 corresponding to one or more cues 330 to the evaluation data 350 for presentation to the user. The evaluation engine 124 may further record the 2D input image 320 and the associated alignment scores 340. For example The data is recorded in storage device 114.

[0041] Figure 4 Example evaluation displays according to various implementation schemes are shown. Evaluation engine 124 can generate evaluation display 400 based on data included in evaluation data 350 and / or stored in storage device 114, as referenced above. Figure 3The evaluation shows that 400 can be displayed on a graphical user interface (GUI).

[0042] The evaluation display element 410 represents a floor plan image of a portion of the building. Display element 410 includes one or more colored and / or shaded circles representing the architectural space being evaluated by the evaluation engine 124. Each circle defines a location from which a 2D image of the architectural space can be captured or rendered. Each circle includes an associated arc indicating the viewing direction of the real or virtual camera used to capture the 2D image of the architectural space.

[0043] Display element 420 represents a detailed view of one or more 2D images of an architectural space associated with a user-selected circle in display element 410 and one or more alignment scores. In various embodiments, display element 420 may include alignment scores for pairs of human-centered assessment criteria representing opposite endpoints of a human-centered dimension. For example, as shown, display element 420 includes: a relevant pair of human-centered assessment criteria, “sociality” and “isolation,” with corresponding alignment scores of 60% and 40%; and a bar chart indicating that the selected architectural space has a higher “sociality” (60%) than “isolation” (40%). In various embodiments, the displayed alignment scores may be a percentage representation of the generated alignment scores, varying from 0 to 1, as referenced above. Figure 3 As stated above.

[0044] Display element 430 represents the display of the available and user-selectable portions of the building for display on the evaluation display 400. As shown, the “2F” portion has been selected, and this portion corresponds to the part of the building shown in display element 410.

[0045] Display element 440 represents a user-selectable depiction of multiple human-centered assessment criteria. As shown, the user has selected the human-centered assessment criterion "Sociality". In various embodiments, selecting a specific human-centered assessment criterion in display element 440 can automatically adjust the coloring and / or shading of one or more circles depicted in display element 410. For example, if the user selects "Sociality" in display element 440, circles in display element 410 corresponding to architectural spaces with relatively high sociality alignment scores can be displayed with a green shading, while circles corresponding to architectural spaces with relatively low sociality alignment scores can be displayed with a red shading. In various embodiments, a specific color or shading level of the circles in display element 410 can be associated with a specific range of alignment scores for the human-centered assessment criterion selected in display element 440.

[0046] Display element 450 displays multiple 2D images and alignment scores corresponding to architectural spaces with the highest alignment scores from a human-centered evaluation criterion selected in display element 440. In various embodiments, display element 450 may display a single representative 2D image for each of the corresponding architectural spaces, and may display the corresponding alignment scores as percentage values, bar charts, or any other technically feasible depiction.

[0047] Figure 5 This is a flowchart of method steps for training a machine learning model to generate alignment scores for 2D input images based on one or more human-centered evaluation criteria, according to various implementation schemes. Although combined... Figures 1 to 3 The system describes the method steps, but those skilled in the art will understand that any system configured to perform the method steps in any order falls within the scope of this disclosure.

[0048] As shown in the figure, in operation 502 of method 500, training engine 122 receives a pre-trained image-text model. Training engine 122 assigns the parameters of the pre-trained image-text model to the machine learning model, such that the initial configuration of the machine learning model is the same as that of the pre-trained image-text model.

[0049] In operation 504, training engine 122 generates one or more cues for 2D input images included in the test dataset of image-text pairs. Each of the generated cues includes one of multiple human-centered evaluation criteria as a word, sentence fragment, or complete sentence. For example, a cue may include the phrase "This is a social space." In various implementations, the multiple human-centered evaluation criteria may include "social," "isolation," "tranquil," "distracting," "inspiring," or "boring."

[0050] In operation 506, training engine 122 generates an alignment score for each of the prompts, based on the prompts and associated 2D input images, via a machine learning model. In various implementations, each of the generated alignment scores can have a value from 0 to 1, where a score closer to 1 indicates a higher degree of alignment between the prompt and the 2D input image, and a score closer to 0 indicates a lower degree of alignment. Training engine 122 may repeat operations 504 and 506 until training engine 122 generates an alignment score associated with each prompt for all 2D images included in the test dataset.

[0051] In operation 508, training engine 122 calculates the accuracy of the machine learning model. In various implementations, the accuracy may include a percentage representation of the portion of the generated alignment score that correctly indicates the alignment relationship between the 2D input image and a cue containing human-centered criteria associated with 2D images in the test dataset.

[0052] In operation 510, training engine 122 determines whether to adjust the machine learning model. If the accuracy of the test dataset calculated according to operation 508 meets or exceeds a predetermined threshold, training engine 122 will not adjust the machine learning model, but will continue to generate a trained model in operation 514. If the accuracy of the test dataset calculated according to operation 508 does not meet or exceed the predetermined threshold, training engine 122 will continue to operation 512.

[0053] In operation 512, training engine 122 iteratively adjusts one or more parameters of the machine learning model. During each iteration, training engine 122 generates cues for the 2D input images included in the training dataset of image-text pairs and generates an alignment score associated with each of the cues and the 2D input images. Based on the generated alignment scores and the ground truth image-text data included in the training dataset, training engine 122 further adjusts one or more parameters of the machine learning model. In various implementations, training engine 122 may continue to perform this iterative training process on a predetermined number of image-text pairs in the training dataset. After training the machine learning model, training engine 122 returns to operation 504 and processes the image-text pairs in the test dataset, recalculates the accuracy of the machine learning model, and evaluates whether further adjustments to the machine learning model are needed.

[0054] Figure 6 This is a flowchart outlining the methodological steps for conducting human-centered assessments of architectural spaces, based on various implementation schemes. Despite the combination... Figures 1 to 3 The system describes the method steps, but those skilled in the art will understand that any system configured to perform the method steps in any order falls within the scope of this disclosure.

[0055] As shown in the figure, in operation 602 of method 600, evaluation engine 124 receives a 2D input image of the architectural space from a user or upstream software application. Evaluation engine 124 further receives a trained model from training engine 122. In various embodiments, the 2D input image may be a photograph, a 2D rendering, or a 2D still image of a single frame representing a 3D video walkthrough of one or more architectural spaces.

[0056] In operation 604, evaluation engine 124 generates one or more cues for the 2D input image. Each of the generated cues includes one of multiple human-centered evaluation criteria as a word, sentence fragment, or complete sentence. For example, a cue may include the phrase "This is a social space." In various implementations, the multiple human-centered evaluation criteria may include "social," "isolation," "tranquil," "distracting," "inspiring," or "boring."

[0057] In operation 606, evaluation engine 124 generates an alignment score for each of the cues, based on the cues and the associated 2D input image, via a trained model. In various implementations, each of the generated alignment scores can have a value from 0 to 1, where a score closer to 1 indicates a higher degree of alignment between the cues and the 2D input image, while a score closer to 0 indicates a lower degree of alignment between the cues and the 2D input image.

[0058] In operation 608, evaluation engine 124 associates each of the human-centered evaluation criteria and the associated alignment score with the 2D input image. In operation 612, evaluation engine 124 records the 2D input image and the associated human-centered evaluation criteria and alignment score in... For example The evaluation engine 124 can further generate one or more building spaces, associated human-centered evaluation criteria, and alignment scores for presentation to human users.

[0059] In summary, the disclosed technology trains and executes machine learning models to evaluate 2D input images of architectural spaces and generates one or more corresponding alignment scores for the 2D input images based on multiple human-centered evaluation criteria. The 2D input images can be photographs of architectural spaces, 2D renderings of architectural spaces, or 2D still images representing a single frame in a 3D video walkthrough of one or more architectural spaces.

[0060] The machine learning model initially receives parameter values ​​from a pre-trained image-text machine learning model. The model receives a 2D input image of the architectural space and a cue as input. The cue includes human-centered descriptive terms such as “boring,” “inspiring,” “social,” or “tranquil.” The machine learning model generates an alignment score, which indicates how well the 2D input image aligns with the human-centered descriptive terms included in the cue. The generated alignment score can have values ​​from 0 to 1, where a score closer to 1 indicates a higher degree of alignment between the cue and the 2D input image, while a score closer to 0 indicates a lower degree of alignment.

[0061] During training, the machine learning model is evaluated against a test dataset of image-text data pairs. For each data pair in the test set, the machine learning model generates an alignment score for each of the multiple prompts. If the generated alignment score indicates a sufficiently high alignment between the 2D image in the test dataset of image-text data pairs and the descriptive terms included in the multiple prompts, the machine learning model is stored as a trained model. If the generated alignment score does not indicate a sufficiently high alignment between the 2D image in the test dataset of image-text data pairs and the descriptive terms included in the multiple prompts, the machine learning model is iteratively trained based on the text-image pairs included in the training data test set until the generated alignment score indicates a sufficiently high alignment between the 2D image in the test dataset of image-text data pairs and the descriptive terms included in the multiple prompts.

[0062] During evaluation, the evaluation engine receives 2D input images from the user or an upstream software application. The engine generates multiple cues, each including a human-centered descriptive term such as "boring," "inspiring," "social," or "tranquil." The engine feeds the generated cues and the 2D input images to a trained model, which generates an alignment score indicating the degree of alignment between the 2D input image and the human-centered descriptive term included in the cues. The generated alignment score can have values ​​from 0 to 1, where a score closer to 1 indicates a higher degree of alignment between the cue and the 2D input image, while a score closer to 0 indicates a lower degree of alignment. The evaluation engine generates and records the alignment score associated with each generated cue and transmits the 2D input image and the associated alignment score to a display for presentation to the user.

[0063] One technological advantage of the disclosed technique over existing technologies is that it provides consistent, quantifiable, and repeatable assessments of architectural spaces based on human-centered evaluation criteria. The disclosed technique improves a pre-trained model by iteratively training the model based on a relatively small number of image-text pairs representing human assessments of architectural spaces. The disclosed technique can be applied to two-dimensional (2D) photographs of architectural spaces, 2D renderings of architectural spaces, or 2D still images representing individual frames in a 3D video walkthrough of one or more architectural spaces. These technological advantages provide one or more technological improvements over existing methods.

[0064] 1. In some embodiments, a computer-implemented method for evaluating an architectural space includes: generating a text prompt, wherein the text prompt includes a human-centered evaluation criterion; generating an alignment score based on the text prompt and a two-dimensional (2D) input image via a machine learning model, wherein the 2D input image is a visual representation of the architectural space, and the alignment score is a quantitative indicator of the accuracy of the text prompt in describing the 2D input image based on the human-centered evaluation criterion; assigning the human-centered evaluation criterion and the alignment score to the 2D input image; and displaying one or more of the 2D input image, the human-centered evaluation criterion, and the alignment score via a graphical user interface.

[0065] 2. The computer-implemented method as described in Clause 1, wherein the human-centered evaluation criteria include one of the terms “social,” “isolated,” “tranquil,” “distracting,” “inspiring,” or “boring.”

[0066] 3. The computer-implemented method as described in Clause 1 or 2, wherein the 2D input image is one of a photograph of the architectural space, a 2D rendering of the architectural space, or a still image of a single frame included in a video recording representing the architectural space.

[0067] 4. The computer-implemented method as described in any one of Clauses 1 to 3, further comprising: initializing a plurality of learnable parameters included in the machine learning model based on a pre-trained model; iteratively adjusting one or more learnable parameters among the plurality of learnable parameters based on a first plurality of alignment scores generated for a first plurality of image-text pairs included in a training dataset; calculating an accuracy associated with the machine learning model based on a second plurality of alignment scores generated for a second plurality of image-text pairs included in a test dataset; and continuing or terminating the iterative adjustment of one or more learnable parameters among the plurality of learnable parameters based on the calculated accuracy.

[0068] 5. The computer-implemented method as described in any one of Clauses 1 to 4, wherein each image-text pair included in the first plurality of image-text pairs and the second plurality of image-text pairs includes an image representing the architectural space and a human-centered realism evaluation criterion associated with the image.

[0069] 6. The computer-implemented method as described in any one of Clauses 1 to 5, further comprising displaying simultaneous visual representations of a plurality of architectural spaces and a plurality of alignment scores associated with the plurality of architectural spaces via the graphical user interface.

[0070] 7. A computer-implemented method as described in any one of Clauses 1 to 6, wherein the simultaneous visual representation of the plurality of architectural spaces is arranged on the graphical user interface based on a plurality of alignment fractions associated with the plurality of architectural spaces.

[0071] 8. In some embodiments, one or more non-transitory computer-readable media store instructions that, when executed by one or more processors, cause the one or more processors to perform the following steps: generating a text prompt, wherein the text prompt includes a human-centered evaluation criterion; generating an alignment score based on the text prompt and a two-dimensional (2D) input image via a machine learning model, wherein the 2D input image is a visual representation of an architectural space, and the alignment score is a quantitative indicator of the accuracy of the text prompt in describing the 2D input image based on the human-centered evaluation criterion; assigning the human-centered evaluation criterion and the alignment score to the 2D input image; and displaying one or more of the 2D input image, the human-centered evaluation criterion, and the alignment score via a graphical user interface.

[0072] 9. One or more non-transitory computer-readable media as described in Clause 8, wherein the human-centered assessment criteria include one of the terms “social,” “isolated,” “quiet,” “distracting,” “inspiring,” or “boring.”

[0073] 10. One or more non-transitory computer-readable media as described in Clause 8 or 9, wherein the 2D input image is one of a photograph of an architectural space, a 2D rendering of an architectural space, or a still image of a single frame included in a video recording representing an architectural space.

[0074] 11. One or more non-transitory computer-readable media as described in any one of Clauses 8 to 10, wherein these instructions further cause the one or more processors to perform the following steps: initializing a plurality of learnable parameters included in the machine learning model based on a pre-trained model; iteratively adjusting one or more of the plurality of learnable parameters based on a first plurality of alignment scores generated for a first plurality of image-text pairs included in a training dataset; calculating an accuracy associated with the machine learning model based on a second plurality of alignment scores generated for a second plurality of image-text pairs included in a test dataset; and continuing or terminating the iterative adjustment of one or more of the plurality of learnable parameters based on the calculated accuracy.

[0075] 12. One or more non-transitory computer-readable media as described in any one of Clauses 8 to 11, wherein each image-text pair included in the first plurality of image-text pairs and the second plurality of image-text pairs includes an image representing an architectural space and a human-centered reality assessment criterion associated with the image.

[0076] 13. One or more non-transitory computer-readable media as described in any one of Clauses 8 to 12, wherein the instructions further cause the one or more processors to perform the following steps: displaying simultaneous visual representations of a plurality of architectural spaces and a plurality of alignment fractions associated with the plurality of architectural spaces via the graphical user interface.

[0077] 14. One or more non-transitory computer-readable media as described in any one of Clauses 8 to 13, wherein the simultaneous visual representation of the plurality of architectural spaces is arranged on the graphical user interface based on a plurality of alignment fractions associated with the plurality of architectural spaces.

[0078] 15. In some embodiments, a system includes: one or more memories storing instructions; and one or more processors configured to execute the instructions to: generate a text prompt, wherein the text prompt includes a human-centered evaluation criterion; generate an alignment score based on the text prompt and a two-dimensional (2D) input image via a machine learning model, wherein the 2D input image is a visual representation of an architectural space, and the alignment score is a quantitative measure of the accuracy of the text prompt in describing the 2D input image based on the human-centered evaluation criterion; assign the human-centered evaluation criterion and the alignment score to the 2D input image; and display one or more of the 2D input image, the human-centered evaluation criterion, and the alignment score via a graphical user interface.

[0079] 16. The system as described in Clause 15, wherein the human-centered assessment criteria include one of the terms “social,” “isolated,” “quiet,” “distracting,” “inspiring,” or “boring.”

[0080] 17. The system as described in Clause 15 or 16, wherein the 2D input image is one of a photograph of the architectural space, a 2D rendering of the architectural space, or a still image of a single frame included in a video recording representing the architectural space.

[0081] 18. The system of any one of Clauses 15 to 17, wherein these instructions further cause the one or more processors to: initialize a plurality of learnable parameters included in the machine learning model based on a pre-trained model; iteratively adjust one or more of the plurality of learnable parameters based on a first plurality of alignment scores generated for a first plurality of image-text pairs included in a training dataset; calculate an accuracy associated with the machine learning model based on a second plurality of alignment scores generated for a second plurality of image-text pairs included in a test dataset; and continue or terminate the iterative adjustment of one or more of the plurality of learnable parameters based on the calculated accuracy.

[0082] 19. The system as described in any one of Clauses 15 to 18, wherein each image-text pair included in the first plurality of image-text pairs and the second plurality of image-text pairs includes an image representing the architectural space and a human-centered reality assessment criterion associated with the image.

[0083] 20. The system of any one of Clauses 15 to 19, wherein the instructions further cause the one or more processors to display simultaneous visual representations of a plurality of architectural spaces and a plurality of alignment scores associated with the plurality of architectural spaces via the graphical user interface.

[0084] Any and all combinations of any element of any claim and / or any element described in this application, in any manner, fall within the scope of this invention and protection.

[0085] Various embodiments have been described for illustrative purposes, but these descriptions are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments.

[0086] Various aspects of embodiments of this invention may be embodied as systems, methods, or computer program products. Therefore, aspects of this disclosure may take the form of entirely hardware implementations, entirely software implementations (including firmware, resident software, microcode, etc.), or implementations combining software and hardware aspects, all of which may be generally referred to herein as “modules,” “systems,” or “computers.” Furthermore, any hardware and / or software techniques, processes, functions, components, engines, modules, or systems described in this disclosure may be implemented as circuits or sets of circuits. Additionally, aspects of this disclosure may take the form of computer program products embodied on one or more computer-readable media having computer-readable program code embodied thereon.

[0087] Any combination of one or more computer-readable media may be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or apparatuses, or any suitable combination thereof. More specific examples (not an exhaustive list) of computer-readable storage media will include: electrical connections having one or more wires, portable computer floppy disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable optical disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the context of this document, a computer-readable storage medium can be any tangible medium that can contain or store programs for use with or in connection with an instruction execution system, device, or apparatus.

[0088] The foregoing description of aspects of this disclosure is based on flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine. When executed via a processor of a computer or other programmable data processing apparatus, the instructions enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such processors can be, but are not limited to, general-purpose processors, special-purpose processors, application-specific processors, or field-programmable gate arrays.

[0089] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each box in a flowchart or block diagram may represent a module, segment, or portion of code comprising one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the boxes may not occur in the order shown in the drawings. For example, two boxes shown consecutively may actually be executed substantially simultaneously, or the boxes may sometimes be executed in reverse order, depending on the functionality involved. It should also be noted that each box in the block diagram and / or flowchart illustrations, and combinations of boxes in the block diagram and / or flowchart illustrations, may be implemented by a system based on special-purpose hardware or a combination of special-purpose hardware and computer instructions that performs the specified function or action.

[0090] While the foregoing describes embodiments of this disclosure, other and additional embodiments of this disclosure may be designed without departing from the basic scope of this disclosure, the scope of which is defined by the appended claims.

Claims

1. A computer-implemented method for evaluating architectural space, the computer-implemented method comprising: Generate text prompts, wherein the text prompts include human-centered evaluation criteria; An alignment score is generated based on the text prompt and a two-dimensional (2D) input image via a machine learning model, wherein the 2D input image is a visual representation of the architectural space, and the alignment score is a quantitative indicator of the accuracy of the text prompt in describing the 2D input image based on the human-centered evaluation criteria. Assign the human-centered evaluation criteria and the alignment score to the 2D input image; and The 2D input image, the human-centered evaluation criteria, and one or more of the alignment scores are displayed via a graphical user interface.

2. The computer-implemented method of claim 1, wherein the human-centered evaluation criteria include one of the terms "social," "isolated," "tranquil," "distracting," "inspiring," or "boring." 3. The computer-implemented method of claim 1, wherein the 2D input image is one of a photograph of the architectural space, a 2D rendering of the architectural space, or a still image of a single frame included in a video recording representing the architectural space.

4. The computer-implemented method as described in claim 1, further comprising: The machine learning model is initialized based on the pre-trained model, which includes multiple learnable parameters. Based on the first plurality of alignment scores generated for the first plurality of image-text pairs included in the training dataset, one or more learnable parameters among the plurality of learnable parameters are iteratively adjusted; The accuracy associated with the machine learning model is calculated based on the second plurality of alignment scores generated for the second plurality of image-text pairs included in the test dataset; as well as Based on the calculated accuracy, the iterative adjustment of one or more of the learnable parameters can be continued or terminated.

5. The computer-implemented method of claim 4, wherein each image-text pair included in the first plurality of image-text pairs and the second plurality of image-text pairs includes an image representing the architectural space and a human-centered realism evaluation criterion associated with the image.

6. The computer-implemented method of claim 1, further comprising displaying simultaneous visual representations of a plurality of architectural spaces and a plurality of alignment scores associated with the plurality of architectural spaces via the graphical user interface.

7. The computer-implemented method of claim 6, wherein the simultaneous visual representation of the plurality of building spaces is arranged on the graphical user interface based on the plurality of alignment fractions associated with the plurality of building spaces.

8. A non-transitory computer-readable medium storing one or more instructions, which, when executed by one or more processors, cause the one or more processors to perform the following steps: Generate text prompts, wherein the text prompts include human-centered evaluation criteria; An alignment score is generated based on the text prompt and a two-dimensional (2D) input image via a machine learning model, wherein the 2D input image is a visual representation of the architectural space, and the alignment score is a quantitative indicator of the accuracy of the text prompt in describing the 2D input image based on the human-centered evaluation criteria. Assign the human-centered evaluation criteria and the alignment score to the 2D input image; and The 2D input image, the human-centered evaluation criteria, and one or more of the alignment scores are displayed via a graphical user interface.

9. One or more non-transitory computer-readable media as claimed in claim 8, wherein the human-centered assessment criteria include one of the terms "social," "isolated," "quiet," "distracting," "inspiring," or "boring." 10. The one or more non-transitory computer-readable media of claim 8, wherein the 2D input image is one of a photograph of an architectural space, a 2D rendering of an architectural space, or a still image of a single frame included in a video recording representing an architectural space.

11. The one or more non-transitory computer-readable media of claim 8, wherein the instructions further cause the one or more processors to perform the following steps: The machine learning model is initialized based on the pre-trained model, which includes multiple learnable parameters. Based on the first plurality of alignment scores generated for the first plurality of image-text pairs included in the training dataset, one or more learnable parameters among the plurality of learnable parameters are iteratively adjusted; The accuracy associated with the machine learning model is calculated based on the second plurality of alignment scores generated for the second plurality of image-text pairs included in the test dataset; as well as Based on the calculated accuracy, the iterative adjustment of one or more of the learnable parameters can be continued or terminated.

12. The one or more non-transitory computer-readable media of claim 11, wherein each image-text pair included in the first plurality of image-text pairs and the second plurality of image-text pairs includes an image representing an architectural space and a human-centered reality assessment criterion associated with the image.

13. The one or more non-transitory computer-readable media of claim 8, wherein the instructions further cause the one or more processors to perform the following steps: The graphical user interface displays simultaneous visual representations of multiple architectural spaces and multiple alignment scores associated with the multiple architectural spaces.

14. One or more non-transitory computer-readable media as claimed in claim 13, wherein the simultaneous visual representation of the plurality of architectural spaces is arranged on the graphical user interface based on the plurality of alignment fractions associated with the plurality of architectural spaces.

15. A system comprising: One or more memories, wherein the one or more memories store instructions; as well as One or more processors, the one or more processors being configured to execute the instructions to: Generate text prompts, wherein the text prompts include human-centered evaluation criteria; An alignment score is generated based on the text prompt and a two-dimensional (2D) input image via a machine learning model, wherein the 2D input image is a visual representation of the architectural space, and the alignment score is a quantitative indicator of the accuracy of the text prompt in describing the 2D input image based on the human-centered evaluation criteria. Assign the human-centered evaluation criteria and the alignment score to the 2D input image; and The 2D input image, the human-centered evaluation criteria, and one or more of the alignment scores are displayed via a graphical user interface.

16. The system of claim 15, wherein the human-centered evaluation criteria include one of the terms "social," "isolated," "tranquil," "distracting," "inspiring," or "boring." 17. The system of claim 15, wherein the 2D input image is one of a photograph of the architectural space, a 2D rendering of the architectural space, or a still image of a single frame included in a video recording representing the architectural space.

18. The system of claim 15, wherein the instructions further cause the one or more processors to: The machine learning model is initialized based on the pre-trained model, which includes multiple learnable parameters. Based on the first plurality of alignment scores generated for the first plurality of image-text pairs included in the training dataset, one or more learnable parameters among the plurality of learnable parameters are iteratively adjusted; The accuracy associated with the machine learning model is calculated based on the second plurality of alignment scores generated for the second plurality of image-text pairs included in the test dataset; as well as Based on the calculated accuracy, the iterative adjustment of one or more of the learnable parameters can be continued or terminated.

19. The system of claim 18, wherein each image-text pair included in the first plurality of image-text pairs and the second plurality of image-text pairs includes an image representing the architectural space and a human-centered reality assessment criterion associated with the image.

20. The system of claim 15, wherein the instructions further cause the one or more processors to display simultaneous visual representations of a plurality of architectural spaces and a plurality of alignment scores associated with the plurality of architectural spaces via the graphical user interface.

Images