Privacy enhanced image for lighting design

Abstract

A computer-implemented method of generating a composite image of a space for an illumination design includes obtaining an input image of the space and performing a parsing of the input image of the space to detect an object in the input image. The method further includes classifying an object detected in the input image based at least on a correlation with a lighting design of the space and a correlation with privacy. The method further includes generating a composite image of the space from the input image of the space. A first object among the objects in the input image is included in the composite image, and a second object among the objects in the input image is excluded from the composite image. The first object is associated with a lighting design and the second object is associated with privacy.

Description

Technical Field

[0001] The present disclosure relates generally to lighting solutions, and more particularly to privacy enhancing images for use in lighting design. Background Art

[0002] Some lighting systems may be designed to provide personalized lighting. Some lighting systems may also be configured to provide different lighting scenes. For example, lighting design may be performed based on the type of room, regular activities in the room, special events that may be held in the room, and the like. In some cases, it may be desirable to obtain personalized lighting recommendations, for example, from a lighting professional. Consumers may also want lighting professionals to perform lighting design for a space, such as a room or an entire home. Lighting professionals may be able to remotely perform lighting design for a space based on one or more images of the space provided by the consumer. However, due to privacy concerns, consumers may be reluctant to provide lighting professionals with images of spaces, such as living rooms, bedrooms, kitchens, and the like. Therefore, a solution may be desired that reduces consumers' privacy concerns about sharing images of spaces with remote lighting design professionals or with other people. Summary of the invention

[0003] The present disclosure relates generally to lighting solutions, and more particularly to privacy-enhanced images for use in lighting design. In an example embodiment, a computer-implemented method for generating a composite image of a space for lighting design includes obtaining an input image of the space, and performing parsing of the input image of the space to detect objects in the input image. The method further includes classifying objects detected in the input image based at least on relevance to the lighting design of the space and relevance to privacy. The method also includes generating a composite image of the space from the input image of the space. A first object of the objects in the input image is included in the composite image, and a second object of the objects in the input image is excluded from the composite image. The first object is related to the lighting design, and the second object is related to privacy.

[0004] In another example embodiment, a device for generating a composite image of a space for lighting design includes a processor configured to obtain an input image of the space and perform parsing of the input image of the space to detect objects in the input image. The processor is further configured to classify objects in the input image of the space based at least on relevance to the lighting design of the space and relevance to privacy. The processor is also configured to generate a composite image of the space from the input image of the space. A first object of the objects in the input image is included in the composite image, and a second object of the objects in the input image is excluded from the composite image. The first object is related to the lighting design, and the second object is related to privacy.

[0005] These and other aspects, objects, features and embodiments will be apparent from the following description and appended claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Reference will now be made to the accompanying drawings, which are not necessarily drawn to scale, and in which:

[0007] Figure 1 A functional system for generating a composite image of a space according to an example embodiment is shown;

[0008] Figure 2 shows a room intended for lighting design using a synthetic image according to an example embodiment;

[0009] Figure 3 A functional system for generating a composite image of a space according to another example embodiment is shown;

[0010] Figure 4 A functional system for generating a composite image of a space according to another example embodiment is shown;

[0011] Figure 5 An apparatus for generating a composite image of a space for lighting design according to an example embodiment is shown; and

[0012] Figure 6 A method of generating a composite image of a space for lighting design according to an example embodiment is shown.

[0013] The accompanying drawings illustrate only example embodiments and therefore should not be considered to limit the scope. The elements and features shown in the drawings are not necessarily to scale, instead emphasis is placed on clearly illustrating the principles of the example embodiments. Additionally, certain dimensions or positions may be exaggerated to help convey these principles visually. In the accompanying drawings, the same reference numerals used in different drawings may represent similar or corresponding, but not necessarily identical, elements. DETAILED DESCRIPTION

[0014] In the following paragraphs, example embodiments will be described in more detail with reference to the accompanying drawings. In the description, well-known components, methods and / or processing techniques are omitted or briefly described. In addition, mentioning various (multiple) features of the embodiment does not mean that all embodiments must include the mentioned (multiple) features.

[0015] Figure 1 A functional system 100 for generating a composite image 110 of a space is shown according to an example embodiment, and Figure 2A room 200 is shown that is intended for lighting design using a synthetic image 110, according to an example embodiment. The synthetic image 110 may be an image generated by the system 100 based on the input image 108, wherein some objects in the input image 108 may be included in the synthetic image 110, wherein some objects in the input image 108 may be excluded from the synthetic image 110, and wherein the synthetic image 110 includes some objects that are not in the input image 108. The synthetic image 110 is intended to provide a level of privacy protection to the owner / occupant of the room 200 shown in the input image 110. The synthetic image 110 may be a completely artificially generated image that looks realistic like a real image, or the synthetic image 110 may be an image that enhances aspects taken from a real image of a user's space (e.g., the room 200) and enhances them with other artificially generated aspects. As shown in FIG. Figure 1 As shown, in some example embodiments, system 100 includes image parsing module 102 and image generator module 104. System 100 may also include image verification module 106. Image parsing module 102 may receive or otherwise obtain input image 108 and parse the image to detect objects in input image 108. For example, input image 108 may be an image of room 200.

[0016] In some example embodiments, the room 200 may include a desk 202, a chair 204 near the desk 202, a sofa 206, tables 208, 210, a TV, a gaming computer, etc. The room 200 may have windows 212, 214 and a door 216. The windows 212, 214 and the door 216 may allow light to enter the room 200. The room 200 may also include a freestanding lamp 218, lamps 220, 222, 224 (e.g., hanging lamps), and a table lamp 228 on the desk 202. A mirror 236 may be attached to a wall of the room 200.

[0017] In some example embodiments, laptop 226 and / or other electronic devices may be on desk 202. Framed artwork 230 may be on the wall of room 200, and ball 232, stack of books 238, and other miscellaneous objects may be on the floor of room 200. Jewelry box 234 and health-related objects such as syringe 244 and medicine box 246 may be on table 208. Objects such as candle 240 and spoon 242 may be on table 210.

[0018] In some example embodiments, the image parsing module 102 may parse the input image 108 to determine the type of room 200 and / or the design style of the room 200. For example, the image parsing module 102 may include one or more neural networks that are trained to classify the rooms included in the image. For illustration, the image parsing module 102 may include a convolutional neural network (CNN) that is trained to classify the room included in the input image 108 as a bedroom, a living room, a home office, a dining room, a bathroom, a kitchen, a walk-in closet, a game room, a patio, a recreation room, a study, a mixed-use room, or another type of room that can be easily thought of by a person of ordinary skill in the art. For example, information related to room type / scene classification methods is described in Zhou, Bolei & Lapedriza, & Xiao, Jianxiong & Torralba, Antonio & Oliva, Aude. (2015), “Learning Deep Features for Scene Recognition using Places Database”. The image parsing module 102 may classify the room based on specific furniture present in the room as captured in the input image 108. In some example embodiments, the image parsing module 102 may also perform a finer classification of the room included in the input image 108 by, for example, classifying the room as a boy's bedroom, a girl's bedroom, a romantic bedroom, etc.

[0019] In some example embodiments, the image parsing module 102 may determine that the room 200 shown in the input image 108 is a home office or a home office-recreational mixed use room. The image parsing module 102 may provide a classification output indicating the type of the room 200 to other components of the image parsing module 102 and / or to other components of the functional system 100, and may save the room type information in a memory device, for example, for subsequent transmission and / or for display.

[0020] In some example embodiments, the image parsing module 102 may determine the design style of the room 200 from the input image 108. For example, the image parsing module 102 may include one or more neural networks that are trained to classify the design style of the room included in the image. For illustration, the image parsing module 102 may include a CNN (or a Transformer or other type of (one or more) neural networks) that is trained to classify the design style of the room included in the input image 108 as modern, casual, classic, natural, or another design style that can be easily thought of by a person of ordinary skill in the art. For example, the image parsing module 102 may classify and determine the design style of the room based on the specific types of furniture, artwork, and other objects present in the room as captured in the input image 108. In some example embodiments, the image parsing module 102 may also perform a more refined classification of the design style of the room included in the input image 108. For example, the image parsing module 102 may classify the design style of the room as colonial, Victorian, Bohemian, contemporary, coastal, country, etc.

[0021] In some example embodiments, the image parsing module 102 may determine that the design style of the room 200 as shown in the input image 108 is, for example, a casual style. The image parsing module 102 may provide a classification output indicating the design style of the room 200 to other components of the image parsing module 102 and / or to other components of the functional system 100, and may save the design style information in a memory device, for example, for subsequent transmission and / or display.

[0022] In some example embodiments, the image parsing module 102 may parse the input image 108 to detect and classify objects in the input image 108. For example, the image parsing module 102 may include one or more CNN components that are trained to detect and classify objects in the input image 108, as can be readily appreciated by a person of ordinary skill in the art. For example, the one or more CNN components of the image parsing module 102 may be based on You Only Look Once (YOLO), Single Shot Multibox Detector (SSD), Region-based CNN (R-CNN), Fast R-CNN, Faster R-CNN, Region-based Fully Convolutional Network (R-FCN), Mask R-CNN, or another neural network architecture suitable for object detection that can be readily appreciated by a person of ordinary skill in the art.

[0023] In some example embodiments, the image parsing module 102 may detect objects in the room 200 included in the input image 108. For example, the image parsing module 102 may detect the desk 202, the chair 204, the sofa 206, the tables 208, 210. The image parsing module 102 may also detect the windows 212, 214, and the door 216. The image parsing module 102 may also detect a freestanding lamp 218, lamps 220, 222, 224, a table lamp 228, and a mirror 236. The image parsing module 102 may also detect a laptop 226, a framed artwork 230, a ball 232, a stack of books 238, and other miscellaneous objects on the floor of the room 200. The image parsing module 102 may also detect a jewelry box 234, a candle 240, a spoon 242, a syringe 244, and a medicine box 246. The image parsing module 102 may also detect the walls, floor, and ceiling of the room 200 from the input image 108.

[0024] In some example embodiments, the parsing module 102 may also determine the relative location and / or position of the detected object in the input image 108. For example, one or more CNN components of the image parsing module 102 may include a Faster R-CNN and / or Mask R-CNN component that can determine the location of the object (e.g., the location of the bounding box of the object or the pixel location corresponding to the object). The image parsing module 102 may determine the relative location of the detected object based on the location of the object. The location information of the detected object may include, but is not limited to, the location of the object within the room 200, the distance to one or more other objects within the room 200, the coordinates of the object within the room 200 and / or within the input image 108, and / or the area of ​​the room 200 where the object is located within the room 200. The location information of the detected object may also include, but is not limited to, the orientation of the first object (e.g., a TV in the center of the room 200) relative to the second object (e.g., a sofa) or the third object (e.g., a dining table) within the input image 108 and / or the area of ​​the room 200 where the object is located within the room 200.

[0025] In some example embodiments, light detection and ranging (Lidar) information, which may be expressed in Cartesian coordinates (i.e., x, y, z coordinates) relative to a reference position, may be embedded in the input image 108 or provided separately to the image parsing module 102 along with the input image 108. For example, the Lidar information may be obtained by a camera device simultaneously with the input image 108. The image parsing module 102 may use the Lidar information in conjunction with the pixel locations in the input image 108 to determine the locations of objects in the room 200 as captured in the input image 108, as may be readily appreciated by one of ordinary skill in the art having the benefit of this disclosure.

[0026] In some example embodiments, the image parsing module 102 may classify detected objects based on relevance to desired design aspects (e.g., lighting design, office design, furniture design, decorated rooms) and relevance to privacy. For example, the image parsing module 102 may classify detected objects based on relevance to lighting design and relevance to privacy. Relevance to lighting includes lighting design objects and objects that affect the lighting design of a space. For example, lighting design objects may include objects that provide light, reflect light, shadow light, block light, or affect light distribution in some way within a space above one or more areas of the space (e.g., corners, walls, ceilings, floors). In general, one or more CNN components of the image parsing module 102 may be pre-trained to classify objects based on various requirements / constraints, as can be readily understood by those of ordinary skill in the art with the benefit of the present disclosure. For example, one or more CNN components of the image parsing module 102 may be pre-trained to classify lamps, windows, light-transmitting objects, and reflective objects as being related to lighting design and as corresponding to lighting design objects. In some example embodiments, one or more CNN components of the image parsing module 102 may be pre-trained to classify wall decorations, furniture, structures, structure positioning (e.g., wall positioning, ceiling height), lamps, and selected objects as being related to a desired design aspect or room layout or as lighting design objects. For illustration, the image parsing module 102 may classify the freestanding lamp 218, the lamps 220, 222, 224, the table lamp 228, the window 212, 214, the door 216, and the mirror 236 as being related to the lighting design of the room 200 and being or corresponding to the lighting design objects of the space. Although the image parsing module 102 is described herein for example embodiments as classifying objects as being related to lighting design, it should be appreciated that the image parsing module 102 may classify any type of object of the input image 108 and be related to any selected or desired filter and / or design. For example, the image parsing module 102 may be configured to classify objects of the input image 108 based in part on provided filters or design aspects provided by a user, an administrator, and / or parameters of the image parsing module 102. Filters and / or design aspects may include, but are not limited to, privacy issues, room architecture, room design, interior decoration, room construction, and / or environmental design.

[0027] In some example embodiments, one or more CNN components of the image parsing module 102 may also be pre-trained to classify some objects as being related to lighting design by default. For example, the image parsing module 102 may classify relatively large pieces of furniture (e.g., desks, chairs, sofas, tables, TVs, gaming computers) as being related to lighting design and as being or corresponding to lighting design objects. For illustration, the image parsing module 102 may classify the desk 202, chair 204, sofa 206, and tables 208, 210 as being related to the lighting design of the room 200 and as lighting design objects by default. The association with lighting design and lighting design objects may include or relate to, but is not limited to, how the objects affect the lighting within the corresponding room 200. For example, in some embodiments, correlations with lighting design can include the effects of lighting provided by light fixtures (e.g., 220, 222, 224) within the room 200, the effects of light reflected from or due to corresponding objects or surfaces of corresponding objects, the effects of the positioning of one or more windows (e.g., 212, 214) that can allow light (e.g., ambient light, light from outside, light from another room) to enter the room 200, the effects of the positioning of one or more doors (e.g., door 216) that can allow light to enter the room 200, and / or the effects of objects such as mirrors (e.g., mirror 236) that can reflect or provide light to one or more areas of the room 200.

[0028] In some example embodiments, the image parsing module 102 may determine the relevance of furniture and other objects to lighting design and lighting design objects based on the type of room 200 as determined by the image parsing module 102 as described above. For example, if the image parsing module 102 determines that the room 200 is a home office, the image parsing module 102 may classify the desk 202 and the chair 204 as being related to lighting design (e.g., lighting design objects), and may classify the sofa 206 as not being related to lighting design (e.g., lighting design objects). As another example, if the image parsing module 102 determines that the room 200 is a home office-lounge hybrid room, the image parsing module 102 may classify the desk 202 and the chair 204 and the sofa 206 as being related to lighting design, and classify the book stack 238 as being related to lighting design because the book stack 238 is close to the sofa 206.

[0029] In some example embodiments, image parsing module 102 may determine whether an object is relevant to the lighting design based on user input provided to image parsing module 102 indicating desired lighting for room 200. For example, if the user input indicates computer-related task lighting, image parsing module 102 may determine that sofa 206 is not relevant to the lighting design, and may determine that desk 202, chair 204, and laptop 226 are relevant to the lighting design for room 200.

[0030] As described above, the image parsing module 102 can classify objects based on the relevance to privacy (e.g., privacy settings). The relevance to privacy used herein may include or refer to privacy settings and / or objects that meet or do not meet one or more privacy settings. For illustration, the image parsing module 102 may include one or more CNN components that are pre-trained to classify objects as being related to privacy (i.e., privacy-sensitive, privacy settings). Privacy settings may include, but are not limited to, default privacy settings, user-provided privacy settings, administrator privacy settings, and / or privacy settings corresponding to specific settings of a room (e.g., work, office-related settings, building-related settings). For illustration, the image parsing module 102 may classify health-related objects as being related to privacy by default. For example, the image parsing module 102 may classify syringes 244 and medicine boxes 246 as privacy-sensitive objects (e.g., meeting one or more privacy settings). The image parsing module 102 may also classify jewelry and related objects as being related to privacy by default. For example, the image parsing module 102 may classify jewelry boxes 234 as privacy-sensitive objects. The image parsing module 102 may also classify the first photo frame showing a person as a privacy-sensitive object, while it may classify the second photo frame showing a sunset scenery as a non-privacy-sensitive object, even though the sunset scenery is related to lighting design.

[0031] In some example embodiments, the image parsing module 102 may classify one or more objects as privacy-related and / or satisfying one or more privacy settings based on the location of the objects. To illustrate, one or more CNN components of the image parsing module 102 may be pre-trained to classify objects as privacy-sensitive based on the relative location of a combination of objects. For example, the image parsing module 102 may classify the candle 240 and / or the spoon 242 as privacy-sensitive based on the proximity of the objects to each other, even though each individual object itself may not be considered a privacy-sensitive object. In some embodiments, the privacy settings may indicate or include a distance metric to identify an object as privacy-sensitive or privacy-insensitive. The distance metric may vary based on at least the type of space and / or the objects involved.

[0032] In some example embodiments, the image parsing module 102 may determine that one or more objects in the room 200 are related to both lighting design and privacy. For example, the image parsing module 102 may determine that some artworks are related to both lighting design and privacy. To illustrate, the artwork 230 may be considered to be related to lighting design because the lighting may affect the aesthetic appearance of the artwork 230, and the artwork 230 may be considered to be privacy sensitive because of the high economic value that may be associated. As another example, the mirror 236 may be related to both lighting design and privacy. For example, the mirror 236 may be considered to be privacy sensitive because, for example, objects outside the room 200 or the camera's field of view may appear in the mirror 236 and may be captured in the input image 108. In this case, the image parsing module 102 may classify the object as being related to lighting design or being related to privacy (i.e., privacy sensitive) based on, for example, user input indicating the user's privacy sensitivity. Alternatively or in addition, the image parsing module 102 may request user input indicating whether a particular object is privacy sensitive. For example, a user may be requested (eg, via a display interface) to provide user input indicating whether mirror 236 should be classified as privacy sensitive, and image parsing module 102 may classify the object as being related to lighting design or privacy based on the user input.

[0033] In some example embodiments, the image parsing module 102 may include one or more CNN components that are pre-trained to extract features from the input image 108 and determine the main activity area in the room 200. The image parsing module 102 may determine whether the object is related to the lighting design based on the relative position of the main activity area and the object. For example, the image parsing module 102 may classify the mirror 236 as not related to the lighting design of the room 200 based on the relative distance of the mirror 236 from the desk 202, which may determine the desk 202 as the main activity area in the room 200. Alternatively, the mirror 236 may be classified as related to the light design considering privacy, so that when the composite image 110 is generated by the image generation module 104, the reflection in the mirror 236 is excluded.

[0034] In some example embodiments, image parsing module 102 may classify various objects, such as ball 232, as not relevant to lighting design. For example, one or more CNN components of image parsing module 102 may be pre-trained such that image parsing module 102 classifies some objects, such as ball 232, as not relevant to lighting design based on classification of room 200 as a home office.

[0035] In some example embodiments, the image parsing module 102 may perform multiple groups (e.g., two groups, more than two groups) of classifications based on different privacy level requirements by default or based on user input. Each of the classifications of the different groups may include or correspond to different privacy levels. For example, based on relatively strict privacy requirements or a first privacy level, the image parsing module 102 may classify electronic devices (e.g., laptop 226), artworks (e.g., artworks 230), health-related objects (e.g., syringes and medicine boxes 246), jewelry and related objects (e.g., jewelry boxes 234), and some combinations of objects (e.g., candles 240 and spoons 242) as privacy-related (i.e., privacy-sensitive). Based on less strict (i.e., relatively loose) privacy requirements or a second privacy level (e.g., lower than the first privacy level), the image parsing module 102 may not classify some objects as privacy-related. For example, the image parsing module 102 may not classify laptops 226, medicine boxes 234, and artworks 230 as privacy-sensitive objects. Different privacy levels may provide dynamic control options to the end user to generate composite images of room 200 for different scenarios (eg, providing to a third party, sharing with friends, sharing with work).

[0036] In some example embodiments, the image parsing module 102 may output a mask of an object that is classified as privacy-related. For example, the pre-trained Mask R-CNN component of the image parsing module 102 may classify some objects in the room 200 as privacy-related as described above, and may generate one or more masks of a specific object. For example, the mask of the object may be a binary mask in which pixels corresponding to the object or a bounding box around the object have a specific value such that the object or bounding box is obscured by the mask (e.g., all white), as can be readily understood by a person of ordinary skill in the art. The one or more masks may also include one or more objects that are not related to lighting design or privacy (e.g., ball 232). The one or more masks generated by the image parsing module 102 may be used by other components of the image parsing module 102, the image generation module 104, and / or other components of the system 100. For example, the image generation module 104 of the system 100 may use one or more masks of a specific object to generate a synthetic image 110 that does not include the specific object corresponding to the mask.

[0037] In some example embodiments, image parsing module 102 may determine the color palette of room 200 based on the RGB values ​​of each pixel of input image 108. For example, image parsing module 102 may generate a list of colors present in input image 108 along with the level of presence of each color (e.g., based on the number of pixels). Image parsing module 102 may determine the color palette of room 200 using software code that may or may not be based on a neural network.

[0038] In some example embodiments, the image generation module 104 may receive or otherwise obtain some or all of the information determined by the image parsing module 102. For example, the image generation module 104 may receive one or more of a room type, a design type, a color palette, detected objects along with labels, relative positions of objects, and objects classified as being related to lighting design, and may generate a drawing (i.e., a composite image 110) indicating the relative positions of objects classified as being related to lighting design. For illustration, the information from the image parsing module 102 may be in a standard format or another format that may be interpreted by the image generation module 104. As one of ordinary skill in the art may readily appreciate, the drawing may be generated in a standard format that is viewable by an image viewer. For illustration, the image generation module 104 may include software code that generates a drawing from the information provided by the image parsing module 102. For example, the drawing generated from the image generation module 104 may use generic shapes to represent objects. Labels (e.g., category labels) associated with objects classified as being related to lighting design may be overlaid on the corresponding generic shapes.

[0039] In some alternative embodiments, objects classified as being related to lighting design may be represented in the drawing using generic equivalent objects selected based on tags associated with lighting design related objects. For example, the drawing may show a generic sofa corresponding to sofa 206. The image generation module 104 may use some of the received information (such as room type and design style) to select a particular generic object. In some embodiments, the generic equivalent objects may include, but are not limited to, replacement objects of objects (e.g., removed objects, filtered objects, edited objects), where an average person recognizes or understands the type of object that the replacement object replaces (e.g., sofa, desk, table), but will not be able to identify the actual original object with specific details that the owner wishes to protect (e.g., removed details, changed color schemes, personal items on removed objects, etc.). For example, the generated drawing may show a replacement mirror, but without a reflected image of an object in room 200. The image generation module 104 may also use relative position information to locate objects in the generated drawing. The image generation module 104 may also associate metadata with the drawing, where the metadata may include information such as room type, design style, object location, etc. received from the image parsing module 102. Alternatively, the image generation module 104 may associate metadata with the drawing in a separate file. Figure 1 to provide information together, rather than as metadata.

[0040] In some example embodiments, instead of generating a drawing as described above, the image generation module 104 may include one or more neural network components that perform an image-to-image transformation of the input image 108 to generate a composite image 110. In general, the image-to-image transformation can transfer an image from one domain to another while preserving a representation of the image content, except for specific objects that are intended to be excluded. For illustration, the composite image 110 generated by the image generation module 104 may include objects classified as being related to lighting design, while objects classified as being privacy-sensitive are excluded. In order to meet one or more privacy settings, an object (e.g., a second object) may be removed or excluded from the composite image 110. The privacy setting may be selected based on at least a user, a spatial setting, or an object type. For example, the one or more CNN components of the image generation module 104 may include one or more generative adversarial networks (GANs), each GAN including a generator and a discriminator, and operating to generate an image as readily understood by one of ordinary skill in the art. Some information related to GAN-based image synthesis is provided in L. Wang, W. Chen, W. Yang, F. Bi and FR Yu, “A State-of-the-Art Review on Image Synthesis With Generative Adversarial Networks,” in IEEE Access, vol. 8, pp. 63514–63537, 2020.

[0041] In some example embodiments, the image generation module 104 may generate a mask corresponding to one or more objects based on the user input. For illustration, the user input may provide privacy settings and / or one or more category labels to indicate objects that should be excluded from the composite image 110. For example, such objects may be objects classified as being related to lighting design by the image parsing module 102. The user input may also indicate a room type, a design style, and / or other information. For illustration, instead of the image parsing module 102 determining some parameters such as a room type, a design style, etc., such information may be received by the image generation module 104 as user input.

[0042] In some example embodiments, the image generation module 104 may receive or otherwise obtain the input image 108 and may perform an image-to-image conversion of the input image 108 based on one or more masks provided by the image parsing module 102 to generate the composite image 110. As described above, the image parsing module 102 may generate one or more masks corresponding to objects classified as privacy-sensitive (i.e., privacy-related). The image generation module 104 may generate the composite image 110 such that objects classified as privacy-sensitive (e.g., corresponding to one or more privacy settings) by the image parsing module 102 are excluded from the composite image 110. For example, the jewelry box 234, the spoon 242, and the syringe 244, which may be classified as privacy-sensitive by the image parsing module 102, may be excluded from the composite image 110. One or more masks may also be applied to objects (e.g., the ball 232) that are not related to the lighting design classified by the image parsing module 102, and such objects may also be excluded from the composite image 110, which may have the effect of reducing the appearance of undesirable clutter in the composite image 110. As one of ordinary skill in the art can readily appreciate, portions of the composite image 110 that are already occupied by excluded objects can be inpainted so that the absence of the excluded objects is not apparent. If an object is classified as being related to both lighting and privacy, the image generation module 104 can include a modified object corresponding to the particular object in the composite object 110. For example, a mirror 236 that can be classified as being related to both lighting and privacy can be included in the composite image 110, where the reflection of an object that may be shown in the mirror 236 is excluded (e.g., due to privacy settings).

[0043] In some example embodiments, the image generation module 104 may generate two versions of the composite image 110, one of which is generated based on a mask corresponding to a relatively strict privacy requirement (e.g., a first privacy level requirement, setting) from the image parsing module 102, and the other of which is generated based on a mask corresponding to a relatively loose privacy requirement (e.g., a second privacy level requirement, setting) from the image parsing module 102. For example, the composite image 110 generated based on the relatively strict privacy requirement may be provided to a plurality of lighting design professionals to obtain an initial response. Upon selecting a lighting design professional based on the composite image 110 corresponding to the relatively strict privacy requirement, the composite image 110 generated based on the relatively loose privacy requirement may be provided to the selected lighting design professional.

[0044] In some example embodiments, in addition to excluding some privacy-sensitive objects, the image generation module 104 may include one or more new objects that are not in the input image 108 in the composite image 110. For example, the image generation module 104 may include one or more CNN components (e.g., one or more GANs) configured to include one or more new objects that are not in the input image 108 in the composite image 110, as one of ordinary skill in the art may readily appreciate. For example, a GAN may be used to generate a composite image by combining an image of a foreground object (e.g., an object to be added) with a background image (e.g., an input image or an intermediate image with some objects removed relative to the input image).

[0045] In some example embodiments, the image generation module 104 may add one or more new objects that do not affect the lighting design decisions while adding a level of privacy by introducing one or more objects that are not in the room 200. For example, the image generation module 104 may add objects that are compatible with the design style of the room 200. Alternatively or in addition, the image generation module 104 may add one or more new objects as indicated by the user input. For example, the composite image 110 may include one or more of a piece of furniture (e.g., a chair or shelf), an electronic device (e.g., a music player), an appliance, etc. that is not in the input image 108 and is not shown in the input image 108. Data that can be used to add the one or more new objects may be accessed from a memory device of a device executing the system 100. By notifying a lighting designer receiving the composite image 110 that the composite image 110 includes virtual objects, the impact on the privacy of the owner / occupant of the room 200 may be reduced because the lighting designer does not know whether the objects in the composite image 110 are really in the room 200 or have been virtually added.

[0046] In some example embodiments, the image generation module 104 may also associate metadata with the composite image 110 (e.g., embed the metadata), wherein the metadata may include information such as room type, design style, object location, etc. received from the image parsing module 102. That is, when the composite image 110 is sent to, for example, a lighting design professional, the metadata is automatically transmitted with the composite image 110. Alternatively, in lieu of metadata, the image generation module 104 may provide the information separately but together with the composite image 110.

[0047] In some example embodiments, the image verification module 106 may receive the composite image 110 and determine whether the design style of the room 200 as shown in the composite image 110 matches the design style of the room 200 as shown in the input image 108. To illustrate, the image verification module 106 may include one or more neural networks (e.g., CNNs) pre-trained to perform design style classification. For example, the image verification module 106 may classify the room 200 shown in the input image 108 as modern, casual, classic, natural, or another design style in the manner described above with respect to the image parsing module 102. The image verification module 106 may also classify the design style of the room 200 shown in the composite image 110 as modern, casual, classic, natural, or another design style in the manner described with respect to the image parsing module 102 and the input image 108. In some alternative embodiments, the image verification module 106 may receive the design style classification for the input image 108 from the image parsing module 102.

[0048] In some example embodiments, a comparison module of the image verification module 106 may compare the two classifications of the design style of the room 200 and determine whether the design styles match (e.g., both are casual). If the design styles match, the composite image 110 may be provided to the user for approval, or the image verification module 106 may approve the composite image 110 through an approval setting performed by the image verification module 106. The approval setting may include settings or requirements previously provided by a user, an administrator, and / or a default setting. If the design styles do not match, the image generation module 104 may receive feedback from the image verification module 106 (e.g., through back propagation or as an external input) and regenerate the composite image 110. If the design style match is above a predefined threshold, the image generation module 104 may receive a resulting match score from the image verification module 106. The image generation module 104 may keep regenerating the composite image 110 until the room 200 as captured in the input image 108 and in the composite image 110 has a matching design style above a threshold metric of match.

[0049] In some example embodiments, the image verification module 106 may check the privacy level of the composite image 110. For example, the image verification module 106 may perform object detection and classification to determine whether some objects, such as health-related objects (e.g., syringes 244) and jewelry-related objects (e.g., jewelry boxes 234), are not included in the composite image 110. To illustrate, some objects may be designated as having high privacy sensitivity (e.g., objects that may induce theft), and the image verification module 106 determines whether such objects are present in the composite image 110. If such objects are detected, the image verification module 106 may provide feedback indicating that such objects are detected, and the image generation module 104 may regenerate the composite image 110 without the identified objects. Alternatively, the image verification module 106 may request user input indicating whether it is acceptable to include such objects in the composite image 110.

[0050] In some example embodiments, the image verification module 106 may check the structural similarity between the input image 108 and the composite image 110. Because one or more objects present in the image 108 may not be present in the composite image 110, some dissimilarity between the two images is expected. The image verification module 106 may perform a structural similarity check to ensure that the images are not too dissimilar. For example, the image verification module 106 may determine a structural similarity (SSIM) index of the input image 108 and the composite image 110. As known to those of ordinary skill in the art, the SSIM index is a measure for indicating the similarity between two images. If the SSIM index is below a threshold (e.g., 0.75 on a scale of 0 to 1.0), where the threshold is set to take into account an expected level of dissimilarity, the image verification module 106 may provide feedback indicating that the images 108 and 110 are too dissimilar, and the image generation module 104 may regenerate the composite image 110 in response to the feedback.

[0051] In some example embodiments, after the image quality check performed by the image verification module 106 is satisfactorily completed, the composite image 110 may be provided to the user for approval. If the user approves the composite image 110, the composite image 110 may be sent to one or more lighting design professionals. As described above, the composite image 110 may be generated based on strict privacy requirements for initial transmission to multiple lighting design professionals. By using relatively strict privacy requirements to generate the composite image 110 for initial transmission, more privacy protection is provided to the user until a particular lighting design professional is selected. The strict privacy requirements may include a first privacy level or a higher privacy level as compared to a looser privacy requirement or a different second privacy level.

[0052] After selecting a lighting design professional, the composite image 110 generated based on the more relaxed privacy requirements can be sent to the selected lighting design professional. Because the composite image 110 is generated to adequately represent the lighting-related characteristics of the room 200 as represented in the input image 108, the lighting design professional can use the composite image 110 to satisfactorily perform the lighting design of the room 200. For example, based on the composite image, the lighting professional can suggest lamps for retrofit, freestanding lamps, tabletop lamps, lighting scenes, dim levels, CCT, dynamic lighting characteristics, light distribution, wall locations of control devices, etc.

[0053] Using the composite image 110 instead of the input image 108 can provide a certain level of privacy protection to the owner / occupant of the room 200. The level of privacy protection can be variable (e.g., dynamically modifiable) and selected or modified based in part on the intended use of the composite image 110 and the party with which the composite image 110 will be shared. By using the composite image 110 to request lighting design services, the user can protect their privacy while providing sufficient information for the lighting design professional to remotely perform the lighting design of the room 200.

[0054] In some example embodiments, without departing from the scope of the present disclosure, user input may be provided to the Figure 1 In some example embodiments, the input image 108 may be a panoramic image that may show more of an area (e.g., more of the room 200) than a non-panoramic image. In some example embodiments, the system 100 may operate on multiple images of the room 200 individually and may generate multiple composite images that may be used for remote lighting design of the room 200 without departing from the scope of the present disclosure.

[0055] In some example embodiments, instead of or in addition to GAN, the image generation module 104 and the image verification module 106 may include components based on other neural network architectures (such as a variational autoencoder architecture). In some example embodiments, without departing from the scope of the present disclosure, in addition to or in place of CNN-based components, components based on neural network architectures other than CNN (e.g., Transformer) may be included in the image parsing module 102, the image generation module 104, and / or the image verification module 106. In some example embodiments, without departing from the scope of the present disclosure, in addition to neural network-based software codes and components, non-neural network software codes and components may be used in the image parsing module 102, the image generation module 104, and the image verification module 106.

[0056] In some alternative embodiments, other image editing and synthesis methods than those described above (e.g., text-to-image conversion) may be used in the generation of the composite image 110 without departing from the scope of the present disclosure. For example, a user may provide text instructions to include new objects or modify existing objects, and the image generation module 104 may perform text-to-image conversion to perform the operation. In some example embodiments, the image verification module 106 may perform some but not all image quality checks without departing from the scope of the present disclosure. For example, the image verification module 106 may check for design style matching, but may not perform privacy level and structural similarity checks. In some alternative embodiments, the image verification module 106 may be omitted without departing from the scope of the present disclosure. In some alternative embodiments, the system 100 may include other components other than the image verification module 106 without departing from the scope of the present disclosure. Figure 1 In some alternative embodiments, some components of system 100 may be integrated into a single component. In some alternative embodiments, room 200 may include more than one component without departing from the scope of the present disclosure. Figure 2 More, fewer, or different objects may be shown. For example, room 200 may include a TV or other appliances that may be related to lighting design. In some example embodiments, input image 108 may be an image of a room type different from room 200 without departing from the scope of the present disclosure. For example, input image 108 may be an image of a living room, bedroom, kitchen, playroom, study, etc.

[0057] Figure 3 A method for generating a space (eg, Figure 2 In some example embodiments, the system 300 includes an image parsing module 102, an image generation module 104, and an image verification module 106, which are similar to those described above. Figure 1 and Figure 2 The system 300 may also include a design style validator 304 that is configured to be similar to the design style validator 304 described above. Figure 1 Whether the design style of the composite image 306 matches the design style of the input image 108 is checked in the same manner as described above for the image verification module 106 , the input image 108 , and the composite image 110 .

[0058] In some example embodiments, compared to system 100, system 300 may include an object insertion module 302 that generates a composite image 306 from a generated image 308 generated by image generation module 104. For illustration, the system 300 may include an object insertion module 302 that generates a composite image 306 from a generated image 308 generated by image generation module 104. Figure 1Compared to the composite image 110 described above, the generated image 308 generated by the image generation module 104 in the system 300 may not include objects that are not in the input image 108. Instead, the object insertion module 302 may be configured to generate an image 308 that is not in the input image 108 by using the image generation module 104 and the generated image 308 to generate the object. Figure 1 One or more new objects are inserted in the described manner to generate the composite image 306 from the generated image 308 .

[0059] In some example embodiments, the image verification module 106 in the system 300 may be configured to be associated with the system 100 and Figure 1 To illustrate, the image verification module 106 in the system 300 may receive the input image 108 and the generated image 308 and perform the image quality checks described above with respect to the image verification module 106 in the system 100 .

[0060] In some example embodiments, the design style validator 304 may receive the composite image 306 and the input image 108 and may generate a composite image 306 and an input image 108 in a manner similar to that described in connection with the image validation module 106 and the input image 108. Figure 1 The design styles of the two images may be checked to see if they match in the same manner as described above. If the design styles of the room 200 in the images 108 and 306 do not match, the design style verifier 304 may provide feedback (e.g., via backpropagation or as external input) to the object insertion module 302 that may regenerate the composite image 306. In some example embodiments, if the image verification module 106 indicates a design style match between the input image 108 and the generated image 308, the design style verifier 304 may check for a design style match.

[0061] In some example embodiments, after the image quality check performed by the image verification module 106 and the design style verifier 304 is satisfactorily completed, the composite image 306 may be provided to the user for approval. If the user approves the composite image 306, the composite image 306 may be sent to one or more lighting design professionals. Figure 1 As described above, the composite image 306 may be generated based on strict privacy requirements (e.g., an increased privacy level) for initial transmission to multiple lighting design professionals. By generating the composite image 306 for initial transmission using relatively strict privacy requirements, more privacy protection is provided to the user until a specific lighting design professional is selected. The composite image 306 generated based on more relaxed privacy requirements (e.g., a reduced or lower privacy level) may then be sent to the selected lighting design professional.

[0062] Using the composite image 306 instead of the input image 108 can provide a certain level of privacy protection to the owner / occupant of the room 200. By using the composite image 306 to request lighting design services, the user can protect their privacy while providing sufficient information for the lighting design professional to perform the lighting design of the room 200 remotely.

[0063] In some example embodiments, without departing from the scope of the present disclosure, user input may be provided to the Figure 3 302 . In some example embodiments, instead of or in addition to GAN, the image generation module 104, the image verification module 106, the object insertion module 302, and the design style verifier 304 may include components based on other neural network architectures (such as a variational autoencoder architecture). In some example embodiments, without departing from the scope of the present disclosure, components based on neural network architectures other than CNN may be included in the image parsing module 102, the image generation module 104, the image verification module 106, the object insertion module 302, and the design style verifier 304. In some example embodiments, without departing from the scope of the present disclosure, in addition to neural network-based software codes and components, non-neural network software codes and components may be used in the image parsing module 102, the image generation module 104, the image verification module 106, the object insertion module 302, and the design style verifier 304.

[0064] In some alternative embodiments, other image editing and compositing methods (e.g., text-to-image conversion) other than those described may be used in the generation of the composite image 306 without departing from the scope of the present disclosure. For example, a user may provide text instructions to include a new object or modify an existing object, and the object insertion module 302 may perform a text-to-image conversion to perform the operation. In some alternative embodiments, the image verification module 106 may be omitted without departing from the scope of the present disclosure. In some alternative embodiments, the system 300 may include other than the image verification module 106 without departing from the scope of the present disclosure. Figure 3 Other components than shown. In some alternative embodiments, some components of system 300 may be integrated into a single component.

[0065] Figure 4 A method for generating a space (eg, Figure 22. The system 400 is a functional system 400 for editing an input image 108 of a room 200 (shown in FIG. 1 ). In some example embodiments, the system 400 includes an image editor 402, an image generation module 404, an image verification module 106, an object insertion module 302, and a design style verifier 304. The image editor 402 may be used to edit an input image 108 of a room 200 to occlude / mask (e.g., covered by a filled shape such as a rectangle) objects that are relevant to the privacy of an owner / occupant of the room 200. For example, the image editor 402 may include executable software code to occlude / mask objects in the input image 108 in response to user input. The image editor 402 may also occlude / mask objects that the user deems unrelated to lighting design but relevant to privacy. For illustration, the image editor 402 may include or interface with an editor such as PHOTOSHOP, ADOBE, or other image editor products to occlude / mask objects in response to user input. The image editor 402 may output an edited image 408 having some occluded / occluded objects.

[0066] In some example embodiments, the image generation module 404 may receive the edited image 408 and perform inpainting on the edited image 408 to generate the generated image 410. For example, the image generation module 404 may include one or more CNN components. For illustration, the image generation module 404 may perform inpainting on the occluded portion of the edited image 408 to match the surrounding background of the edited image 408, as can be readily understood by those of ordinary skill in the art.

[0067] In some example embodiments, system 400 may include information about Figure 3 The object insertion module 302 of the system 300 shown in FIG. The object insertion module 302 can generate a composite image 406 from a generated image 410. For illustration, the system 100 and Figure 1 Compared to the composite image 110 described above, the generated image 410 may not include objects that are not in the input image 108. Instead, the object insertion module 302 may be configured to generate the image 410 by using the image generation module 104 and the Figure 1 One or more new objects are inserted in the described manner to generate the composite image 406 from the generated image 410 .

[0068] In some example embodiments, the image verification module 106 in the system 400 may be configured to be associated with the system 100 and Figure 1To illustrate, the image verification module 106 in the system 400 may receive the input image 108 and the generated image 410 and perform the image quality checks described above with respect to the image verification module 106 in the system 100 .

[0069] In some example embodiments, the design style verifier 304 may receive the composite image 406 and the input image 108 and may generate a composite image 406 and an input image 108 in a manner similar to that described in connection with the image verification module 106 and the input image 108. Figure 1 The design styles of the two images are checked to see if they match in the same manner as described above. If the design styles of the room 200 in the images 108 and 406 do not match, the design style verifier 304 may provide feedback (e.g., via backpropagation or as external input) to the object insertion module 302 that may regenerate the composite image 406. In some example embodiments, if the image verification module 106 indicates a design style match between the input image 108 and the generated image 410, the design style verifier 304 may check for a design style match.

[0070] In some example embodiments, after the image quality check performed by the image verification module 106 and the design style verifier 304 is satisfactorily completed, the composite image 406 can be provided to the user for approval. If the user approves the composite image 406, the composite image 406 can be sent to one or more lighting design professionals. Figure 1 As described, the composite image 406 can be generated based on strict privacy requirements for initial transmission to multiple lighting design professionals. By using relatively strict privacy requirements (e.g., a first privacy level, an increased privacy level) to generate the composite image 406 for initial transmission, more privacy protection is provided to the user until a specific lighting design professional is selected. The composite image 406 generated based on more relaxed privacy requirements (e.g., a second privacy level, a lower privacy level) can then be sent to the selected lighting design professional.

[0071] Using the composite image 406 instead of the input image 108 can provide a certain level of privacy protection to the owner / occupant of the room 200. By using the composite image 406 to request lighting design services, the user can protect their privacy while providing sufficient information for the lighting design professional to perform the lighting design of the room 200 remotely.

[0072] In some example embodiments, without departing from the scope of the present disclosure, user input may be provided to the Figure 4Components of more or different system 400 are shown. In some example embodiments, instead of or in addition to GAN, image generation module 404, image verification module 106, object insertion module 302, and design style verifier 304 may include components based on other neural network architectures (such as a variational autoencoder architecture). In some example embodiments, components based on neural network architectures other than CNN may be included in image generation module 404, image verification module 106, object insertion module 302, and design style verifier 304 without departing from the scope of the present disclosure. In some example embodiments, in addition to neural network-based software codes and components, non-neural network software codes and components may be used in image generation module 404, image verification module 106, object insertion module 302, and design style verifier 304 without departing from the scope of the present disclosure.

[0073] In some alternative embodiments, other image editing and compositing methods other than those described (e.g., text-to-image conversion) may be used in the generation of composite image 406 without departing from the scope of the present disclosure. For example, the functions of image editor 402 may be performed by a text-to-image conversion neural network component. In some alternative embodiments, image verification module 106 may be omitted without departing from the scope of the present disclosure. In some alternative embodiments, system 400 may include other than the text-to-image neural network component without departing from the scope of the present disclosure. Figure 4 For illustration, the system 400 may include an image parsing module 102 coupled to receive and parse the edited image 408 to obtain information such as room type, design style, and information about the room. Figure 1 For example, the image parsing module 102 may provide information to the image generation module 404, which may generate the generated image 410 based on the information. In some alternative embodiments, some components of the system 400 may be integrated into a single component.

[0074] Figure 5 A device 500 for generating a composite image of a space for lighting design according to an example embodiment is shown. In some example embodiments, the device 500 may be used to implement Figure 1 , Figure 3 and Figure 4100, 300, and 400 of the functional systems 100, 300, and 400. In some example embodiments, the device 500 may include a processor unit 502, a memory unit 504, a camera unit 506, a Lidar unit 508, a user interface 510, a communication interface 512, and (one or more) other components such as a ToF (time of flight) sensor unit. For example, the processor unit 502 may include one or more microprocessors and / or a graphics processing unit configured to execute software code for the device 500 to perform the operations described herein with respect to the functional systems 100, 300, 400. For example, the memory device 504 may store software code (e.g., neural network and other software code) that can be executed by the processor unit 502 to perform the operations described herein with respect to the systems 100, 300, 400.

[0075] In some example embodiments, the memory unit 504 may include one or more memory devices, such as flash memory devices, static random access memory devices, and / or other types of memory devices, as would be readily appreciated by one of ordinary skill in the art. In some example embodiments, data used and / or generated by the device 500, such as image data corresponding to the input image 108, the synthesized images 110, 306, and 406, the generated images 308 and 410, the edited image 408, mask data, and Lidar information, may be stored in the memory unit 504.

[0076] In some example embodiments, the camera unit 506 may be used to capture the input image 108 of the room 200 and other images that may be used as input images for the systems 100, 300, and 400. The Lidar unit 508 may be used to obtain Lidar information that may be used or otherwise processed by the processor unit 502 along with the input image 108, as described above with respect to Figure 1 and Figure 2 described.

[0077] In some example embodiments, the processor unit 502 may receive user input and provide information to the user via the user interface 510. For example, the processor unit 502 may request the user to approve the composite image 110, 306, 406 via the user interface 510. As another example, during execution of the image parsing module 102, the processor unit 502 may request the user to indicate whether an object should be classified as privacy sensitive. For illustration, the user interface 510 may include a touch screen that may be used to receive user input and display information, as may be readily understood by one of ordinary skill in the art. Alternatively or additionally, the user interface 510 may include other input devices, such as a keyboard, a mouse, etc., without departing from the scope of the present disclosure.

[0078] In some example embodiments, processor unit 502 may transmit and receive information via communication interface 512. Communication interface 512 may include one or more transmitters and / or receivers that communicate with devices using signals compatible with one or more wireless or wired communication standards, for example, locally and / or over the Internet.

[0079] In some example embodiments, device 500 may be a smartphone, a tablet computer, a laptop computer, or another electronic device, as will be readily appreciated by one of ordinary skill in the art having the benefit of this disclosure. In some example embodiments, device 500 may include, without departing from the scope of this disclosure, Figure 5 More or fewer components may be shown. In some alternative embodiments, one or more components of the device 500 may be omitted without departing from the scope of the present disclosure. For example, in some example embodiments, the Lidar unit 508 may be omitted. In some alternative embodiments, some components of the device 500 may be integrated into a single component without departing from the scope of the present disclosure.

[0080] Figure 6 A method 600 of generating a composite image of a space for lighting design according to an example embodiment is shown. Figure 1-Figure 6 In some example embodiments, the method 600 includes, at step 602, obtaining an input image 108 of a space (e.g., room 200). For example, the processor unit 502 may obtain the input image 108 from the memory unit 504. For illustration, the input image 108 may be captured by the camera unit 506, and data corresponding to the input image 108 may be stored in the memory unit 504. In some embodiments, the input image 108 may be transmitted to the processing unit 502 of the device 500, or transmitted by the system 100 from a separate camera unit and / or imaging device. At step 604, the method 600 may include performing parsing of the input image 108 of the space to detect objects in the input image 108. For example, the processor unit 502 may execute software code corresponding to the image parsing module 102 to parse the input image 108 to detect objects in the room 200, such as the desk 202, the chair 204, etc.

[0081] In some example embodiments, at step 606, method 600 may include classifying objects detected in input image 108 based at least on relevance to a lighting design for the space and relevance to privacy, as described above with respect to image parsing module 102. For example, processor unit 502 may execute software code corresponding to image parsing module 102 to classify some objects in room 200 (as detected in input image 108) as being related to a lighting design and to classify some other objects in room 200 as being related to privacy (i.e., privacy sensitive). Processor unit 502 may execute software code corresponding to image parsing module 102 or image generation module 104 to determine to include a first object (218) of the objects in the input image in a composite image (110, 306) of the space from the input image of the space, for example, at least in part because the first object is related to a lighting design object. Processor unit 502 may execute software code corresponding to image parsing module 102 or image generation module 104 to determine, for example, based at least on one or more privacy settings, to exclude a second object (242, 244) of the objects in the input from the composite image. At step 608, method 600 may include generating a composite image (e.g., composite image 108, 306) of a space (e.g., room 200) from input image 108 of the space. For example, processor unit 502 may execute software code corresponding to image generation module 104 of system 100 to generate composite image 110. Processor unit 502 may also execute software code corresponding to image generation module 104 and object insertion module 302 of system 300 to generate composite image 306. A first object (e.g., Figure 2 218) may be included in the composite image, and a second object (eg, Figure 2 The syringe 244 shown in the figure can be excluded from the composite image (e.g., due to privacy settings, satisfying one or more privacy settings). In an embodiment, the privacy setting is selected based on at least the user, the setting of the space, or the type of object. For illustration, the first object can be related to lighting design (e.g., corresponding to a lighting design object), and the second object can be related to privacy (i.e., privacy sensitive, satisfying the privacy setting). Some other objects classified as being related to lighting design can also be included in the composite image generated by the processor unit 502 executing the image generation module 104 and / or the object insertion module 302, such as with respect to Figure 1 and Figure 3 As described.

[0082] In some example embodiments, method 600 may further include verifying whether the design style of the space (e.g., room 200) as shown in input image 108 matches the design style of the space as shown in a composite image (e.g., composite image 110). Method 600 may further include generating a second composite image from input image 108 of the space based on a relaxed privacy requirement that is less stringent than the strict privacy requirement used in generating the first composite image.

[0083] In some alternative embodiments, the method 600 may include more or fewer steps than shown without departing from the scope of the present disclosure. In some alternative embodiments, the steps of the method 600 may be performed in a different order than shown without departing from the scope of the present disclosure.

[0084] Although specific embodiments have been described in detail herein, these descriptions are by way of example. The features of the example embodiments described herein are representative, and in alternative embodiments, certain features, elements, and / or steps may be added or omitted. In addition, those skilled in the art may modify various aspects of the example embodiments described herein without departing from the scope of the appended claims, the scope of which should be consistent with the broadest interpretation so as to cover modifications and equivalent structures.

Claims

1. A computer-implemented method (600) of generating a composite image of a space (200) for lighting design, the method comprising: Obtaining (602) a spatial input image (108); performing (604) parsing of the spatial input image to detect objects (202, 204, 206, 212, 218, 236, 240, 242, 244) in the input image; classifying objects detected in the input image based at least on the objects corresponding to one or more lighting design objects of the space and a privacy setting (606), wherein the privacy setting is selected based at least on a user, a setting of the space, or a type of the object; determining to include a first object (218) of the objects in the input image in a composite image (110, 306) of the space from the input image of the space, wherein the first object is related to a lighting design object; determining to exclude a second object among the objects in the input from the composite image (242, 244), wherein the second object is excluded from the composite image due to a privacy setting; and A composite image (110, 306) of the space is generated (608) from an input image of the space, wherein a first object (218) of the objects in the input image is included in the composite image, and wherein a second object (242, 244) of the objects in the input image is excluded from the composite image.

2. The method of claim 1, wherein the first object is a light fixture (218), a light-transmitting object (212, 214), or a light-reflecting object (236).

3. The method of claim 1, wherein the first object is a piece of furniture (202, 204, 206), an appliance, or a work of art (230).

4. The method of claim 1, wherein a third object (206) in the input image (108) is excluded from the composite image based on a desired illumination of the space.

5. The method of claim 1, wherein the one or more new objects are in the synthesized image, and wherein the one or more new objects are not present in the input image.

6. The method of claim 1, wherein determining that the second object (242) satisfies at least one privacy setting is based on a proximity of the second object to a third object (240) in the input image of the space.

7. The method of claim 1, wherein a third object (240) in the input image that is a lighting design object for the space corresponds to a privacy setting and is excluded from the composite image.

8. The method of claim 1, wherein a third object (236) in the input image that is a lighting design object for the space corresponds to a privacy setting and is included in the composite image. 9 . The method of claim 1 , further comprising verifying whether a design style of the space as shown in the input image matches a design style of the space as shown in the synthesized image.

10. The method of claim 1, further comprising generating a second composite image from the spatial input image based on a relaxed privacy requirement that is less stringent than a strict privacy requirement used in generating the composite image.

11. A device (500) for generating a composite image (110, 306, 406) of a space (200) for lighting design, the device comprising a processor (502), the processor (502) being configured to: Obtaining a spatial input image (108); performing parsing of the input image of the space to detect objects in the input image (202, 204, 206, 212, 218, 236, 240, 242, 244); classifying objects in an input image of the space based at least on the objects corresponding to one or more lighting design objects of the space and a privacy setting, wherein the privacy setting is selected based at least on a user, a setting of the space, or a type of the object; determining to include a first object (218) of the objects in the input image in a composite image (110, 306) of the space from the input image of the space, wherein the first object is related to a lighting design object; determining to exclude a second object among the objects in the input from the composite image (242, 244), wherein the second object is excluded from the composite image due to a privacy setting; and A composite image (110, 306) of a space is generated from an input image of the space, wherein a first object (218, 212, 236) of the objects in the input image is included in the composite image, and wherein a second object (242, 244) of the objects in the input image is excluded from the composite image. 12 . The apparatus according to claim 11 , wherein the first object is a lamp, a light-transmitting object or a light-reflecting object.

13. The apparatus of claim 11, wherein the one or more new objects are in the synthesized image, and wherein the one or more new objects are not present in the input image.

14. The apparatus of claim 11, wherein the second object (242) is determined to be a privacy-sensitive object based on a proximity of the second object to a third object (240) in the input image of the space.

15. The device of claim 11, further wherein the processor (502) is further configured to verify whether a design style of the space as shown in the input image (108) matches a design style of the space as shown in the synthesized image (110, 306).

Images