Visual modeling system and method thereof

Abstract

The present invention provides a visual modeling system, which is operable to model the appearance of a scanned object on a structure (109). The visual modeling system associates a first scan data set with lighting elements (410) corresponding to locations on the structure (109) defined in a second data set via a mapping engine. An electronic lighting control system (420) energizes the addressable lighting elements (410) based on the mapping engine relationship between the first scan data set and the second scan data set to represent the scanned image on the structure. The visual modeling system includes a structure scanning module (400), which is operable to generate the second scan data set by constructing a 3D virtual model of the structure (109), the 3D virtual model defining the locations of the lighting elements and the order in which the electronic lighting control system addresses the lighting elements.

Description

Technical Field

[0001] The present invention generally relates to tourist attraction facilities. More specifically, the present invention relates to visual modeling devices, methods and systems that can be used to generate and produce large visual models on sculptures and other objects. Background Art

[0002] Major entertainment and tourist attractions, from landmark structures to entertainment centers such as Disney World, Times Square and Grand Central Station in the United States and other entertainment centers around the world, receive hundreds of millions of visitors each year. In order to keep visitors coming back, there is always a need for fresh and innovative ways to attract visitors' attention and generate those amazing reactions that keep visitors coming back and encourage other visitors to visit. There is a wide range of attractions, from exciting attractions (such as roller coasters and other amusement park rides that go higher and faster) to visually stunning attractions (such as the Eiffel Tower or Mount Rushmore). While known visually stunning attractions are undoubtedly impressive, there is a need for more fascinating and interactive visual attractions that will fascinate a new generation of visitors.

[0003] An example of a tourist attraction facility is described in US20170004228. The tourist attraction facility described therein allows visitors to visually project their own image into a large-scale model. However, one of the problems with the described facility is that inaccuracies between the scanned object data set and the exact position of the lighting elements on the surface of the large-scale model can affect the quality of the projected image. Summary of the invention

[0004] The present invention aims to provide apparatus, methods and systems for providing interactive tourist attraction facilities that overcome the disadvantages of prior art solutions.

[0005] This object is achieved according to the present invention by a visual modeling system showing the technical characteristics of the first technical solution.

[0006] According to one aspect of the present invention, a visual modeling system is provided. The visual system includes: a first scan data set associated with a subject; a structure having a plurality of addressable lighting elements disposed proximate to a surface of the structure; a second scan data set associated with the locations of the addressable lighting elements on the surface of the structure; a processor-based image processing system including a data receiving module for receiving the generated first scan data set, and a mapping engine for associating the first scan data set with lighting elements corresponding to locations on the structure defined by the second data set; and an electronic lighting control system including a power supply and a lighting controller in communication with the plurality of addressable lighting elements, wherein the electronic control system energizes the addressable lighting elements according to a mapping engine relationship between the first scan data set and the second scan data set. According to the present invention, the visual modeling system includes a structure scanning module operable to generate the second scan data set by constructing a 3D virtual model of the structure, the 3D virtual model defining the locations of the lighting elements and the order in which the electronic lighting control system addresses the lighting elements.

[0007] It has been found that building a 3D virtual model of a structure has the following advantages: increasing the accuracy of the image projected onto the surface of the structure. The 3D virtual model includes more data points than a traditional 2D scan. Therefore, the mapping engine can more accurately map the first scan data set to the second scan data set, thereby producing a more realistic and detailed visual representation of the scanned object on the surface of the structure. The increase in mapping accuracy can significantly increase the number of tourists who visit tourist attraction facilities each year, which can boost the economy of the local community.

[0008] According to an embodiment of the present invention, the structure scanning module is configured to build a 3D virtual model of the structure by performing the following steps:

[0009] a) operating the electronic lighting control system to cycle through the addressable lighting elements one at a time;

[0010] b) operating a scanning device configured to scan the structure from different angles, the scanning device configured to detect light generated by each energized addressable lighting element;

[0011] c) constructing a point cloud from the detected light emitted from each addressable lighting element, each point defining the location of the addressable lighting element on the surface of the structure and the order in which the addressable lighting elements are addressed by the electronic lighting control system; and

[0012] d) converting the point cloud into a 3D virtual model of the structure.

[0013] It has been found that performing the above steps has the following advantages: Reducing the processing power and storage required to build a 3D virtual model. Generating a point cloud consisting of a set of points located in a three-dimensional coordinate system requires significantly less storage and processing power than an equivalent 3D model consisting of a surface. The collection of data points can be converted into a polygonal model that can be rendered like a traditional 3D model. The processing of the data obtained by the scanning equipment and the construction of the point cloud can be performed by a processing unit, which can be part of a processor-based image processing system. In addition, the generation of the point cloud can significantly improve mapping reliability and accuracy, while reducing mapping distortions caused by the inability to map all data points from one group to another group.

[0014] According to an embodiment of the present invention, step a) and step b) are performed synchronously so that when each addressable lighting element is powered on, the scanning device is operated simultaneously. Therefore, a more accurate mapping of the positions of the addressable lighting elements can be achieved. According to an embodiment of the present invention, step a) and step d) for generating the 3D model can be performed by an executable computer program loaded into the memory of the processor.

[0015] According to an embodiment of the present invention, the scanning equipment may be in the form of a movable scanner. The scanner is configured to scan the structure from different angles. The scanner may include a lidar scanner arranged to identify the spatial position of the addressable lighting elements, and a raster scanner arranged to identify the addressable lighting elements as bright spots. According to an embodiment of the present invention, multiple scanners may be deployed at different locations around the structure. According to an embodiment of the present invention, the scanner may be positioned on the ground. According to an embodiment of the present invention, the scanner may be part of an aerial drone device that can be remotely operated by a user, and the aerial drone device is also called an unmanned aerial vehicle.

[0016] According to an embodiment of the present invention, the electronic lighting control system is configured to compare the generated 3D virtual model with the expected addressable lighting element map of the structure to identify the presence of faulty addressable lighting elements on the surface of the structure that cause differences in luminous uniformity. Brightness and / or color non-uniformity can be caused by faulty addressable lighting elements (e.g., non-operation, brightness color problems). In addition, brightness and / or color non-uniformity can be caused by addressable lighting elements positioned at the wrong position on the surface of the structure. The position of the addressable lighting elements can be affected by several factors, such as weather, misalignment during assembly of the structure, ordinary wear and tear, etc. According to an embodiment of the present invention, the electronic control system is configured to generate a list containing the locations of detected faulty addressable lighting elements, which is communicated to a system administrator via a communication network. In this way, faulty addressable lighting elements can be quickly repaired, thus ensuring that the quality of the projected scanned image is not affected. The generated list may further include other information, such as possible causes of the failure, e.g., location or operation of the addressable lighting element, and / or image resolution lost due to the detected faulty addressable lighting element. In this way, a system administrator may better assess the severity of brightness and / or color non-uniformity and command corrective action accordingly.

[0017] According to an embodiment of the present invention, the electronic control system is configured to adjust the current supplied to the addressable lighting elements when brightness and / or color non-uniformity is detected. In this way, the color intensity and / or brightness of the addressable light elements adjacent to the affected structural area can be adjusted to at least partially correct the brightness and / or color non-uniformity caused by the detected faulty addressable lighting element.

[0018] According to an embodiment of the present invention, the electronic control system is configured to adjust the pulse width supplied to each addressable lighting element. The pulse width is applied to each addressable light element by an electronic lighting controller. Pulse width modulation (PWM) is a widely used technique to control the brightness of addressable lighting elements and can be used to perform uniformity correction. PWM works by operating the addressable lighting elements fully on or fully off at a very high rate. The flashes are so fast that the human eye cannot notice them, and the duration (pulse width) of the individual flashes determines the perceived brightness.

[0019] According to an embodiment of the present invention, the structure is formed into a general shape of the type of subject being scanned. For example, the subject being scanned may be in the shape of a portrait. The scan data set may be a set of three-dimensional data, and the structure is formed into a general three-dimensional shape of the type of subject being scanned. A three-dimensional scanner may be used to scan the subject for visual modeling. Using a set of three-dimensional data has the following advantages: improving the reliability and accuracy of mapping the scanned image to the structure, so that an accurate representation of the scanned subject is achieved.

[0020] According to an embodiment of the present invention, the mapping engine is configured to map the 3D point positions defined in the second scan data set to the 2D image points defined in the first data set. In this way, a simple camera can be used to replace the 3D scanner to scan the subject, thereby significantly reducing the cost of the subject scanning equipment. The mapping of the 2D image of the first scan data set to the 3D virtual model defined in the second scan data set can be performed by UV mapping technology. UV mapping technology involves a 3D modeling process in which a 2D image is projected onto the surface of a 3D model for texture mapping. The letters "U" and "V" represent the axes of the 2D texture, because "X", "Y" and "Z" have been used to represent the axes of the 3D object in the model space.

[0021] According to an embodiment of the present invention, a humanoid structure includes at least one of a movably attached head, neck, arms, torso, or legs and an electric actuator for moving at least one of the head, neck, arms, torso, or legs. The system may include at least one motion sensor and a motion processing engine, the motion processing engine being used to translate the movement of the subject into movement commands sent to at least one actuator, whereby a user can control the movement of the structure. In this way, the humanoid structure can mimic basic movements made by the scanned subject, such as moving the head, arms, bending, etc. According to an embodiment of the present invention, the overall shape of the humanoid structure can be changed depending on the shape of the scanned subject. For example, portions of the structure can be moved inwardly or outwardly to increase or decrease the diameter of a predetermined section of the structure, such as an arm, abdomen, etc. In this way, a more realistic representation of the scanned subject can be achieved.

[0022] According to an embodiment of the invention, the structure is at least ten times larger than the subject being scanned.

[0023] According to an embodiment of the present invention, the scanned data set may be in the form of a video, and wherein the electronic lighting control system energizes the addressable lighting elements to generate a moving image on the structure.

[0024] According to an embodiment of the present invention, the lighting elements are outwardly facing light emitting diodes (LEDs) arranged in a plurality of vertically spaced horizontal rows.

[0025] According to an embodiment of the present invention, the frontal portion of the head and torso has a higher density of light emitting diodes than the rest of the body. For example, the face of the head has a higher density of light emitting diodes than the frontal portion of the head and torso.

[0026] According to an embodiment of the present invention, the structure comprises a plurality of interconnected blocks, each block comprising at least one addressable lighting element. Thus, the structure can be easily and quickly assembled and disassembled. Each block may comprise at least one slot configured to receive at least one addressable lighting element. The slot may be an electrical slot arranged to provide a power signal to the addressable lighting element secured therein. For example, the addressable lighting element may be positioned on the structure after the structure has been assembled.

[0027] According to an embodiment of the present invention, the addressable lighting elements may be corrugated and fixed after assembling the structure. In this way, it may not be necessary to provide slots on the surface of the structure.

[0028] According to an embodiment of the present invention, the blocks forming the structure may be in the form of pixel tiles or pixel blocks. The pixel tiles may be provided with a plurality of addressable lighting elements.

[0029] According to an embodiment of the present invention, each block includes an interconnection element configured to releasably connect each block to each adjacent block. The interconnection element may be configured to carry power signals between the blocks to control the operation of the addressable lighting elements. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The following drawings are provided as examples to further illustrate and describe various aspects of the present invention.

[0031] Figure 1 and Figure 2 The tourist attraction facilities according to the embodiment of the present invention are shown and illustrated.

[0032] Figure 3 An example of a method for structurally visually modeling an image of a scanned object according to an embodiment of the present invention is shown.

[0033] Figure 4 An exemplary representation of a visual modeling system according to an embodiment of the present invention is shown.

[0034] Figure 5 An exemplary data flow diagram for obtaining a 3D virtual model of a structure according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0035] Will be used in Figures 1 to 5The present invention is illustrated by the exemplary embodiment shown in the figure, which will be described in more detail below. It should be noted that any reference to size is indicative only and does not constrain the present invention in any way. Although the present invention has been shown and described with reference to the specific illustrated embodiment of the present invention, it will be understood by those skilled in the art that various changes in form and detail can be made therein without departing from the scope of the present invention covered by the appended claims. In addition, although the present invention has been described with reference to a specific interactive tourist attraction facility, it will be understood by those skilled in the art that changes in form and detail can be made to promote other types of tourist attraction facilities without departing from the scope of the present invention covered by the appended claims.

[0036] All publications, patents, patent applications, and other references mentioned herein are hereby incorporated by reference in their entirety for all purposes to the same extent as if each individual publication, patent or patent application was specifically and individually indicated to be incorporated by reference and was set forth in its entirety.

[0037] The visual modeling system described herein provides systems and methods for producing amazing fun and entertainment structures. Most large structures today are static and, unless they have some historical significance, are generally attractions that only give people a three-minute heat. The systems and methods disclosed herein produce unique and attractive attractions by producing interactive large structures that can be used to model smaller characters on large characters with increased accuracy compared to known methods such as the method disclosed in US20170004228.

[0038] In various embodiments, the system may be used to scan and generate a three-dimensional ("3D") representation of a person, such as an attraction visitor, and the representation may then be modeled on a large structure. Various other embodiments may also include movement through streamed video of the visitor and movable appendages provided on the structure that may be controlled by the attraction visitor.

[0039] In short, in a preferred embodiment, each person or visitor entering a visitor attraction will enter a 2D or 3D scanner and their image, video and audio will be scanned and digitized using known scanning and recording techniques. The data will then be processed and transformed into a corresponding matrix mapping of the lighting elements embedded in the structure so that the visitor's appearance can be replicated on a large structure. The audio-video images can also be scanned remotely or obtained from stored data, for example, to replicate celebrities or other famous figures, and transmitted and saved for upload at a later time on a large "giant" structure.

[0040] See also Figure 1, shows an illustration of a tourist attraction facility 100 having a large structure 109 built in the form of a standing human figure having a head, arms, torso, and legs built on a base 110. Although the embodiments depicted herein are humanoid, any 3D object may be modeled using an appropriate form, for example, a structure may be built to represent or model a dinosaur or other animal. A tower 111 and a platform 112 may be provided to allow visitors to climb to the top of the structure.

[0041] The massive structure 109 may be constructed by any known means of construction, such as steel or wood frame construction, and may have a skin or no skin. In one embodiment, the structure is constructed of a wood frame with horizontally rendered slats or a frame that defines the form of the object to be modeled. In some embodiments, the front edges of several sections of wood or another material used to construct the structure or giant may be covered with a transparent skin, such as diffuse Plexiglas. Addressable lighting elements may be mounted behind and recessed into the glass or other see-through membrane. In some embodiments, the lighting elements are preferably clusters of red, green, and blue light emitting diodes, however, any known light emitters may be used.

[0042] As previously described, the large structure 109 can be built from interconnected pixel blocks, each block including at least one addressable lighting element. Each block can be configured to be connected to an adjacent block via an interconnection member. For example, each block can have an opening on at least one side and a protruding element on the other side. In this way, the pixel blocks can be interconnected by pushing the provided protruding opening into the opening of the adjacent block. The block can have an opening and a protruding element on at least one side. The interconnection member can be arranged to carry electrical signals, such as power signals, data signals, control signals, etc. Each block can include a plurality of pixels, and the plurality of pixels include at least one addressable lighting element addressed in a manner similar to known LED displays. It should be noted that the blocks can be fixed by any means known to the technician, such as bolts, clamps, etc.

[0043] The lighting elements can be placed at different depths of the recess, and the spacing of the pixels can be changed to alter the visual appearance of each pixel. By controlling the spacing and depth of the lighting elements, the visual appearance can be configured to produce sharp edges or giant pixels that blend into each other.

[0044] The array or matrix of lighting elements on the structure will preferably cover the entire structure, including the front, back and sides. The pixel resolution or distribution, density or size of the lighting elements on the structure can be uniform and consistent, or in some embodiments can be distributed to provide finer resolution in key areas of the structure. For example, in a human form structure of one embodiment, most of the horizontal segments in the structure (which together produce the body of the sculpture) can be installed with large pixels, and the large pixels have a horizontal spacing between 1.0cm and 25cm. For example, the head (excluding the face) and the front torso can be constructed with 4000 medium-spaced pixels with a horizontal spacing of about 10cm. In some embodiments, the vertical spacing is preferably determined by the distance between the individual segments of the horizontal structure. For example, the segment that produces the face is embedded with 3,000 small pixels in total, and the small pixels have a pixel spacing of about 5cm, which is sufficient for a viewing distance of more than 50m, and a clear image can be seen when standing directly in front of the structure. These exemplary pixel / illumination element distributions may, in one embodiment, be used on structures having a predetermined height (e.g., at least 10 times greater than the size of the object being scanned), it being understood that higher or lower numbers of illumination elements may be used on structures of proportional size, or when increased or decreased resolution is desired.

[0045] See also Figure 2 , a block diagram illustrating a visual modeling system 200. The system, in one embodiment, includes a 2D or 3D scanner 202 for scanning visitors to the attraction 100. The system may also include a storage unit 204 and a database to buffer or store the scanned data. In addition to the image data received from the 3D scanner 202, the system and method may receive or use previously scanned or generated data stored on the storage unit 204 or streamed in real time for display on a large-scale model. Although the image, in one embodiment, preferably relates to the form of a sculpture, the image may be any photographic image or any of a large number of static or moving visual images (representational or abstract).

[0046] The visual modeling system 200 may include one or more servers 206, 208. In other embodiments, the entire system may be independent and operated from a single server. The servers 206, 208 may be any known type of readily available system including a processor, memory, and communication ports for receiving and transmitting data from and to external devices via any known data receiving and transmitting hardware and software modules.

[0047] The servers 206, 208 may be connected to and / or communicate with entities such as, but not limited to: one or more users from user input devices such as a keyboard and mouse used to control the system; peripheral devices such as a scanner 202 or other devices such as position input devices (e.g., to allow a user to control the movement of a large structure); and output controllers used to activate and command control circuits (e.g., to power and control a matrix of lighting elements and motion actuators in a large structure).

[0048] like Figure 2 , the visual modeling system is distributed and includes two or more computers or servers 206, 208 connected by a communication network 200. In these embodiments, the scanning and storage system can be close to or remote from the large structure. For example, in this configuration, the large structure can be used to allow remote performances, lectures, or demonstrations, where singers, actors, or speakers can appear in real time at different locations before a large audience by transmitting a live feed via the communication network 200 to multiple large "giant" structures located across the country or the world.

[0049] The network 200 facilitates the transfer of information from a source point to a destination. There are many forms of networks that may be suitable for use, such as a local area network (LAN), a piconet, a wide area network (WAN), a wireless network (WLAN), etc. For example, the Internet is generally accepted as an interconnection of numerous networks whereby remote clients and servers can access and interoperate with each other, and can be used to facilitate such remote video and image transmission to large structures.

[0050] The visual modeling system 200 also includes software modules including a communication module, a mapping engine, a controller module, a motion processing engine, and a structure scanning module. The communication module is any known interface module that uses standard communication protocols to communicate on a local area network (LAN), a piconet, a wide area network (WAN), a wireless network (WLAN) and communicates with peripheral devices such as a scanner 202, a storage unit 204, and an electronic control system to control the lighting matrix. The mapping engine and controller module software may be separate modules or combined into a single module.

[0051] The function of the mapping engine is to transform the image data into a 3D matrix of lighting elements on a large "giant" structure. This process is preferably performed by a correlation mapping of the scanned or stored image data to the lighting matrix. The process may include using a known reduction or downsampling algorithm (e.g., nearest neighbor, linear, cubic or bicubic) to reduce the resolution of the original, which will generally be higher than the resolution of the matrix of light elements on the large structure. In the case of using high pixel density on the large structure, multiplication or upsampling may be required and may be implemented by any similar known technology. In some embodiments, since the pixel resolution or density varies on the large structure, selective multiplication or reduction may be used on portions of the image to map the image to the lighting matrix of the large giant structure. For example, in a human form embodiment, where the face has a high pixel density, multiplication may be required in certain embodiments, while the legs and arms of the large structure may have a low pixel density and may need to be reduced to map to the lighting element matrix.

[0052] The controller module interfaces with a lighting control system that energizes the addressable lighting elements. The lighting elements may be addressed by any known method used in conjunction with conventional LED displays, such as arranging the light elements in a passive matrix and using multiplexing techniques to reduce the number of physical connections. Alternatively, the lighting element matrix may be configured to use active matrix technology to provide faster response times in video and multimedia versions of the systems and methods.

[0053] The lighting control system circuitry may also include a power supply and / or lighting controls for the matrix of lighting elements, and may be wired in a combination of series and parallel configurations using an AC mains voltage supply that is converted to a lower voltage and rectified to a stable DC output. Pulse width modulation may be used to dim the lighting elements. Data communications may use a TCP / IP based protocol such as Artnet or sACN, or other protocols described above for rapid display of lighting changes.

[0054] The control module provides a software communication interface with the lighting control system and in some embodiments provides commands to drive the lighting sequence of the lighting element matrix. In alternative embodiments, the control module functionality and the lighting control system may be combined in a single unit, such as in the server 208 in the base 110.

[0055] The movement processing engine provides software for receiving movement data and correlating the movement into commands to control actuators on the giant. Movement data may be received from a movement sensor attached to the user, a scan of the user's movement, or from a movement file stored in memory. The movement processing engine is connected directly to the actuators that control the giant's attachments, or wirelessly to the actuators via a server or controller, and sends movement commands to the actuators to produce movement.

[0056] See also Figure 3 , depicting a flow chart of an embodiment. The exemplary flow chart depicts how the system can act as a tourist attraction. The process begins at 300, where a visitor or user enters the attraction. At startup, the system will load a lighting matrix map in step 302, which contains a map of the addressable lighting element layout of a large giant structure. Next, the system will load or scan data associated with the visitor (e.g., a first scan data set) in step 304. In one embodiment, the visitor preferably walks into a 3D scanner after entering the attraction. The 3D scanner then scans the visitor and generates visualization and / or position data of the visitor. The visualization data is essentially a scan of both the 3D form and image data of the visitor. The position data can be optionally used to pose the appendage of the large structure to match the posture of the visitor, or use a real-time scanner to read the visitor's movement and move the structure synchronously with the movement of the visitor.

[0057] In step 306, the mapping engine transforms and maps the visualization data, such as the image data of the visitor, onto the matrix mapping of the lighting elements using multiplication, reduction and / or correlation mapping as discussed above. In step 308, the lighting control system and modules energize the lighting elements according to the transformed visualization data.

[0058] In the embodiment with movable appendages, in step 310, control modules and systems are used to move actuators in arms, legs, necks and torsos to position the appendages according to the movement of the visitors so that the structure poses. In a system configured as interactive or real-time video or movement, in step 312, the system will look for updated frames or movement data. Information about movement can be read from a 3D scanner, other sensors or actuators attached to the visitors or controlled by the visitors, or from pre-recorded movement files stored in a memory. Similarly, images or video data can be retrieved from pre-recorded videos in real-time scanning or storage systems. If there is additional movement or video in the next frame of movement and display, data is retrieved in step 314. Thereafter, the process is repeated, and the characters are displayed and positioned according to the next frame in steps 308 to 310. This process can be repeated until the entire video is played or a certain allocated period for the visitors expires, and the process ends in step 316.

[0059] Steps 310-316 may also be accompanied by audio or sound that is synchronized with the image display and movement of the giant. In some embodiments, the audio is from a user or visitor communicating with the giant. In other embodiments, the audio may be part of a recorded video segment such as from a speech.

[0060] Figure 4 An exemplary embodiment of a structure scanning module 400 for generating an illumination matrix map according to an embodiment of the present invention is shown. The structure scanning module 400 may be operable to generate a 3D virtual model of the structure 109 by mapping the positions of LEDs 410 positioned on the surface of the structure and the order in which each illumination element is addressed by the electronic illumination control system 420. The structure scanning module 400 may be configured to generate a 3D virtual model of the structure by performing a series of steps, which may include:

[0061] a) operating the electronic lighting control system 420 to cycle through the LEDs 410 one at a time;

[0062] b) operating a scanning device 430 configured to scan the structure 109 from different angles, the scanning device being configured to detect light emitted by each powered LED 410;

[0063] c) constructing a point cloud from the detected light emitted from each LED 410, each point defining the location of the LED 410 on the surface of the structure 109 and the order in which they are addressed by the electronic lighting control system 420; and

[0064] d) Converting the point cloud into a 3D virtual model of the structure 109 .

[0065] The structure scanning module 400 may include a triggering software module 450 configured to synchronously trigger the operation of the electronic lighting control system 420 and the scanning equipment 430 .

[0066] The scanning equipment used to scan the structure may include known scanning equipment 430, such as lidar and raster scanning systems, which may operate via a wireless or wired communication data network 440. For example, a lidar scanner may be arranged to capture the spatial position of LEDs 410, while a raster scanner may be arranged to identify LEDs 410 as bright spots. It should be noted that any other known scanning equipment may be used to generate a point cloud representing the positions of addressable lighting elements on a structure.

[0067] The content of the lighting element matrix, movement, and audio can be generated in a real-time or quasi-real-time interactive program that uses a video stream from a camera and a facial detection algorithm to capture images and video, such as the movement and sound of a person or face. These images are then processed to obtain an abstract image or motion picture that is mapped onto the lighting element matrix. The mapping can be implemented using known coding environments and techniques, including visual field programming environments such as the freely available MaxMSP / Max 7.1 (https: / / cycling74.com / products / max / ) or vvvv (https: / / vvvv.org). Similarly, as discussed above, scanned or computer-generated images, videos, and / or audio can be loaded onto a flash memory data storage card inserted into a media player to be mapped to the lighting element matrix.

[0068] See also Figure 5 , presents an exemplary data flow for building a 3D virtual data model. The structure scanning module 400 operates the electronic lighting control system 420 to energize one of the LEDs 410 on the surface of the structure. The electronic lighting control system 420 energizes the LED 410 according to the addressable LED mapping that can be stored in the lighting control system 420, and confirms to the structure scanning module 400 that the LED is turned on. Once the confirmation is received from the electronic lighting control system 420, the structure scanning module 400 triggers the scanning equipment 430 to perform a lidar and raster scan. The scanning equipment 430 performs a lidar and raster scan upon receiving the trigger command, stores the obtained raster and 3D point model, and transmits the stored model to the structure scanning module 400. Once the model is received at the structure scanning module 400, the transmitted scanned data is processed to identify the LED location on the structure 109, and a signal is transmitted to the electronic lighting control system 420 to turn off the energized LED 410. Once confirmation has been received from the electronic lighting control system 420 that the LEDs 410 have been turned off, the structure scanning module 400 adds the detected LED positions to the 3D model and the mapping model of the structure. The process is repeated until the electronic lighting control system 420 has energized all or at least the desired number of LEDs 410 on the structure.

[0069] The method for building a 3D virtual data model may be performed at regular intervals, for example once a day or more frequently if necessary. In this way, faulty addressable lamp elements may be quickly identified and repaired.

[0070] In general, the routines executed to implement the embodiments of the present invention, whether implemented as part of an operating system or a specific application, component, program, object, module or sequence of instructions, or even as a subset thereof, may be referred to herein as "computer program code" or simply "program code". Program code typically includes computer-readable instructions that reside in various memories and storage devices in a computer at different times, and which, when read and executed by one or more processors in a computer, cause that computer to perform the operations and / or elements required to implement the various aspects of the embodiments embodying the present invention. The computer-readable program instructions for performing the operations of the embodiments of the present invention may be, for example, in assembly language, or source code or object code may be written in any combination of one or more programming languages.

[0071] The program code embodied in any of the applications / modules described herein can be distributed as a program product in a variety of different forms, either individually or collectively. Specifically, the program code can be distributed using a computer-readable storage medium having computer-readable program instructions for causing a processor to perform aspects of the embodiments of the present invention.

[0072] Inherently non-transitory computer-readable storage media may include volatile and non-volatile and removable and non-removable tangible media implemented in any method and technology for storing information such as computer-readable instructions, data structures, program modules or other data. Computer-readable storage media may also include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state memory technology, portable compact disc read-only memory (CD-ROM) or other optical storage devices, cassettes, magnetic tapes, magnetic disk storage devices or other magnetic storage devices, or any other medium that can be used to store the desired information and can be read by a computer. The computer-readable storage medium itself should not be understood as a transient signal (e.g., a radio wave or other propagating electromagnetic wave, an electromagnetic wave propagated through a transmission medium such as a waveguide, or an electrical signal transmitted through a wire). Computer-readable program instructions can be downloaded from the computer-readable storage medium to a computer, another type of programmable data processing device, or another device, or downloaded to an external computer or external storage device via a network.

[0073] Computer-readable program instructions stored in a computer-readable medium can be used to direct a computer, other type of programmable data processing device, or other apparatus to function in a particular manner, such that the instructions stored in the computer-readable medium produce an article of manufacture including instructions that implement the functions / actions specified in the flowchart, sequence diagram, and / or block diagram. The computer program instructions can be provided to one or more processors of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine, such that the instructions executed via the one or more processors cause a series of calculations to be performed to implement the functions and / or actions specified in the flowchart, sequence diagram, and / or block diagram.

[0074] In a specific alternative embodiment, without departing from the scope of the present invention, the functions and / or actions specified in the flow chart, sequence diagram and / or block diagram may be rearranged, processed continuously and / or processed simultaneously. In addition, any one of the flow chart, sequence diagram and / or block diagram may include more or less blocks than those depicted in accordance with the embodiments of the present invention.

[0075] The terms used herein are only for the purpose of describing a specific embodiment and are not intended to limit the embodiments of the present invention. As used herein, unless the context clearly indicates otherwise, it is hoped that the singular forms "one" and "described" also include plural forms. It will be further understood that the terms "comprises" and / or "comprising" specify the existence of stated features, integers, steps, operations, elements and / or parts when used in this specification, but do not exclude the existence or addition of one or more other features, integers, steps, operations, elements, parts and / or their groups. In addition, with regard to the use of the terms "includes", "having", "has", "with", "comprised of" or its variants in the specific embodiments or claims, it is hoped that such terms are similar to the term "comprising" to be inclusive.

[0076] Although the present invention has been described in its entirety by describing various embodiments, and although these embodiments have been described in great detail, applicants do not intend to restrict or in any way limit the scope of the appended claims to such details. Additional advantages and modifications will be readily apparent to those skilled in the art. The broader aspects of the present invention are therefore not limited to the specific details, representative devices and methods, and illustrative examples shown and described. Therefore, these details may be departed from without departing from the spirit or scope of the applicant's general inventive concept.

Claims

1. A visual modeling system, the visual modeling system comprising: a first scan data set, the first scan data set being associated with a subject; A structure (109) having a plurality of addressable lighting elements (410); a second scan data set associated with a location of the addressable lighting elements (410) on a surface of the structure (109); A processor-based image processing system (206), the processor-based image processing system comprising: a data receiving module, the data receiving module is used to receive the generated first scan data set; and a mapping engine, the mapping engine is used to associate the first scan data set with addressable lighting elements (410) corresponding to locations on the structure (109) defined by the second scan data set; and an electronic lighting control system (420) comprising a power source and a lighting controller in communication with a plurality of said addressable lighting elements (410), wherein said electronic lighting control system (420) energizes said addressable lighting elements (410) based on a mapping engine relationship between said first scan data set and said second scan data set; The visual modeling system comprises a structure scanning module (400) operable to generate the second scan data set by constructing a 3D virtual model of the structure (109), the 3D virtual model defining the positions of the addressable lighting elements (410) and the order in which the electronic lighting control system addresses the addressable lighting elements, The structure scanning module (400) is configured to construct the 3D virtual model of the structure by performing the following steps: a) operating the electronic lighting control system (420) to cycle through the addressable lighting elements (410) one at a time; b) operating a scanning device (430) of the visual modeling system, the scanning device being configured to scan the structure (109) from different angles, the scanning device (430) being configured to detect light emitted by each energized addressable lighting element (410); c) constructing a point cloud from the detected light emitted from each addressable lighting element (410), each point defining the location of the addressable lighting element (410) on the surface of the structure (109) and the order in which the addressable lighting elements are addressed by the electronic lighting control system (420); and d) converting the point cloud into a 3D virtual model of the structure (109).

2. The system of claim 1, wherein steps a) and b) are performed synchronously such that when each addressable lighting element (410) is powered on, the scanning device (430) is operated simultaneously.

3. The system of claim 2, wherein said steps a) and d) are performed by an executable computer program loaded into a memory of a processor.

4. The system of claim 1, wherein the scanning equipment (430) includes at least one rotatable scanner.

5. A system as described in claim 1, wherein the scanning equipment (430) includes at least one lidar scanner arranged to identify the spatial position of the addressable lighting element (410), and at least one raster scanner arranged to identify the addressable lighting element (410) as a bright spot.

6. A system as described in claim 5, wherein the electronic lighting control system (420) is configured to compare the generated 3D virtual model with an expected addressable lighting element map of the structure to identify the presence of faulty addressable lighting elements (410) on the surface of the structure that cause differences in luminous uniformity.

7. The system of claim 6, wherein the electronic lighting control system (420) is configured to generate a list containing the detected locations of the faulty addressable lighting elements, the list being communicated to a system administrator via a communication network.

8. The system of claim 6, wherein the electronic lighting control system (420) is configured to adjust operation of addressable lighting elements adjacent to affected structural areas when illumination level and / or color non-uniformity is detected.

9. The system of claim 8, wherein the electronic lighting control system (420) is configured to adjust the current supplied to each addressable lighting element.

10. The system of claim 8, wherein the electronic lighting control system (420) is configured to adjust the pulse width supplied to each addressable lighting element.

11. The system of claim 1, wherein the structure (109) is formed in the general shape of the type of the subject being scanned.

12. The system of claim 1, wherein the scan data set is a three-dimensional set of data and the structure (109) is formed into a general three-dimensional shape of the type of the subject being scanned.

13. The system of claim 1, wherein the mapping engine is configured to map 3D point locations defined in the second scan data set to 2D image points defined in the first scan data set.

14. The system of claim 13, wherein the mapping is performed by UV mapping techniques.

15. The system of claim 1, further comprising a three-dimensional scanner (202) for scanning a subject for visual modeling.

16. The system of claim 15, wherein the subject is a human being and the structure (109) is formed in the shape of a human figure.

17. The system of claim 16, wherein the structure formed in the shape of a human figure comprises at least one of a movably attached head, neck, arms, torso, or legs and an electric actuator for moving at least one of the head, neck, arms, torso, or legs.

18. A system as claimed in claim 17, wherein the system includes at least one motion sensor and a motion processing engine, wherein the motion processing engine is used to translate the movement of the subject into movement commands sent to at least one actuator, whereby a user can control the movement of the structure (109).

19. The system of claim 1, wherein the structure (109) is at least ten times larger than the subject being scanned.

20. The system of claim 1, wherein the scanned data set is a video, and wherein the electronic lighting control system energizes the addressable lighting elements to generate a moving image on the structure (109).

21. The system of claim 17, wherein the addressable lighting elements are outward-facing light emitting diodes arranged in a plurality of vertically spaced horizontal rows.

22. The system of claim 21, wherein the head and a front portion of the torso have a higher density of light emitting diodes than the rest of the body.

23. The system of claim 22, wherein a facial portion of the head has a higher density of light emitting diodes (410) than a frontal portion of the head and torso.

24. The system of claim 1, wherein the structure (109) comprises a plurality of interconnected blocks, each interconnected block comprising at least one addressable lighting element (410).

25. The system of claim 24, wherein the interconnected blocks are in the form of pixel tiles.

26. The system of claim 24 or 25, wherein each interconnect block comprises an interconnect element configured to releasably connect each interconnect block to each adjacent interconnect block.

27. A method for operating the visual modeling system according to claim 1, characterized in that: The method comprises the steps of generating a second scan data set by constructing a 3D virtual model of the structure using the structure scanning module (400), the 3D virtual model defining the positions of the addressable lighting elements (410) and the order in which the electronic lighting control system addresses the addressable lighting elements, the method comprising the steps of: a) operating an electronic lighting control system using a structure scanning module (400) to energize one of a plurality of addressable lighting elements (410) on a surface of a structure according to an addressable lighting element map stored in a memory of the electronic lighting control system (420), wherein the electronic lighting control system (420) confirms to the structure scanning module (400) that the addressable lighting element (410) is energized; b) upon receiving the confirmation at the structure scanning module (400), operating at least one scanning device (430) to generate a 3D point model of the structure, c) processing the 3D point model of the structure by the structure scanning module (400) to identify the locations of the energized addressable lighting elements (410) on the surface of the structure; d) transmitting a signal to the electronic lighting control system (420) to turn off the energized addressable lighting elements (410); e) adding, by the structure scanning module (400), the detected locations of the addressable lighting elements to the 3D point model and mapping model of the structure once confirmation has been received from the electronic lighting control system (420) that the addressable lighting elements (410) are turned off; and f) repeating steps a) to e) for the remaining addressable lighting elements (410), Wherein step b) comprises the following steps: b1) storing the scanned model; and b2) transmitting the stored model to the structure scanning module (400) for further processing; The method further comprises the following steps: g) receiving the generated first scan data set; h) correlating the first scan data set with the addressable lighting elements (410) corresponding to the locations; and i) energizing the addressable lighting elements (410) based on a mapping engine relationship between the first scan data set and the second scan data set.

Images