Lighting effect playback device and lighting effect playback control method thereof and corresponding device and medium

Abstract

The present application relates to a lighting effect playback device and a lighting effect playback control method thereof and corresponding devices and media, the method comprising: obtaining identification information of each of a plurality of lamps, determining the type of lamp and lamp bead layout information of the lamp according to the identification information of the lamp, wherein at least some lamp types are different from each other; receiving a video stream, and based on the position information of each lamp in the actual physical space and the lamp bead layout information, performing regional segmentation on the target image frame determined in the video stream, and respectively determining the sub-lamp bead layout information of the target lamp corresponding to each segmented region; determining the feature label of each region; and generating a lighting effect playback instruction corresponding to the type of lamp to which the target lamp belongs according to the feature label of each region and the sub-lamp bead layout information corresponding to each region. The present application is used to control various types of lamps to display corresponding lighting effects according to the visual dynamic characteristics of the video stream.

Description

Technical Field

[0001] The present application relates to the field of lighting control, and in particular to a lighting effect playback device and a lighting effect playback control method thereof, and corresponding devices and media. Background Art

[0002] Smart lamps can decorate interior spaces and display information. As people's quality of life improves, smart lamps are no longer just meeting basic lighting needs, but are gradually developing towards personalization and interactivity. Specifically, smart lamps achieve complex lighting effects through programming control to enhance user experience and environmental atmosphere.

[0003] In the prior art, smart lamps are usually connected to display devices in communication to display corresponding lighting effects according to the visual dynamic characteristics of the video stream in the display device. For example, when the video stream plays a concert, multiple smart LED bulbs of the same type can change color and brightness synchronously to match the rhythm of the music. When the music tempo speeds up, the lamps can flash quickly or change color; when the music becomes softer, the lamps can slowly change color or dim the brightness. Although this method improves the visual effect to a certain extent, it still has obvious limitations. Specifically, the prior art mainly relies on a single type of lamp to realize the corresponding lighting effects of display devices, which limits the diversity and flexibility of lighting effects, leads to limited user experience and difficulty in meeting personalized needs. Summary of the invention

[0004] The primary purpose of the present application is to solve at least one of the above problems and to provide a lighting effect playback device and a lighting effect playback control method thereof and corresponding devices and media.

[0005] In order to meet the various objectives of this application, this application adopts the following technical solutions:

[0006] A lighting effect playback control method provided to meet one of the purposes of the present application includes the following steps:

[0007] Obtain identification information of each lamp from a plurality of lamps, and determine the type of lamp and lamp bead layout information of the lamp according to the identification information of the lamp, wherein the lamp type is used to indicate the type of modeling freedom corresponding to the lamp bead layout of the lamp, wherein at least some of the lamp types are different from each other;

[0008] Receive a video stream, and based on the position information of each lamp in the actual physical space and the lamp bead layout information, perform region segmentation on the target image frame determined in the video stream, and respectively determine the sub-lamp bead layout information of the target lamp corresponding to each segmented region;

[0009] Determining feature labels of each region, wherein the feature labels represent visual dynamic characteristics of each region in the target image frame;

[0010] According to the characteristic tags of each area and the sub-lamp bead layout information corresponding to each area, a lighting effect playback instruction corresponding to the type of lamp to which the target lamp belongs is generated, and the lighting effect playback instruction is sent to the corresponding lamp to play the corresponding lighting effect.

[0011] A lighting effect playback control device is proposed to meet one of the purposes of the present application. The lighting effect playback control method includes:

[0012] an identification information acquisition module, configured to acquire identification information of each of the plurality of lamps, and determine the type of lamp and lamp bead layout information of the lamp according to the identification information of the lamp, wherein the lamp type is used to indicate the type of modeling freedom corresponding to the lamp bead layout of the lamp, wherein at least some of the lamp types are different from each other;

[0013] A region segmentation module is configured to receive a video stream, and based on the position information of each lamp in the actual physical space and the lamp bead layout information, perform region segmentation on the target image frame determined in the video stream, and respectively determine the sub-lamp bead layout information of the target lamp corresponding to each segmented region;

[0014] A feature label determination module, configured to determine a feature label of each region, wherein the feature label represents a visual dynamic characteristic of each region in a target image frame;

[0015] The lighting effect generation module is configured to generate a lighting effect playback instruction corresponding to the type of lighting fixture to which the target lighting fixture belongs according to the feature labels of the respective areas and the sub-lamp bead layout information corresponding to the respective areas, and send the lighting effect playback instruction to the corresponding lighting fixture to play the corresponding lighting effect.

[0016] On the other hand, a lighting effect playback device is provided to meet one of the purposes of the present application, including a controller and multiple lamps, wherein the controller is used to execute the steps of the lighting effect playback control method to control the multiple lamps to display corresponding lighting effects in a linked manner according to the visual dynamic characteristics of the video stream.

[0017] On the other hand, a computer-readable storage medium is provided to meet another purpose of the present application, which stores a computer program implemented according to the lighting effect playback control method in the form of computer-readable instructions. When the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

[0018] The technical solution of this application has many advantages, including but not limited to the following aspects:

[0019] This application obtains the identification information of each lamp and determines the type of lamp it belongs to and its lamp bead layout information to distinguish different types of lamps, providing a basis for the subsequent generation of lighting effects that match the type of lamp to which the lamp belongs. At the same time, the position of the lamp in the actual physical space is mapped to the virtual matrix of the graphical user interface, and a reference coordinate system is set to accurately obtain the coordinate information of each lamp in the virtual matrix. Using the acquired coordinate information and lamp bead layout information, the target image frame in the video stream is segmented into regions, and the lamp and lamp bead layout information corresponding to each region is determined to ensure that each lamp can display corresponding lighting effects according to the visual dynamic characteristics of its corresponding region. By determining the feature labels of each region, and based on the feature labels of each region and the corresponding lamp and lamp bead layout information, a lighting effect playback instruction that matches the lamp type is generated, and the instruction is sent to the corresponding lamp for lighting effect playback, providing users with a more immersive and dynamic visual experience and enhancing the overall viewing effect.

[0020] The lighting effect playback control method of the present application controls different types of lamps to correspond to one or more areas divided in the video stream, so as to control each lamp to collaboratively play the lighting effects corresponding to the visual dynamic characteristics of each area in the video stream. This method realizes the collaborative control of multiple types of lamps and enhances the user's visual experience by displaying corresponding lighting effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0022] Figure 1 A schematic diagram of the electrical structure of the lighting effect playback device in an embodiment of the present application;

[0023] Figure 2 The structural form of multiple lamps in the exemplary lighting effect playing device of this application;

[0024] Figure 3 A flowchart of a typical embodiment of the lighting effect playback control method of the present application;

[0025] Figure 4 This is a principle block diagram of the lighting effect playback control device of the present application;

[0026] Figure 5 A schematic diagram of the structure of a computer device used in this application. DETAILED DESCRIPTION

[0027] See also Figure 1As can be seen from the structural schematic diagram of a lighting effect playback device provided in the present application, the lighting effect playback device includes a controller 1 and a plurality of lamps 2. In some embodiments, the lighting effect playback device also includes a display device 3. The display device 3 can be any device capable of displaying a video stream, such as a television, a projector, an LED screen, and the like. The controller 1 is communicatively connected to the display device 3 for receiving and processing the video stream. In an embodiment of the present application, the controller 1 divides the target image frame in the video stream into a plurality of independent regions according to a preset segmentation rule, and each lamp corresponds to one or more regions 2, and the controller 1 can accurately identify the visual dynamic characteristics of each region.

[0028] The types of multiple lamps 2 are the same or different. Multiple lamps 2 are respectively connected to the controller 1 in communication so as to be controlled by the computer program running in the controller 1 and work together to realize the lighting effect playback. Each lamp 2 is connected to the controller 1 through a wireless or wired network. Each lamp 2 is usually composed of one or more light-emitting units, and each light-emitting unit of the lamp 2 is uniformly controlled by a control chip. The controller 1 obtains the lamp bead layout information and lamp type of multiple lamps 2, combines the visual dynamic characteristics of each area and the lamp bead layout information and lamp type of the lamp 2 corresponding to the area, and generates a lighting effect playback instruction that matches the lamp type. The controller 1 sends these lighting effect playback instructions to the corresponding lamp 2 to ensure that the lamp 2 can collaboratively play the lighting effects corresponding to the visual dynamic characteristics of each area in the video stream.

[0029] Based on the type of styling freedom corresponding to the layout of the lamp beads, lamps 2 are classified into first type lamps, second type lamps and third type lamps, wherein the first type lamps consist of a single lamp bead or multiple lamp beads concentrated together to form a light-emitting point (point light source), that is, the type of styling freedom exhibited tends to be a point light source type, for example, a bulb lamp. The second type of lamp is composed of multiple point light sources arranged along a line, and the formed linear light source is a one-dimensional degree of freedom type (for example, a light strip), or a two-dimensional degree of freedom type composed of multiple linear light sources (the linear light sources are on the same plane, for example, an atmosphere light strip set along the periphery of the TV) or a three-dimensional degree of freedom type (the linear light sources are on different planes, for example, a three-dimensional decorative lamp composed of linear light strips). At this time, the shape freedom type of the lamp tends to be a linear light source type; the third type of lamp is a planar light source composed of multiple point light sources or multiple linear light sources, that is, a two-dimensional degree of freedom type formed by an arrangement on the same plane (for example, a spliced ​​lamp located on the same plane), or a three-dimensional degree of freedom type composed of multiple point light sources or multiple linear light sources. For example, a three-dimensional dragon is arranged using multiple point light sources or multiple linear light sources. At this time, the shape freedom type of lamp 2 tends to be a planar light source type or a three-dimensional arrangement degree of freedom type.

[0030] In some embodiments, the controller 1 includes a control chip, a control panel, and a display screen. The control chip can be implemented using various embedded chips, such as Bluetooth SoC (System on Chip), WiFi SoC, MCU (MicroController Unit), DSP (Digital Signal Processing), and other types of chips. The control chip usually includes a central processing unit and a memory, and the memory and the central processing unit are used to store and execute program instructions, respectively, to achieve corresponding functions. The control panel usually provides one or more buttons for implementing switch control of the controller 1, and can also set a lamp selection control in the display screen of the controller 1 to enable the user to manually select the corresponding lamp for each area in the video stream, etc. The display screen can be used to display various control information, so as to cooperate with the buttons in the control panel to support the implementation of human-computer interaction functions. In some embodiments, the control panel and the display screen can be integrated into the same touch display screen.

[0031] The present application provides a lighting effect playback control method, wherein a controller 1 can obtain identification information of a plurality of connected lamps 2, thereby determining the lamp type and lamp bead layout information of each lamp 2, and at the same time, based on the actual physical space position of each lamp 2 connected to the controller 1, map each lamp 2 to a graphical user interface virtual matrix provided by a control panel to obtain coordinate information of each lamp 2, and then perform regional segmentation on a target image frame determined in a video stream according to the coordinate information of each lamp 2 and preset segmentation rules, and use a trained convolutional neural network model to determine feature labels of each region, and finally, generate a lighting effect playback instruction corresponding to the lamp type to which the matching lamp belongs according to the feature labels of each region in the video stream and the lamp and lamp bead layout information corresponding to each region, and send the lighting effect playback instruction to its corresponding lamp to play the corresponding lighting effect.

[0032] exist Figure 2In the scene shown, there are many different types of lamps 2, including a first downlight 21, a first light strip 22, a second downlight 23, a spliced ​​lamp 24, a curtain lamp 25, a first floor lamp 26, a second floor lamp 27 and a second light strip 28. In addition, a display device 3 is provided in the scene, and the display device 3 is divided into 9 areas, wherein each lamp 2 corresponds to one or more areas, for example, the first downlight 21 and the first light strip 22 correspond to area 1. It should be noted that the first light strip 22 can correspond to area 1 and area 2 at the same time. At this time, according to this typical embodiment, each area in the display device 3 is monitored in real time. For example, when it is detected that the visual dynamic characteristics in area 1 (the area corresponding to the first downlight 21 and the first light strip 22) change, the controller 1 will identify the feature label of the area according to the trained convolutional neural network model, and generate a lighting effect playback instruction matching the visual dynamic characteristics of the area in combination with the lamp bead layout information of the first downlight 21 and the lamp type. The controller 1 sends these lighting effect play instructions to the first downlight 21 and the first light strip 22, so that they can play the lighting effects corresponding to the visual dynamic characteristics of area 1 in the video stream in real time. For example, when the visual dynamic characteristics in area 1 (the area corresponding to the first downlight 21 and the first light strip 22) are detected to change from the original static blue to dynamic red flashing, the controller 1 will identify the characteristic labels of the area as "red", "flash", "0.1s / time" according to the trained convolutional neural network model, and the format of the characteristic label can be freely set. Combined with the lamp bead layout information and the lamp type (point light source type) of the first downlight 21, the controller 1 generates the corresponding lighting effect play instruction, which is described as "first downlight 21, first light strip 22, red flashing 0.1s / time mode". The controller 1 sends the lighting effect play instruction to the first downlight 21 and the first light strip 22 through a wireless or wired network, so that they can play the red flashing lighting effect corresponding to the visual dynamic characteristics of area 1 in the video stream in real time.

[0033] In some embodiments, the controller 1 of the present application can be implemented in an independent computer device, as long as the computer device is equipped with the corresponding hardware of the controller 1, and the corresponding business logic of the controller 1, including the business logic executed by the corresponding method of the present application, is implemented as a computer program installed in the computer device for operation. When the controller 1 is implemented in a computer device, various resources inherent to the computer device can be shared to save the overall implementation cost. The computer device referred to here can be any terminal device for use by the user, such as a smart phone, a personal computer, a laptop computer, a tablet computer, etc.

[0034] According to the product architecture and working principle of the above lighting effect playback device, the lighting effect playback control method of the present application can be implemented as a computer program, stored in the storage medium of the controller of the lighting effect playback device, and called from the storage medium by the control chip of the controller to run, so as to control each lamp 2 in the entire lighting effect playback device to play the corresponding sub-lighting effect. The lighting effect playback control method of the present application can be implemented as a computer program, stored in the storage medium of each lamp 2 of the lighting effect playback device, and called from the storage medium by the corresponding control chip of each lamp 2 to run, so as to control each lamp 2 to play the corresponding sub-lighting effect.

[0035] In some embodiments, combined with Figure 3 , the lighting effect playback control method of the present application includes:

[0036] Step S5100, obtaining identification information of each lamp among a plurality of lamps, and determining the type of lamp and lamp bead layout information of the lamp according to the identification information of the lamp, wherein the lamp type is used to indicate the type of modeling freedom corresponding to the lamp bead layout of the lamp, wherein at least some of the lamp types are different from each other;

[0037] The lamp bead layout information includes information such as the number of lamp beads, arrangement mode, and serial position. The type of styling freedom (lamp type) corresponding to the lamp bead layout of the lamp is determined by the lamp bead layout information, wherein the lamp bead layout may include at least one type of styling freedom. For example, a single lamp bead or multiple lamp beads gathered together to form a light-emitting point has a modeling freedom type that tends to be a point light source type (zero-dimensional freedom type); a linear light source formed by multiple point light sources arranged along a line exhibits a one-dimensional freedom type, or a two-dimensional freedom type (the linear light sources are on the same plane) or a three-dimensional freedom type (the linear light sources are on different planes) composed of multiple linear light sources. In this case, the modeling freedom type of the lamp tends to be a linear light source type; a planar light source composed of multiple point light sources or multiple linear light sources, that is, a two-dimensional freedom type arranged on the same plane, or a three-dimensional freedom type composed of multiple point light sources or multiple linear light sources in a three-dimensional arrangement. For example, multiple point light sources or multiple linear light sources are used to arrange a three-dimensional dragon. In this case, the modeling freedom type of the lamp tends to be a planar light source type or a three-dimensional arrangement freedom type.

[0038] The type of lamp to which the lamp belongs and the layout information of the lamp beads can be determined by the identification information of the corresponding lamp. In one embodiment, by using the data such as the lamp type, the layout information of the lamp beads, the production batch and the network configuration, a unique identification code is generated by a hash algorithm. The unique characteristics of the unique identification code can be used to identify the identity of the lamp, that is, the identification information of the lamp can include the type of lamp to which the lamp belongs, the layout information of the lamp beads and the unique identification code. Further, the type of modeling freedom (lamp type) corresponding to the lamp bead layout of the lamp can be determined by the lamp model. Specifically, when designing the lamp model, certain coding rules will be followed, which are usually composed of specific prefixes, suffixes or intermediate codes to indicate the type of lamp to which the lamp belongs, functions or other characteristics. In one embodiment, a database containing all lamp models and their corresponding lamp type information is usually provided. When the lamp model is received, the database is searched to determine the type of lamp corresponding to the model. In another embodiment, when the type of the lamp cannot be automatically identified according to the identification information of the lamp, or when it is desired to override the automatic identification result, the user is allowed to manually enter or select the type of the lamp from the preset options. The purpose is to avoid classification problems caused by missing or incorrect information about the lamp type corresponding to certain models in the database. Through manual configuration, ensure that each lamp is correctly classified. A drop-down menu can be set in the user interface to list all possible lamp types for users to directly select the type of lamp.

[0039] In another embodiment, a request may be sent to each lamp through a wireless communication protocol, and the built-in control chip of the lamp responds to the request to receive identification information including lamp type, lamp bead layout information, unique identification code, etc. In another embodiment, each lamp is equipped with a unique barcode or QR code, which contains detailed data of the identification information of the corresponding lamp. The user can use a terminal device to scan the barcode or QR code on the lamp to obtain the corresponding identification information of each lamp to determine the lamp type and lamp bead layout information of the lamp.

[0040] Step S5200, receiving a video stream, and based on the position information of each lamp in the actual physical space and the lamp bead layout information, performing region segmentation on the target image frame determined in the video stream, and respectively determining the sub-lamp bead layout information of the target lamp corresponding to each segmented region;

[0041] In one embodiment, the controller 1 and the display device are connected in communication, and the communication connection can be wired communication or wireless communication, which does not affect the embodiment of the creative spirit of the present application. The video stream can be captured using technical methods such as screen sharing software, video capture card or API call, and then the captured video stream is transmitted to the controller 1 by wired or wireless means. After the controller 1 receives the video stream, the decoder can be used to convert the compressed video data into an uncompressed target image frame. In addition, the video stream can also be a video stream stored on a storage device such as a disk, a hard disk drive, a CD-ROM, a storage area network (SAN), a web server, a server cluster, etc. The transmission and acquisition of the video stream can also be achieved by connecting the controller 1 to the storage device in communication. In another embodiment, a graphical user interface is provided in the display screen set in the controller, and a virtual matrix with a certain proportion to the actual physical space where the lamp is placed is provided on the graphical user interface control. A reference coordinate system is constructed in the virtual matrix, and the reference coordinate system can be used to quantify the position information of the lamp in the actual physical space. Specifically, after acquiring the target image frame in the video stream, in order to perform regional segmentation on the target image frame, firstly, the actual physical space where each lamp is located is scanned by a three-dimensional scanning device to acquire the corresponding spatial scanning data, which includes the geometric structure information and relative position information of each lamp. Then, based on the geometric structure information and relative position information in the spatial scanning data, each lamp can be mapped to the reference coordinate system of the virtual matrix of the graphical user interface of the controller 1. In this way, the position information of each lamp can be converted into the coordinate information in the reference coordinate system. Then, using the obtained coordinate information of each lamp and the preset segmentation rules, it is determined how many independent regions the target image frame determined in the video stream will be segmented into, and the boundary of each region is determined. In one embodiment, the target image frame determined in the video stream is directly divided into regular areas, and then the sub-lamp bead layout information of the target lamp corresponding to each area is determined based on the coordinate information of the lamp and the lamp bead layout information. If a lamp corresponds to multiple areas, the sub-lamp bead layout information is the lamp bead layout information corresponding to some of the lamp beads of the lamp, that is, one area corresponds to some of the lamp beads of the lamp, and the sub-lamp bead layout information is the lamp bead layout information of this part of the lamp beads.

[0042] It should be noted that the same lamp can correspond to one or more areas. Assuming that the target image frame determined in the video stream is divided into 9 independent areas, including area 1, area 2, ..., area 9, after the target image frame determined in the video stream is divided into each area, at this time, the first light strip 22 corresponds to area 1, area 2 and area 3, that is, the first light strip 22 is divided into three parts, one part corresponds to area 1, one part corresponds to area 2, and one part corresponds to area 3. If area 1 also corresponds to the first downlight 21, at this time, it can be determined that the lamp corresponding to the divided area 1 is the first downlight 21 and the first light strip 22. At the same time, it can be determined that the lamp bead layout information corresponding to area 1 includes the sub-lamp bead layout information of the first downlight 21, and the sub-lamp bead layout information of the first light strip 22 corresponding to some lamp beads in area 1. It can also be understood that the sub-lamp bead layout information of the first light strip 22 corresponding to some lamp beads in area 2 can be determined.

[0043] Step S5300, determining feature labels of the respective regions, wherein the feature labels represent visual dynamic characteristics of the respective regions in the target image frame;

[0044] Determine the feature labels of each area. In one embodiment, the feature labels include effect labels, direction labels, speed labels, and main color labels. According to the label mapping rules, the effect labels can be divided into explosion, lightning, flow, etc., the direction labels can be divided into from top to bottom, from left to right, from top right to bottom left, outward diffusion, inward gathering, etc., the speed labels can be flashing at 0.2 seconds per time, flowing at 1 meter per second, etc., the main color can be directly classified according to a single color, or according to a combination of colors, or according to a color range, such as warm colors (red, orange, yellow), cool colors (blue, green, purple), neutral colors (white, gray, black), etc. After defining the feature labels, the convolutional neural network model is trained using multiple video data sets to determine the feature labels of each area. Specifically, the trained convolutional neural network model is used to extract the optical flow information of each frame image and adjacent frames in each area of ​​the video stream. The optical flow information refers to the motion vector of the pixel points between adjacent frames in each target image frame of the video stream, which describes the displacement of each pixel point in the image in time, that is, it indicates the displacement of a pixel point from one frame to another frame, which can be used to capture the movement direction and speed of the object. Based on the optical flow information, the feature vector corresponding to each area is determined, and the feature label corresponding to the feature vector is determined according to the label mapping rule to represent the visual dynamic characteristics of each area in the target image frame. Among them, it is easy to understand that the visual dynamic characteristics refer to the dynamic visual effects and characteristics displayed by the target image frame in the video stream. These characteristics can be identified by analyzing the changes of the continuous target image frames determined in the video stream over time, such as the movement direction and speed of the object in the video stream. The video stream can be processed by using the convolutional neural network model to extract the optical flow information between multiple consecutive frames of images in real time, so as to determine the visual dynamic characteristics of each area, and these visual dynamic characteristics are characterized by feature labels. For example, in the feature labels of a region in a video stream of a display device, the effect label is explosion, the direction label is outward diffusion, the speed label is 1 meter per second, and the main color is blue.

[0045] Step S5400: Generate a lighting effect playback instruction corresponding to the type of lamp to which the target lamp belongs according to the feature tags of each area and the sub-lamp bead layout information corresponding to each area, and send the lighting effect playback instruction to the corresponding lamp to play the corresponding lighting effect.

[0046] First, the type of lighting effect required for each area is determined based on the feature tags of each area extracted from the video stream, including effect tags, direction tags, speed tags, and main color tags. For example, if the feature tags of an area are displayed as an "explosion" effect, an "outward diffusion" direction, a "1 meter per second" speed, and a "blue main color", the controller 1 will generate a corresponding lighting effect mode based on these feature tags.

[0047] By combining the sub-lamp bead layout information of the target lamps corresponding to each area, including the number of lamp beads, arrangement mode and serial position of some lamp beads corresponding to each area, the generated lighting effect is adjusted to ensure that the lighting effect can match the actual layout of the lamp. For example, for a lamp of a linear light source type, a lighting effect flowing along a linear path is generated according to the linear arrangement mode of its lamp beads; and for a lamp of a planar light source type, a lighting effect covering the entire plane is generated. In the process of generating lighting effects, the type of lamp to which the target lamp belongs is also considered to generate a lighting effect playback instruction that matches the type of lamp to which the corresponding lamp belongs. Finally, the generated lighting effect playback instruction is sent to the corresponding lamp through a wireless communication protocol or other communication methods. After receiving the lighting effect playback instruction, the built-in control chip of the lamp will control the on and off, color and dynamic changes of the lamp beads according to the lighting effect playback instruction, thereby realizing a lighting effect display that matches the visual dynamic characteristics of the target image frame in the video stream. When watching a video, the user can not only see the dynamic effects on the screen, but also get a more immersive visual experience through the lighting changes of the surrounding lamps.

[0048] It can be known from the typical embodiments of the present application that the technical solution of the present application has many advantages, including but not limited to the following aspects:

[0049] This application obtains the identification information of each lamp and determines the type of lamp it belongs to and its lamp bead layout information to distinguish different types of lamps, providing a basis for the subsequent generation of lighting effects that match the type of lamp to which the lamp belongs. At the same time, the position of the lamp in the actual physical space is mapped to the virtual matrix of the graphical user interface, and a reference coordinate system is set to accurately obtain the coordinate information of each lamp in the virtual matrix. Using the acquired coordinate information and lamp bead layout information, the target image frame in the video stream is segmented into regions, and the lamp and lamp bead layout information corresponding to each region is determined to ensure that each lamp can display corresponding lighting effects according to the visual dynamic characteristics of its corresponding region. By determining the feature labels of each region, and based on the feature labels of each region and the corresponding lamp and lamp bead layout information, a lighting effect playback instruction that matches the lamp type is generated, and the instruction is sent to the corresponding lamp for lighting effect playback, providing users with a more immersive and dynamic visual experience and enhancing the overall viewing effect.

[0050] The lighting effect playback control method of the present application controls different types of lamps to correspond to one or more areas divided in the video stream, so as to control each lamp to collaboratively play the lighting effects corresponding to the visual dynamic characteristics of each area in the video stream. This method realizes the collaborative control of multiple types of lamps and enhances the user's visual experience by displaying corresponding lighting effects.

[0051] Based on any embodiment of the method of the present application, obtaining identification information of each of the plurality of lamps, and determining the type of lamp and lamp bead layout information of the lamp according to the identification information of the lamp, including:

[0052] Step S5110, receiving a broadcast signal of the lamp;

[0053] The broadcast signal from each lamp can be directly received through wireless communication protocols (such as Zigbee, WiFi, Bluetooth, etc.), where the broadcast signal contains the identification information of each lamp, such as lamp type, lamp bead layout information, unique identification code, etc. The receiving process usually sets up one or more receivers that can capture the signal sent by the lamp at a specific frequency.

[0054] In one embodiment, a lamp search command can be sent to the lamp through a wireless communication protocol (such as Zigbee, WiFi, Bluetooth, etc.), or a search command can be sent directly to the connected lamp through a physical interface. After the search command is sent, all lamps within the communication range will be activated and start to send broadcast signals to notify their existence, usually by sending a broadcast signal containing lamp identification information. Then, by turning on the wireless receiving module to monitor and capture these broadcast signals, the lamp identity is identified and connected based on the identification information contained in the broadcast signal.

[0055] In another embodiment, the user triggers a lamp search operation on the graphical user interface of the terminal device. At this time, the terminal device sends a lamp search instruction through the WiFi network. The search instruction is a specific data packet designed to be recognized by all lamps connected to the same WiFi network. After all lamps within the communication range receive the search instruction, they will be activated and start sending broadcast signals containing their identification information. The terminal device can then use its WiFi receiving module to listen to and capture these broadcast signals for identifying the lamp identification information and connecting the lamp to the terminal device.

[0056] Step S5120: determining the identification information of the lamp according to the broadcast signal, wherein the identification information includes the lamp type, lamp bead layout information and a unique identification code;

[0057] In one embodiment, the terminal device uses a predefined protocol or data format to decode the broadcast signal received in the previous step, and extracts the identification information of the lamp, including the lamp type, lamp bead layout information, and unique identification code. The terminal device records the extracted lamp identification information and associates it with the corresponding lamp, so as to determine the lamp type and lamp layout to which it belongs according to the lamp identification information, and obtain the unique identification code of each lamp.

[0058] In another embodiment, the terminal device also receives and decodes the broadcast signal, and the obtained identification information includes a unique identification code, and then the type of the lamp is determined by the unique identification code. Specifically, the type information of the lamp is encoded into the unique identification code, and the terminal device determines the type of the lamp by parsing a specific part of the unique identification code or using a preset mapping table.

[0059] Step S5130: Identify and connect the lamp based on the unique identification code;

[0060] A unique identification code is a unique sequence, which refers to a number that uniquely identifies or manages a lamp. Each lamp is assigned a unique identification code to ensure that each lamp has a unique identification. The unique identification code can be generated by a hash algorithm based on data such as lamp type, lamp bead layout information, generation batch, and network configuration. Specifically, after receiving the broadcast signal of each lamp and determining its identification information, the terminal device will use the unique identification code extracted from the broadcast signal to identify and establish a connection with each lamp. Specifically, using the unique identification code as the identifier of each lamp, it is connected to a specific terminal device through a wireless communication protocol (such as Zigbee, WiFi, Bluetooth, etc.) or a wired connection. A stable communication link will be established between the terminal device and the lamp. The communication link allows the terminal device to send control instructions to the lamp, obtain the identification information of the lamp, etc., and perform other necessary communication operations. In this way, the terminal device can ensure that the connection with each lamp is accurate and reliable, providing a solid foundation for subsequent lamp control and management. Through this identification and connection mechanism based on a unique identification code, the terminal device can effectively manage multiple lamps and achieve centralized control and coordinated operation of the lamps.

[0061] Step S5140: The lamp bead layout information, lamp type and unique identification code of the lamp are stored in association.

[0062] Based on the communication link established between the terminal device and the lamp in the previous step, after receiving the identification information of the lamp, the terminal device will parse and process the identification information of the lamp to extract the information such as the lamp bead layout information, the lamp type and the unique identification code. Then, the terminal device will associate the lamp bead layout information with the lamp type and the unique identification code, and store the associated information in the terminal device database. That is, the terminal device can quickly retrieve the lamp bead layout information and the corresponding lamp type according to the unique identification code of the lamp.

[0063] In this embodiment, by receiving the broadcast signal of each lamp, all lamps within the communication range are automatically identified, which reduces manual operations and improves the identification efficiency. Then, by parsing the data packets in the broadcast signal, the identification information of the lamp is accurately extracted, including the lamp type, lamp bead layout information and unique identification code, etc., to ensure the accuracy of the information, and the identification information is stored in association with the lamp, which is convenient for subsequent data management and query, and improves the efficiency of data processing.

[0064] Using a unique identification code as an identifier ensures that the connection between the terminal device and the lamp is accurate and reliable, reducing the possibility of connection errors, and verifying through a unique identification code enhances the security of the connection and prevents unauthorized access and operation. The lamp bead layout information is stored in association with the lamp type and unique identification code, which facilitates quick retrieval and use, improving the efficiency of data management.

[0065] On the basis of any embodiment of the method of the present application, a video stream is received, and based on the position information of each lamp in the actual physical space and the lamp bead layout information, the target image frame determined in the video stream is segmented into regions, and the sub-lamp bead layout information of the target lamp corresponding to each segmented region is determined respectively, including:

[0066] Step S5210: Acquire spatial scanning data of the actual physical space where the multiple lamps are located, wherein the spatial scanning data includes geometric structure information and relative position information of the multiple lamps;

[0067] In one embodiment, the actual physical space where each lamp is located can be scanned by a three-dimensional scanning device to obtain corresponding spatial scanning data, including the geometric structure information of the lamps and the relative position information between them. Before starting the scan, ensure that the scanning environment is suitable for the operation of the device, for example, ensure that there is sufficient lighting and no obstructions. Then, operate the scanning device to perform a comprehensive scan of the space. During the scanning process, the device will record the geometric structure information of each lamp, including its size, shape and surface details, and will also record the relative positions of each lamp, such as distance and direction. After the scan is completed, the device will generate a three-dimensional point cloud data set containing the geometric structure information and relative positions of all lamps.

[0068] In one exemplary example of this embodiment, a laser scanner can be used to scan the actual physical space where each lamp is located. Specifically, a method for scanning lamps and surrounding environments in a large area or complex environment by installing a laser scanner on a mobile platform. A laser scanner installed on a mobile platform such as a cart is used to scan lamps and surrounding environments by moving in the actual physical space. Laser scanners are generally suitable for scanning large areas or complex environments. During the scanning process, the laser scanner emits laser beams, which are reflected in the space and received by the scanner. By analyzing the changes in the reflected laser beams, the scanner can calculate the distance and precise shape of the lamps in the actual physical space. The mobile platform moves step by step to cover the actual physical space where the entire lamp is located to ensure that all lamps are scanned. During the scanning process, the device records the geometric structure information of each lamp, including its size, shape and surface details, and also records the relative positions between the lamps, such as distance and direction. After the scan is completed, the device generates a three-dimensional point cloud data set containing the geometric structure information and relative positions of all lamps. The 3D point cloud dataset will then be imported into the virtual matrix of the graphical user interface to create a virtual 3D model that accurately reflects the layout of the lamps in the actual physical space. For the creation of the 3D model, please refer to the subsequent specific implementation method, which will not be repeated here.

[0069] In another exemplary example of this embodiment, an ultrasonic scanner can be used to scan the actual physical space where each lamp is located. The ultrasonic scanner is placed on a moving platform and moves in the actual physical space to scan the lamps and the surrounding environment in the same way as a laser scanner. During the scanning process, the ultrasonic scanner emits ultrasonic pulses, which are reflected back and received by the sensor when encountering an object. By measuring the time difference between the emission and reception of the ultrasonic pulse, the ultrasonic scanner can calculate the distance between the object and the ultrasonic scanner. After the scan is completed, the ultrasonic scanner generates a data set containing the location information of all lamps. The data set is then imported into the virtual matrix of the graphical user interface to create a virtual three-dimensional model that reflects the layout of the lamps in the actual physical space. For the creation of the three-dimensional model, please refer to the subsequent specific implementation methods, which will not be repeated here.

[0070] In another exemplary example of this embodiment, photogrammetry can be used to map each lamp to a virtual matrix of a graphical user interface, wherein photogrammetry refers to a method of reconstructing a three-dimensional space and an object by analyzing two-dimensional photos taken from multiple angles. Use a high-resolution camera (which can be a professional camera or a high-pixel smartphone camera) to take photos of lamps and actual physical spaces from multiple angles and heights, ensuring that the important feature points of each lamp are covered by multiple photos to improve the accuracy of three-dimensional reconstruction. When shooting, the position and angle of the camera should be distributed as evenly as possible to ensure that there is enough overlapping area between the photos, and then the captured photos are image matched to determine the correspondence between the feature points in the photos in different photos. Based on the results of image matching, the three-dimensional coordinates of each feature point are calculated, and a dense point cloud data set is generated, and an accurate three-dimensional model is formed based on the point cloud data set. Check the generated three-dimensional model to ensure that the position and shape of the lamp are consistent with those in the actual physical space. If the position and shape of the lamp are inconsistent with those in the actual physical space, specific parts of the three-dimensional model can be manually adjusted, or photos of certain areas can be retaken to improve the accuracy of the model. For the creation of the three-dimensional model, please refer to the subsequent specific implementation method, which will not be described in detail here.

[0071] Step S5220: Based on the geometric structure information and the relative position information, the lamp is mapped into a virtual matrix of a graphical user interface, wherein a reference coordinate system is provided in the virtual matrix and a plurality of point identifiers are used for filling;

[0072] First, create a virtual matrix in the graphical user interface, which is used to simulate the layout of the lamps in the actual physical space. The virtual matrix can be a two-dimensional or three-dimensional virtual space, depending on the distribution of the lamps in the actual physical space. For example, if all the lamps are located on the same plane, a two-dimensional matrix can be used; if the lamps are distributed on different planes, a three-dimensional matrix is ​​required.

[0073] In order to improve the friendliness and intuitiveness of the user interface, the virtual matrix is ​​filled with multiple point markers. These point markers can be simple dots or other visual elements. Through the setting and display of point markers, users can intuitively see the relative position and distribution of each lamp on the graphical user interface, making it easier to understand and operate.

[0074] While constructing the virtual matrix, a reference coordinate system is also set in the virtual matrix. The reference coordinate system usually includes the X-axis, Y-axis and Z-axis (if a three-dimensional matrix is ​​used) or only the X-axis and Y-axis (if a two-dimensional matrix is ​​used). The reference coordinate system is set to ensure that the position of each lamp in the virtual matrix can be accurately mapped to the position in the actual physical space. Specifically, the position information of each lamp in the actual physical space is converted into the coordinate value in the reference coordinate system. In this way, the position of the lamp seen by the user on the graphical user interface corresponds one-to-one to the position of the lamp in the actual physical space, thereby ensuring the accuracy and consistency of the lamp mapping. In this way, an intuitive and accurate virtual space model can be provided to the user, which is convenient for subsequent lamp management and control operations.

[0075] In one embodiment, a reference coordinate system is constructed in the virtual matrix to facilitate accurate positioning and adjustment of the lamp model. The process of constructing the reference coordinate system includes determining the origin of the coordinate system and selecting a point as the origin of the coordinate system. The origin can be the center of the virtual matrix or the position of the virtual matrix corresponding to each corner of the ground in the actual physical space. Then, along the direction of the lamp, two orthogonal coordinate axes are set: X-axis and Y-axis, or three orthogonal coordinate axes: X-axis, Y-axis and Z-axis, which represent the length, width or length, width and height in the virtual matrix respectively. Then, a unit length is defined for each axis in the coordinate system, for example, 1 unit may represent 1 meter or 1 centimeter, and the proportional relationship between the virtual matrix and the actual physical space is determined to ensure that the size of the three-dimensional model is consistent with that of the actual lamp. Finally, the constructed reference coordinate system is applied to the entire virtual matrix to ensure that all lamp models are positioned and adjusted using this coordinate system.

[0076] In another embodiment, the three-dimensional point cloud data set generated in the previous step is first obtained, and the three-dimensional point cloud data set records in detail the geometric structure information of the lamps, such as size, shape and surface details, and the relative position information between the lamps, such as distance and direction. Then the obtained three-dimensional point cloud data set is analyzed and processed to accurately extract the key features and position coordinates of each lamp. Then the processed data is imported into the reference coordinate system of the virtual matrix. In the virtual matrix, a three-dimensional model of each lamp is created based on the geometric structure information and relative position information of the lamp, and it is ensured that the position and direction of the three-dimensional model of each lamp in the virtual matrix are completely consistent with its position and direction in the actual physical space. Through this mapping process, the three-dimensional model seen by the user in the reference coordinate system of the virtual matrix can accurately reflect the layout of the actual lamps.

[0077] In another embodiment, the user can select one or more lamp models on the graphical user interface, and then use the move, rotate and scale tools provided by the graphical user interface to adjust the position, direction and size of the corresponding lamp model. The user can directly operate the model through a mouse or touch screen, or automatically adjust the model according to the parameters entered by the user. During the adjustment process, the user can refer to the layout of lamps in the actual physical space to ensure that the three-dimensional model of the lamp is completely consistent with the position and direction of the lamp in the actual physical space. In addition, auxiliary tools such as reference lines, grids and alignment tools can be provided to help with precise alignment. Through these adjustments, it is ensured that the position and direction of the three-dimensional model of each lamp in the virtual matrix are completely consistent with its position and direction in the actual physical space, so that the three-dimensional model of each lamp can accurately reflect the layout of lamps in the actual physical space, thereby providing users with an intuitive and accurate virtual matrix for easy management and control of lamps.

[0078] Step S5230, converting the relative position information into coordinate information in the reference coordinate system;

[0079] In one embodiment, the three-dimensional point cloud data set generated by the three-dimensional scanning device in the above embodiment is used to convert the position of each lamp in the actual physical space into coordinate information in the virtual matrix reference coordinate system. The three-dimensional scanning device emits a laser beam or a grating pattern, which is reflected in the space and received by the scanning device. By analyzing the changes in the reflected laser beam or grating pattern, the scanning device can calculate the distance and shape of the object in the space. After the scan is completed, the three-dimensional scanning device generates a three-dimensional point cloud data set containing the geometric structure and relative position of all lamps.

[0080] By processing the generated 3D point cloud data set, the position information of each lamp is extracted. To convert the position information of each lamp into the coordinate information in the reference coordinate system, it is necessary to determine the transformation matrix between the two coordinate systems. The transformation matrix contains transformation parameters such as rotation, translation, and scaling. By applying this transformation matrix, the position of each lamp in the actual physical space can be converted into the coordinates in the virtual matrix reference coordinate system. Through this step, the position information of each lamp is unified in the virtual matrix coordinate system.

[0081] In this embodiment, by calling a three-dimensional scanning device to scan the actual physical space, not only can the geometric structure and relative position of the lamps be accurately captured, but also a detailed three-dimensional point cloud data set is generated, providing an accurate data basis for subsequent modeling and control. An accurate three-dimensional model is created in the reference coordinate system of the virtual matrix using the three-dimensional point cloud data set, ensuring that the position and orientation of the three-dimensional model of each lamp in the virtual matrix are completely consistent with its position and orientation in the actual physical space, thereby enhancing the practicality and accuracy of the virtual matrix. The lamp position information obtained by the three-dimensional scanning device is converted into the reference coordinate system, achieving a seamless connection between the position information of the actual physical space and the virtual matrix, and providing accurate coordinate information for subsequent three-dimensional model adjustments.

[0082] Step S5240, receiving a video stream, and based on the coordinate information of each lamp and the lamp bead layout information, performing region segmentation on the target image frame determined in the video stream, and respectively determining the sub-lamp bead layout information of the target lamp corresponding to each segmented region.

[0083] Receive the video stream sent from the display device, which can be real-time video data or a pre-recorded video file. After receiving the video stream, extract the target image frame, which is a continuous frame in the video stream. Then, based on the coordinate information of each lamp in the actual physical space and the lamp bead layout information obtained in the previous step, use the pre-set segmentation rules to perform regional segmentation on the target image frame in the video stream.

[0084] Multiple segmentation rules can be preset and stored in the controller 1. The multiple segmentation rules are set based on different segmentation methods. In one embodiment, a certain type of lamp can be set as a whole. In this embodiment, when it is detected that the type of the lamp is the set type, the lamp is regarded as a whole, that is, the lamp only corresponds to one segmentation area of ​​the target image frame.

[0085] In another embodiment, segmentation can be performed according to the content of the video stream, that is, the segmented area of ​​the target image frame is irregularly segmented according to the content of the video stream. Specifically, after receiving the video stream, the target image frame is subjected to content analysis. The content analysis may include color distribution, brightness distribution, edge detection, etc., to identify different areas in the image. Based on the results of the content analysis, irregular segmented areas are dynamically generated. The segmented areas can be divided by irregular boundary lines, and for each segmented area, the lamp bead layout information is dynamically adjusted according to the content characteristics in the segmented area.

[0086] In another embodiment, the user can manually adjust the segmented area of ​​the target image frame by dragging the boundary line or setting the segmentation parameters to ensure that the layout of the lamp beads in each area matches the layout expected by the user. Specifically, after receiving the video stream, the target image frame is initially segmented according to the preset segmentation rules. The initial segmentation can be a rule-based segmentation based on the position of the lamp. The target image frame after the initial segmentation is displayed on the graphical user interface. The boundary line of each segmented area is displayed in the graphical user interface. The user can adjust the segmented area by dragging the boundary line or setting the segmentation parameters on the graphical user interface through a mouse or touch screen. According to the segmented area adjusted by the user, the lamp bead layout information is dynamically adjusted, and then the adjusted lamp bead layout information is applied to the corresponding lamp in the target image frame according to the coordinate information of the lamp.

[0087] The segmentation method is used to set segmentation rules to perform regional segmentation on the target image frame in the video stream, and then the sub-lamp bead layout information of the target lamp corresponding to each segmented area is determined.

[0088] On the basis of any embodiment of the method of the present application, after converting the relative position information into coordinate information in the reference coordinate system, the method includes:

[0089] Step S5250, responding to the sequential lighting test signal, and verifying whether the lighting sequence in the graphical user interface is consistent with the lighting sequence in the actual physical space based on the coordinate information of the lamp;

[0090] A sequential lighting test is performed using the coordinate information of the lamps in the virtual matrix of the graphical user interface, the purpose of which is to ensure that the lamps can respond correctly in a predetermined order. In one embodiment, a sequential lighting test signal is first sent, and the sequential lighting test signal contains sequential instructions for lighting the lamps. According to the coordinate information of the lamps in the virtual matrix, the position of each lamp in the actual physical space is determined, and the lamps are controlled to light up in sequence according to this position sequence. During the test, each lamp will light up in sequence according to a predetermined sequence, and the user can observe the lighting sequence of the lamps in the actual physical space to verify whether it is consistent with the sequence in the virtual matrix. If the lighting sequence of the lamps is consistent with expectations, it means that the response sequence of the lamps is correct, and the layout of the lamps in the virtual matrix matches the actual physical space. In this way, it is verified and ensured that the layout of the lamps in the virtual matrix of the graphical user interface is completely matched with the layout of the lamps in the actual physical space, thereby providing an accurate basis for the subsequent specific implementation steps.

[0091] Step S5260: When the lighting order in the graphical user interface is inconsistent with the lighting order in the actual physical space, the coordinate information of the lamp in the space scanning data is updated according to the position of the lamp after position adjustment in the virtual matrix of the graphical user interface or the actual physical space.

[0092] In one embodiment, when the lighting sequence in the graphical user interface is inconsistent with the lighting sequence in the actual physical space, the user can further adjust the position of the lamps in the virtual matrix, or check the connection and control settings of the lamps to ensure that the lamps can respond in the correct order. If it is found in the sequential lighting test that the lamps do not respond in the preset order, the user can select the lamps that do not respond correctly on the graphical user interface and use the moving tools provided by the interface to adjust the positions of these lamps. The adjustment includes moving the lamps to a new coordinate position or rotating the lamps to change their direction. After the adjustment is completed, the coordinate information of the lamps in the virtual matrix of the graphical user interface is automatically updated, providing an accurate basis for the subsequent specific implementation steps.

[0093] In another embodiment, when the lighting order in the graphical user interface is inconsistent with the lighting order in the actual physical space, the position of the lamps in the actual physical space is adjusted. That is, by physically moving the lamps, ensure that the position of each lamp matches the layout in the virtual matrix. After the adjustment is completed, it is necessary to reacquire the updated spatial scanning data. The adjusted lamp positions can be accurately scanned by using a three-dimensional scanning device, such as a laser scanner or an ultrasonic scanner, so as to obtain new geometric structure information and relative position information, and then the new spatial scanning data is imported into the virtual matrix of the graphical user interface, and the coordinate information of the lamps is updated based on the updated spatial scanning data to ensure that the lamp position information in the virtual matrix is ​​consistent with the latest layout in the actual physical space.

[0094] Step S5270: Based on the updated coordinate information of the lamp, re-issue the sequential lighting test signal for verification until the lighting sequence in the graphical user interface is consistent with the lighting sequence in the actual physical space.

[0095] The sequential lighting test signal is re-issued based on the updated fixture coordinate information. The fixtures will light up in the new order again, and the user can observe the lighting order of the fixtures in the graphical user interface to verify whether it is consistent with the lighting order in the actual physical space. If the lighting order of the fixtures is still incorrect, the user will continue to make adjustments and repeat the above process until all fixtures can correctly respond to the test signal in the preset order. This process ensures that the fixture layout in the virtual matrix fully matches the fixture layout in the actual physical space, thereby providing an accurate basis for the subsequent specific implementation steps.

[0096] In this embodiment, by performing a sequential lighting test, the response order of the lamps is verified using the lamp coordinate information in the virtual matrix of the graphical user interface, thereby ensuring the consistency between the virtual model and the actual physical space. If the lighting order is found to be inconsistent during the test, the deviation in the layout is corrected by adjusting the position of the lamps in the actual physical space or the virtual matrix and updating the space scan data. This step not only ensures a high degree of synchronization between the virtual matrix and the actual physical space, but also accurately reflects the actual position of the lamps by acquiring new coordinate information. Through repeated testing and adjustment until all lamps can correctly respond to the test signal in the preset order, the problem of incorrect lamp response order is solved, and finally it is ensured that the layout of the lamps in the virtual matrix is ​​completely matched with the layout of the lamps in the actual physical space, providing an accurate basis for the subsequent specific implementation steps.

[0097] On the basis of any embodiment of the method of the present application, based on the coordinate information of each lamp and the lamp bead layout information, the target image frame determined in the video stream is segmented into regions, including:

[0098] Step S5310: Obtain the coordinate information of the lamp, and determine the maximum and minimum values ​​of the coordinate values ​​of each dimension in the coordinate information;

[0099] Before performing region segmentation, you first need to obtain the coordinate information of each connected luminaire in the virtual matrix. This coordinate information may include the coordinate values ​​of the X-axis, Y-axis, and Z-axis (if a three-dimensional matrix is ​​used) or only the X-axis and Y-axis (if a two-dimensional matrix is ​​used). After obtaining this coordinate information, analyze it to determine the maximum and minimum values ​​of the coordinate values ​​of each dimension. Specifically, by traversing the coordinate information of all luminaires, the maximum and minimum values ​​on each dimension are recorded. For example, in a two-dimensional matrix, the maximum and minimum values ​​on the X-axis and Y-axis are recorded respectively; in a three-dimensional matrix, the maximum and minimum values ​​on the X-axis, Y-axis, and Z-axis are recorded respectively. By determining these maximum and minimum values, we can better understand the distribution range of luminaires in the virtual matrix, thereby providing basic data for subsequent region segmentation.

[0100] Step S5320, calculating the difference between the maximum value and the minimum value of the coordinate values ​​of each dimension;

[0101] After determining the maximum and minimum values ​​of the coordinate values ​​of each dimension, the next step is to calculate the difference between these maximum and minimum values. Specifically, for each dimension, calculate the difference between the maximum and minimum values ​​on that dimension. For example, in a two-dimensional matrix, calculate the difference between the maximum and minimum values ​​on the X-axis and Y-axis respectively; in a three-dimensional matrix, calculate the difference between the maximum and minimum values ​​on the X-axis, Y-axis, and Z-axis respectively. These differences reflect the distribution range of the lamps in each dimension, providing important reference data for subsequent area segmentation.

[0102] Step S5320: divide the target image frame determined in the video stream into a plurality of independent regions according to the difference, and the lamp corresponds to one or more regions.

[0103] After calculating the difference between the maximum and minimum values ​​of the coordinate values ​​of each dimension, these differences are used to perform regional segmentation on the target image frame determined in the video stream. In one embodiment, the difference between the maximum and minimum values ​​of the coordinate values ​​of each dimension and the number of segmented regions can be stored in a mapping table accordingly. The difference between the maximum and minimum values ​​of the coordinate values ​​of each dimension in the mapping table exists in the form of an interval. Specifically, the difference does not exist as a separate numerical point, but divides the coordinate values ​​of each dimension into several areas according to a specific interval within a certain range, and then combines the coordinate value intervals of each dimension as an entry of the mapping table. Among them, the number of segmented regions determined by the difference between the maximum and minimum values ​​of the coordinate values ​​of each dimension can be customized through experience or experimental data. Specifically, after determining the maximum and minimum values ​​of the coordinate values ​​of each dimension, the number of regional segmentations is determined by retrieving the mapping table, and then the target image frame is divided into multiple independent areas. Ensure that each lamp has a clear corresponding area in the target image frame, which can be one area or multiple areas.

[0104] In this embodiment, by obtaining the coordinate information of the lamp and determining the maximum and minimum values ​​of the coordinate values ​​of each dimension, and by calculating the difference between the maximum and minimum values ​​of the coordinate values ​​of each dimension, the distribution range of the lamp in each dimension is quantified, providing important reference data for regional segmentation. Further, by using the difference between the maximum and minimum values ​​of the coordinate values ​​of each dimension, the target image frame is segmented into multiple independent areas by retrieving the mapping table of the number of segmented areas, ensuring that each lamp corresponds to one or more areas, thereby achieving accurate segmentation of the target image frame and providing basic data for subsequent lighting effect playback control.

[0105] Based on any embodiment of the method of the present application, determining feature labels of the respective regions, wherein the feature labels represent visual dynamic characteristics of the respective regions in the target image frame, includes:

[0106] Step S5410, obtaining a plurality of video data sets containing a plurality of visual dynamic characteristics, and training a convolutional neural network model according to the video data sets;

[0107] First, obtain multiple video datasets containing various visual dynamic features, which will be used to train the convolutional neural network model. The video dataset should contain rich visual dynamic features to ensure that the trained convolutional neural network model can more accurately identify and classify the corresponding visual dynamic features.

[0108] Specifically, data can be obtained from public video datasets, or custom video datasets can be recorded and annotated to meet specific needs. Each video dataset should contain multiple video clips, and each video clip should be annotated with corresponding visual dynamic feature labels, such as "explosion", "lightning", "flow", "from left to right", "diffusion", "blue main color", etc. These feature labels will serve as supervisory signals for training convolutional neural network models, helping convolutional neural network models learn how to identify and classify different visual dynamic features, and use the acquired video datasets to train convolutional neural network models. Convolutional neural network models can automatically extract feature vectors from images and videos, and learn how to classify these feature vectors. The trained convolutional neural network model will be able to accurately extract and classify visual dynamic features on new video data, thereby providing an accurate basis for subsequent lighting effect generation.

[0109] Step S5420, acquiring multiple consecutive target image frames in the video stream in real time, and inputting them into the trained convolutional neural network model to extract optical flow information between each frame image and adjacent frames corresponding to each region in the target image frame;

[0110] Use the convolutional neural network model trained in the previous step to extract the optical flow information between each frame image and its adjacent frames in each area of ​​the video stream. Optical flow information refers to the motion vector of pixels between adjacent frames in each target image frame of the video stream. It describes the displacement of each pixel in the image over time and can be used to capture the direction and speed of movement of the object.

[0111] In one embodiment, multiple consecutive target image frames are first captured in real time from a video stream played on a display device, and these target image frames constitute a video stream. The acquired target image frames are input into a trained convolutional neural network model. The convolutional neural network model has been trained with a large number of video data sets, and can automatically extract feature vectors from images and videos, and learn how to classify these feature vectors. When the target image frame is input, the convolutional neural network model will automatically extract the optical flow information between each frame image and its adjacent frames corresponding to each region in the target image frame.

[0112] Step S5430, determining feature vectors corresponding to the respective regions in the target image frames of the consecutive multiple frames based on the optical flow information, and determining feature labels corresponding to the feature vectors according to a predefined label mapping rule to represent the visual dynamic characteristics of the respective regions.

[0113] Based on the optical flow information extracted in the previous step, the feature vectors corresponding to each region in the video stream are determined, and according to the pre-defined label mapping rules, these feature vectors are mapped to the corresponding feature labels to represent the visual dynamic characteristics of each region.

[0114] Specifically, the optical flow information is first analyzed to extract features such as the direction of motion, speed, and effect of each area. These features will be used as components of the feature vector to describe the visual dynamic characteristics of each area. Then, according to the pre-defined label mapping rules, the feature vectors are mapped to the corresponding feature labels. The label mapping rule is a predefined mapping relationship used to associate feature vectors with feature labels. For example, if the feature vector of a region shows that its direction of motion is from left to right, the speed is 1 meter per second, etc., then according to the label mapping rules, the feature labels of the region may be "flowing", "from left to right", and "1 meter per second". In this way, the visual dynamic characteristics of each area in the video stream can be accurately represented as feature labels, thereby providing an accurate basis for the subsequent generation of lighting effects.

[0115] In this embodiment, by acquiring a video data set containing various visual dynamic characteristics and training a convolutional neural network model, it is ensured that the convolutional neural network model can accurately identify and classify different visual dynamic characteristics, thereby providing a solid foundation for subsequent feature extraction and lighting effect generation; then, by acquiring multiple consecutive target image frames in the video stream in real time and inputting them into the trained convolutional neural network model, the optical flow information between each frame image corresponding to each region and its adjacent frames is extracted, the feature vector of each region is determined based on the optical flow information, and the feature vector is mapped to the corresponding feature label according to a pre-defined label mapping rule, so as to accurately represent the visual dynamic characteristics of each region, ensure that the lighting effect of the lamp can match the visual dynamic characteristics in the video stream, and provide users with a more immersive visual experience.

[0116] On the basis of any embodiment of the method of the present application, according to the feature labels of each area and the sub-lamp bead layout information corresponding to each area, a lighting effect playback instruction corresponding to the type of the lamp to which the target lamp belongs is generated, and the lighting effect playback instruction is sent to the corresponding lamp to play the corresponding lighting effect, including:

[0117] Step S5510: Obtain a lighting effect description template corresponding to each lamp, wherein the lighting effect description template includes a feature label attribute item corresponding to the lamp;

[0118] The lighting effect description template is a predefined template that contains the lighting effect performance of the lamp under different feature tag attribute items. These feature tag attribute items usually include effect tags, direction tags, speed tags, and main color tags, which are used to describe the response of the lamp under different visual dynamic characteristics. Obtaining the lighting effect description template can be achieved through the predefined lighting effect description template library. The lighting effect description template library contains lighting effect description templates for various types of lamps, and each lighting effect description template corresponds to a specific lamp type.

[0119] Step S5520: assigning a value to a characteristic label attribute item of a lamp corresponding to each area in the video stream according to the characteristic label determined for each area in the video stream;

[0120] Get the feature labels determined in each area of ​​the video stream, and assign values ​​to the feature label attribute items of the lamps corresponding to each area. These feature label attribute items describe the specific performance of the lamps under different feature labels. By assigning values ​​to these attribute items, a lighting effect that matches the visual dynamic characteristics of the target image frame in the video stream can be generated.

[0121] Specifically, the feature labels determined for each area in the video stream are first obtained. These feature labels are determined by the optical flow information extracted by the trained convolutional neural network model, and describe the visual dynamic characteristics of each area in the video stream. Then, based on these feature labels, the feature label attribute items of the lamps corresponding to each area are assigned values. For example, if the feature label of an area is displayed as an "explosion" effect, an "outward diffusion" direction, a speed of "1 meter per second", and a "blue main color", then the feature label attribute items of the lamps corresponding to the area will be assigned values, including: assigning the "effect" attribute item to "explosion", the "direction" attribute item to "outward diffusion", the "speed" attribute item to "1 meter per second", and the "main color" attribute item to "blue".

[0122] In this way, the visual dynamic characteristics of each area in the video stream can be accurately mapped to the characteristic label attribute items of the lamp, thus providing an accurate basis for the subsequent lighting effect generation.

[0123] Step S5530, in combination with the lamp bead layout information, convert the lighting effect description template that assigns the feature tag attribute item into the lighting effect playback instruction, the lighting effect playback instruction matches the lighting fixture type to which the lamp belongs, and controls each lamp to collaboratively play the lighting effects corresponding to the each area in the video stream.

[0124] Combined with the lamp bead layout information, the lighting effect description template assigned to the feature tag attribute item is converted into a lighting effect playback instruction that matches the type of lamp to which the lamp belongs. The converted lighting effect playback instruction will control each lamp to collaboratively play the lighting effects corresponding to each area in the video stream, thereby achieving a lighting effect that matches the visual dynamic characteristics of the target image frame in the video stream.

[0125] Specifically, the layout information of the lamp beads of each lamp will be obtained first, including the number of lamp beads, arrangement and serial position. This lamp bead layout information will be used to adjust the generated lighting effects to ensure that the lighting effects can match the actual layout of the lamp. At the same time, lighting effects of different complexities will be generated according to the type of lamp to which the lamp belongs, that is, the type of styling freedom of the lamp. Different types of lamps require lighting effects of different complexities to show their styling freedom. Then, the generated lighting effect playback instructions will be sent to the corresponding lamps through wireless communication protocols or other communication methods. After the built-in control chip of the lamp receives the lighting effect playback instruction, it will control the on and off, color and dynamic changes of the lamp beads according to the instruction, so as to achieve a lighting effect display that matches the visual dynamic characteristics of the target image frame in the video stream.

[0126] In this embodiment, based on the feature labels determined for each area in the video stream, the feature label attribute items of the lamps corresponding to each area are assigned values, ensuring that the lighting effect performance of the lamps can accurately map the visual dynamic characteristics of the video stream, providing accurate data support for the subsequent lighting effect generation; combined with the lamp bead layout information, the lighting effect description template assigned to the feature label attribute items is converted into a lighting effect playback instruction that matches the lamp type to which the lamp belongs, ensuring that the generated lighting effect can match the lamp bead layout information and type of the lamp, thereby achieving a lighting effect display corresponding to the visual dynamic characteristics in the video stream, providing users with a more immersive visual experience.

[0127] See also Figure 3, a lighting effect playback control device provided to meet one of the purposes of the present application is a functional embodiment of the lighting effect playback control method of the present application. On the other hand, the device is a lighting effect playback control device provided to meet one of the purposes of the present application, including an identification information acquisition module 5100, a region segmentation module 5200, a feature label determination module 5300 and a lighting effect generation module 5400, wherein the identification information acquisition module 5100 is configured to obtain identification information of each of a plurality of lamps, and determine the type of lamp and lamp bead layout information to which it belongs according to the identification information of the lamp, wherein the lamp type is used to indicate the type of modeling freedom corresponding to the lamp bead layout of the lamp, wherein at least some of the lamp types are different from each other; the region segmentation module 5200 is configured to A video stream is received, and based on the position information of each lamp in the actual physical space and the lamp bead layout information, the target image frame determined in the video stream is segmented into regions, and the sub-lamp bead layout information of the target lamp corresponding to each segmented region is determined respectively; a feature label determination module 5300 is configured to determine the feature labels of each region, and the feature labels represent the visual dynamic characteristics of each region in the target image frame; a lighting effect generation module 5400 is configured to generate a lighting effect playback instruction corresponding to the type of lamp to which the target lamp belongs according to the feature labels of each region and the sub-lamp bead layout information corresponding to each region, and send the lighting effect playback instruction to the corresponding lamp to play the corresponding lighting effect.

[0128] In a further embodiment, the identification information acquisition module 5100 includes: a broadcast signal receiving submodule, configured to receive the broadcast signal of the lamp; an identification information determination submodule, configured to determine the identification information of the lamp based on the broadcast signal, the identification information including the lamp type, lamp bead layout information and a unique identification code; a lamp connection submodule, configured to identify and connect the lamp based on the unique identification code; and an associated storage submodule, configured to associate and store the lamp bead layout information, lamp type and unique identification code of the lamp.

[0129] In a further embodiment, the region segmentation module 5200 includes: a lamp scanning submodule, configured to obtain spatial scanning data of the actual physical space where the multiple lamps are located, and the spatial scanning data includes geometric structure information and relative position information of the multiple lamps; a lamp mapping submodule, configured to map the lamps to a virtual matrix of a graphical user interface based on the geometric structure information and the relative position information, wherein the virtual matrix is ​​provided with a reference coordinate system and is filled with multiple point identifiers; a coordinate information conversion submodule, configured to convert the relative position information into coordinate information in the reference coordinate system; a sub-lamp bead layout information determination submodule, configured to receive a video stream, and based on the coordinate information of each lamp and the lamp bead layout information, perform region segmentation on the target image frame determined in the video stream, and respectively determine the sub-lamp bead layout information of the target lamp corresponding to each segmented region.

[0130] In a further embodiment, after the coordinate information conversion submodule, it includes: a test signal response submodule, which is configured to respond to a sequential lighting test signal, and verify whether the lighting sequence in the graphical user interface is consistent with the lighting sequence in the actual physical space based on the coordinate information of the lamp; a coordinate information update submodule, which is configured to update the coordinate information of the lamp in the spatial scanning data according to the position of the lamp after position adjustment in the virtual matrix of the graphical user interface or the actual physical space when the lighting sequence in the graphical user interface is inconsistent with the lighting sequence in the actual physical space; a repeated verification submodule, which is configured to re-issue a sequential lighting test signal for verification based on the updated coordinate information of the lamp until the lighting sequence in the graphical user interface is consistent with the lighting sequence in the actual physical space.

[0131] In a further embodiment, the sub-lamp bead layout information determination submodule includes: a coordinate information acquisition submodule, configured to obtain the coordinate information of the lamp and determine the maximum and minimum values ​​of the coordinate values ​​of each dimension in the coordinate information; a difference calculation submodule, configured to calculate the difference between the maximum and minimum values ​​of the coordinate values ​​of each dimension; and an area segmentation submodule, configured to segment the target image frame determined in the video stream into multiple independent areas according to the difference, and the lamp corresponds to one or more areas.

[0132] In a further embodiment, the feature label determination module 5300 includes: a model training submodule, configured to obtain multiple video data sets containing multiple visual dynamic characteristics, and train a convolutional neural network model based on the video data sets; an optical flow information extraction submodule, configured to obtain multiple consecutive target image frames in the video stream in real time, and input them into the trained convolutional neural network model to extract the optical flow information between each frame image and its adjacent frames corresponding to each region in the target image frame; a feature label determination submodule, configured to determine the feature vectors corresponding to each region in the multiple consecutive target image frames based on the optical flow information, and determine the feature labels corresponding to the feature vectors according to a predefined label mapping rule to represent the visual dynamic characteristics of the each region.

[0133] In a further embodiment, the lighting effect generation module 5400 includes: a lighting effect description template acquisition submodule, configured to obtain a lighting effect description template corresponding to each lamp, the lighting effect description template containing characteristic label attribute items corresponding to the lamp; an attribute item assignment submodule, configured to assign characteristic label attribute items of the lamps corresponding to the respective areas in the video stream according to characteristic labels determined for the respective areas in the video stream; a lighting effect playback instruction conversion submodule, configured to convert the lighting effect description template assigned with characteristic label attribute items into the lighting effect playback instruction in combination with the lamp bead layout information, the lighting effect playback instruction matching the lamp type to which the lamp belongs, and controlling the respective lamps to collaboratively play the lighting effects corresponding to the respective areas in the video stream.

[0134] In order to solve the above technical problems, the present application also provides a computer device. Figure 5 As shown, a schematic diagram of the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. Among them, the computer-readable storage medium of the computer device stores an operating system, a database, and computer-readable instructions. The database may store a control information sequence. When the computer-readable instructions are executed by the processor, the processor can implement a lighting effect playback control method. The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The memory of the computer device may store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor can execute the lighting effect playback control method of the present application. The network interface of the computer device is used to connect and communicate with a terminal. Those skilled in the art can understand that Figure 5The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0135] In this embodiment, the processor is used to execute Figure 4 The memory stores the program code and various data required to execute the above modules or submodules. The network interface is used to transmit data between user terminals or servers. The memory in this embodiment stores the program code and data required to execute all modules / submodules in the lighting effect playback control device of this application, and the server can call the program code and data of the server to execute the functions of all submodules.

[0136] The present application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the lighting effect playback control method of any embodiment of the present application.

[0137] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments of the present application can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a computer-readable storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0138] It will be understood by those skilled in the art that the various operations, methods, steps, measures, and schemes in the processes discussed in this application may be alternated, changed, combined, or deleted. Furthermore, other steps, measures, and schemes in the various operations, methods, and processes discussed in this application may also be alternated, changed, rearranged, decomposed, combined, or deleted. Furthermore, the steps, measures, and schemes in the various operations, methods, and processes in the prior art that are open source and in this application may also be alternated, changed, rearranged, decomposed, combined, or deleted.

[0139] The above description is only a partial implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A lighting effect playback control method, characterized in that: include: Obtain identification information of each lamp from a plurality of lamps, and determine the type of lamp and lamp bead layout information of the lamp according to the identification information of the lamp, wherein the lamp type is used to indicate the type of modeling freedom corresponding to the lamp bead layout of the lamp, wherein at least some of the lamp types are different from each other; Receive a video stream, and based on the position information of each lamp in the actual physical space and the lamp bead layout information, perform region segmentation on the target image frame determined in the video stream, and respectively determine the sub-lamp bead layout information of the target lamp corresponding to each segmented region; Determining feature labels of each region, wherein the feature labels represent visual dynamic characteristics of each region in the target image frame, including: acquiring multiple consecutive target image frames in the video stream in real time, and inputting them into a trained convolutional neural network model to extract optical flow information between each frame image and adjacent frames corresponding to each region in the target image frame; determining feature vectors corresponding to each region in the multiple consecutive target image frames based on the optical flow information, and determining feature labels corresponding to the feature vectors according to a predefined label mapping rule to represent the visual dynamic characteristics of each region; According to the characteristic tags of each area and the sub-lamp bead layout information corresponding to each area, a lighting effect playback instruction corresponding to the type of lamp to which the target lamp belongs is generated, and the lighting effect playback instruction is sent to the corresponding lamp to play the corresponding lighting effect.

2. The lighting effect playback control method according to claim 1, characterized in that: Obtaining identification information of each of the plurality of lamps, and determining the type of the lamp and lamp bead layout information of the lamp according to the identification information of the lamp, including: receiving a broadcast signal of the lamp; Determine identification information of the lamp according to the broadcast signal, wherein the identification information includes lamp type, lamp bead layout information and a unique identification code; Based on the unique identification code, identifying and connecting the lamp; The lamp bead layout information, lamp type and unique identification code of the lamp are stored in association.

3. The lighting effect playback control method according to claim 1, characterized in that: The video stream is received, and based on the position information of each lamp in the actual physical space and the lamp bead layout information, the target image frame determined in the video stream is segmented into regions, and the sub-lamp bead layout information of the target lamp corresponding to each segmented region is determined respectively, including: Acquire spatial scanning data of an actual physical space where the plurality of lamps are located, the spatial scanning data including geometric structure information and relative position information of the plurality of lamps; Based on the geometric structure information and the relative position information, the lamp is mapped into a virtual matrix of a graphical user interface, wherein a reference coordinate system is provided in the virtual matrix and a plurality of point identifiers are used for filling; Converting the relative position information into coordinate information in the reference coordinate system; A video stream is received, and based on the coordinate information of each lamp and the lamp bead layout information, the target image frame determined in the video stream is segmented into regions, and the sub-lamp bead layout information of the target lamp corresponding to each segmented region is determined respectively.

4. The lighting effect playback control method according to claim 3, characterized in that: Based on the coordinate information of each lamp and the lamp bead layout information, the target image frame determined in the video stream is segmented into regions, including: Obtaining coordinate information of the lamp, and determining the maximum and minimum values ​​of coordinate values ​​of each dimension in the coordinate information; Calculate the difference between the maximum value and the minimum value of the coordinate values ​​of each dimension; According to the difference, the target image frame determined in the video stream is divided into a plurality of independent regions, and the lamp corresponds to one or more regions.

5. The lighting effect playback control method according to claim 1, characterized in that: Determining feature labels of the respective regions, wherein the feature labels represent visual dynamic characteristics of the respective regions in the target image frame, includes: A plurality of video data sets containing a plurality of visual dynamic characteristics are obtained, and a convolutional neural network model is trained according to the video data sets.

6. The lighting effect playback control method according to claim 1, characterized in that: According to the characteristic tags of each area and the sub-lamp bead layout information corresponding to each area, a lighting effect playing instruction corresponding to the type of the target lamp is generated, and the lighting effect playing instruction is sent to the corresponding lamp to play the corresponding lighting effect, including: Obtaining a lighting effect description template corresponding to each lamp, wherein the lighting effect description template includes a feature label attribute item corresponding to the lamp; Assigning a value to a characteristic label attribute item of a lamp corresponding to each area in the video stream according to the characteristic label determined for each area in the video stream; Combined with the lamp bead layout information, the lighting effect description template that assigns the feature tag attribute item is converted into the lighting effect playback instruction, and the lighting effect playback instruction matches the lighting fixture type to which the lamp belongs, and controls each lamp to collaboratively play the lighting effects corresponding to the various areas in the video stream.

7. The lighting effect playback control method according to claim 3, characterized in that: After the relative position information is converted into coordinate information in the reference coordinate system, the method further comprises: In response to the sequential lighting test signal, based on the coordinate information of the lamp, verify whether the lighting sequence in the graphical user interface is consistent with the lighting sequence in the actual physical space; When the lighting order in the graphical user interface is inconsistent with the lighting order in the actual physical space, updating the coordinate information of the lamp in the space scanning data according to the position of the lamp after position adjustment in the virtual matrix of the graphical user interface or the actual physical space; Based on the updated coordinate information of the lamp, a sequential lighting test signal is reissued for verification until the lighting sequence in the graphical user interface is consistent with the lighting sequence in the actual physical space.

8. A lighting effect playback control device, characterized in that: include: an identification information acquisition module, configured to acquire identification information of each of the plurality of lamps, and determine the type of lamp and lamp bead layout information of the lamp according to the identification information of the lamp, wherein the lamp type is used to indicate the type of modeling freedom corresponding to the lamp bead layout of the lamp, wherein at least some of the lamp types are different from each other; A region segmentation module is configured to receive a video stream, and based on the position information of each lamp in the actual physical space and the lamp bead layout information, perform region segmentation on the target image frame determined in the video stream, and respectively determine the sub-lamp bead layout information of the target lamp corresponding to each segmented region; A feature label determination module is configured to determine the feature labels of each region, wherein the feature labels represent the visual dynamic characteristics of each region in the target image frame, including: acquiring a plurality of consecutive target image frames in the video stream in real time, and inputting them into a trained convolutional neural network model to extract optical flow information between each frame image and adjacent frames corresponding to each region in the target image frame; determining feature vectors corresponding to each region in the plurality of consecutive target image frames based on the optical flow information, and determining feature labels corresponding to the feature vectors according to a predefined label mapping rule to represent the visual dynamic characteristics of each region; The lighting effect generation module is configured to generate a lighting effect playback instruction corresponding to the type of lighting fixture to which the target lighting fixture belongs according to the feature labels of the respective areas and the sub-lamp bead layout information corresponding to the respective areas, and send the lighting effect playback instruction to the corresponding lighting fixture to play the corresponding lighting effect.

9. A lighting effect playing device, comprising a controller and a plurality of lamps, characterized in that: The controller is used to execute the steps of the lighting effect playback control method as described in any one of claims 1 to 7, so as to control multiple lamps to display corresponding lighting effects in a linked manner according to the visual dynamic characteristics of the video stream.

10. A computer-readable storage medium, characterized in that: It stores a computer program implemented according to the method described in any one of claims 1 to 7 in the form of computer-readable instructions, and when the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

Images