Lighting effect display method and its device, equipment and medium

Abstract

The present application relates to a lighting effect display method and its device, equipment, and medium in the field of lighting control, the method comprising: determining the lighting effect content style and lighting effect motion type according to the lighting effect generation text, and extracting the keyword set in the lighting effect generation text; when the lighting effect content style belongs to the target style, matching the keyword set and the lighting effect motion type lighting effect image from the lighting effect library; when the lighting effect image is not matched from the lighting effect library, generating the lighting effect image according to the keyword set and the lighting effect motion type; controlling the light screen device to play the corresponding lighting effect according to the lighting effect image, so as to display the lighting effect image through the light screen device. The present application can display corresponding high-quality lighting effects based on a deep and accurate understanding of user needs.

Description

Technical Field

[0001] The present application relates to the field of lighting control, and in particular to a lighting effect display method and its corresponding device, computer equipment, and computer-readable storage medium. Background Art

[0002] With the improvement of people's economic level, light screen equipment is becoming more and more popular. As a kind of intelligent lamp, light screen equipment can play the role of decorating indoor space and displaying information. The characteristic of light screen equipment is that it uses lamp beads with larger visual granularity as basic pixels. It is manifested in various forms of products through different implementation methods of its light screen, such as curtain lights, pixel screens, splicing lights, etc. One of the functions of light screen equipment is to display the corresponding lighting effects according to the given lighting effect image.

[0003] In traditional technology, the corresponding lighting effect image of a given light screen device is usually directly generated according to user needs, so as to display the corresponding lighting effect on the light screen device according to the lighting effect image. However, no careful consideration is given to whether the display effect of the lighting effect image corresponding to the user needs is dynamic or static, and whether the content of the lighting effect image is abstract or concrete, which makes it difficult for the correspondingly generated lighting effect image to fully and accurately match the user needs, resulting in poor user experience.

[0004] It can be seen that the traditional light screen equipment has not achieved good results in generating images of lighting effects, which affects the promotion of light screen equipment. It is necessary to upgrade the technology of light screen equipment. Summary of the invention

[0005] The primary purpose of the present application is to solve at least one of the above problems and provide a lighting effect display method and its corresponding device, computer equipment, and computer program product.

[0006] In order to meet the various objectives of this application, this application adopts the following technical solutions:

[0007] A lighting effect display method provided to meet one of the purposes of the present application includes the following steps:

[0008] Determine the lighting effect content style and lighting effect motion type according to the lighting effect generation text, and extract a keyword set in the lighting effect generation text;

[0009] When the lighting effect content style belongs to the target style, matching the keyword set and the lighting effect image of the lighting effect motion type from the lighting effect library;

[0010] When no lighting effect image is matched from the lighting effect library, generating a lighting effect image according to the keyword set and the lighting effect motion type;

[0011] According to the lighting effect image, the light screen device is controlled to play the corresponding lighting effect, so as to display the lighting effect image through the light screen device.

[0012] On the other hand, a lighting effect display device provided to meet one of the purposes of the present application includes a text parsing module, an image matching module, an image generation module and a lighting effect playback module, wherein the text parsing module is used to determine the lighting effect content style and the lighting effect motion type based on the lighting effect generation text, and extract the keyword set in the lighting effect generation text; the image matching module is used to match the keyword set and the lighting effect motion type of the lighting effect image from the lighting effect library when the lighting effect content style belongs to the target style; the image generation module is used to generate a lighting effect image based on the keyword set and the lighting effect motion type when no lighting effect image is matched from the lighting effect library; the lighting effect playback module is used to control the light screen device to play the corresponding lighting effect according to the light effect image, so as to display the lighting effect image through the light screen device.

[0013] On the other hand, a computer device provided to meet one of the purposes of the present application includes a central processing unit and a memory, wherein the central processing unit is used to call and run a computer program stored in the memory to execute the steps of the lighting effect display method described in the present application.

[0014] On the other hand, a computer program product provided to meet another purpose of the present application includes a computer program / instruction, which, when executed by a processor, implements the steps of the method described in any embodiment of the present application.

[0015] The technical solution of this application has many advantages, including but not limited to the following aspects:

[0016] First of all, this application clarifies the lighting effect content style and lighting effect movement type of the lighting effect generation text by parsing the lighting effect generation text, and extracts the keyword set in the lighting effect generation text. It can be seen that the user's specific personalized requirements for the lighting effect content style and movement type are refined, achieving a high degree of consistency with user expectations.

[0017] Secondly, when the lighting effect content style belongs to the target style, the lighting effect images that match the keyword set and lighting effect movement type in the lighting effect library are directly used. Taking into account the quality uncertainty of the corresponding lighting effect images generated for the target style, the lighting effect images in the existing lighting effect library that accurately match user needs are directly used. These lighting effect images are usually pre-designed and optimized, so as to ensure a certain degree of good visual effects and high artistic components, avoid the possibility of poor image quality due to instant generation, and greatly reduce the user's confusion and time consumption in the selection process, thereby improving the user experience.

[0018] Furthermore, in the case where no lighting effect image is matched from the lighting effect library, by generating corresponding lighting effect images according to the keyword set and the lighting effect motion type, it is possible to generate lighting effect images that meet the actual needs of the user based on clear user needs.

[0019] Finally, the light screen device is controlled to play the corresponding light effect according to the light effect image, so that the light effect image is displayed on the light screen device, providing users with the required and high-quality light effect visual experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0021] Figure 1 It is a schematic diagram of the principle of the electrical structure of the light screen device in the embodiment of the present application;

[0022] Figure 2 The structural form of the exemplary light screen device of this application;

[0023] Figure 3 A flowchart of a typical embodiment of the lighting effect display method of the present application;

[0024] Figure 4 This is a principle block diagram of the lighting effect display device of the present application;

[0025] Figure 5 A schematic diagram of the structure of a computer device used in this application. DETAILED DESCRIPTION

[0026] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.

[0027] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0028] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0029] See also Figure 1 As can be seen from the structural schematic diagram of a lamp screen device provided in an embodiment of the present application, the lamp screen device includes a controller 1 and a lamp screen 2. The lamp screen 2 is electrically connected to the controller 1 so as to accept the control of the computer program running in the controller 1 and work together to realize the lighting effect playback.

[0030] The controller 1 generally includes a control chip, a communication component, and a bus connector. In some embodiments, the controller 1 may also be configured with a power adapter, a control panel, a display screen, etc. as required.

[0031] The power adapter is mainly used to convert AC power into DC power to power the entire light screen device. The control chip can be implemented using various embedded chips, such as Bluetooth SoC (System on Chip), WiFi SoC, MCU (Micro Controller Unit), DSP (Digital Signal Processing) and other types of chips. The control chip usually includes a central processing unit and a memory, and the memory and the central processing unit are used to store and execute program instructions respectively to achieve corresponding functions. The above various types of control chips can have their own communication components, or they can be configured with additional communication components as needed. The communication component can be used to communicate with external devices, for example, it can communicate with terminal devices such as personal computers or various smart phones, so that after the user issues various configuration instructions through his terminal device, the control chip of the controller 1 can receive the configuration instructions through the communication component to complete the basic configuration so as to control the light screen. In addition, the controller 1 can also obtain the lighting effect application package encapsulating multiple lighting effect layers through the communication component. The bus connector is mainly used to connect the light screen 2 connected to the bus to the power supply and provide the light effect playing instruction, so the corresponding pins of the power bus and the signal bus are provided. Therefore, when the light screen 2 needs to be connected to the controller 1, it can be connected to the bus connector through the corresponding connector of the light screen. The control panel usually provides one or more buttons for implementing switch control of the controller 1, selecting various preset light effect control modes, etc. The display screen can be used to display various control information so as to cooperate with the buttons in the control panel to support the realization of human-computer interaction functions. In some embodiments, the control panel and the display screen can be integrated into the same touch display screen.

[0032] The specific appearance of the light screen can be designed according to actual conditions. The more common product forms include but are not limited to curtain lights, pixel screens, splicing lights, etc.

[0033] For example, Figure 2 The light screen in the embodiment is arranged in the form of a curtain light. The light screen 2 includes a plurality of light strips 21 connected to a bus. Each light strip 21 includes a plurality of lamp beads 210 connected in series. Each lamp bead 210 serves as a light emitting unit. In addition, the number of lamp beads 210 in each light strip 21 is usually the same and arranged at equal intervals. When the light screen 2 used as a curtain light is used, it is usually arranged as follows: Figure 2 The layout shown unfolds each of the light strips 21, so that all the lamp beads in all the light strips 21 are arranged in an array to form a lamp bead matrix structure. Since all the lamp beads can provide a frame effect when they emit light in coordination, the surface where the entire lamp bead matrix structure is located constitutes a display frame 4. When the lighting effects are played, a certain pattern effect can be formed within the display frame 4.

[0034] Each light strip 21 can be composed of a plurality of lamp beads 210 connected in series, each lamp bead 210 is a light-emitting unit, and each lamp bead 210 in the same light strip 21 transmits working current through the same set of cables connected to the bus. The lamp beads 210 in the same light strip 21 can be connected in parallel in terms of electrical connection. In one embodiment, each light strip 21 in the same lamp bead matrix structure can be arranged at equal intervals along the bus direction, and the lamp beads 210 in each light strip 21 are also arranged correspondingly in number and position, so that the entire display frame 4 plays a role similar to a screen when viewing its luminous effect from a distance, and can form a pattern effect in the human eye.

[0035] Each lamp bead 210 of each light strip 21 of the light screen 2 is also provided with a corresponding control chip, which can be selected according to the above disclosure, or other more economical control chips can be selected, and its main function is to extract the luminous color value corresponding to the lamp bead 210 from the lighting effect playback instruction, and control the light-emitting element in the lamp bead 210 to emit the corresponding color light. The light-emitting element can be an LED lamp.

[0036] The implementation principle of the pixel screen is basically the same as that of the curtain light, but the pixel screen provides a frame to fix each light-emitting unit, and arranges each light-emitting unit in a row and column matrix in the same rectangular frame, so as to obtain a product form effect closer to the daily display screen. The pixel screen is similar to the curtain light, and each light-emitting unit can be organized according to a structure with multiple light strips arranged side by side, and can be controlled in the same way as the curtain, so I will not go into details.

[0037] The light screen device of the present application, when powered on, the control chip of the controller can call and execute a computer program from the memory, and through the default initialization process of the computer program, the light screen is powered on and initialized, thereby completing the driver configuration of the light screen and other hardware devices.

[0038] In one embodiment, when the controller starts the light screen, it can first send a self-test instruction to the light screen to drive each light strip or each lamp bead in the light block of the light screen to return its position information in the light strip or light block. Each lamp bead is provided with a corresponding control chip for data communication with the control chip in the controller, so that the characteristic information of the lamp bead itself and the characteristic information of other lamp beads can be serially connected in sequence according to the serial communication protocol to achieve the characterization of its own position information. The serial communication protocol executed between the controller and the lamp bead can be any one of IIC (Inter-Integrated Circuit, integrated circuit bus), SPI (serial peripheral interface, serial peripheral interface), and UART (Universal Asynchronous Receiver-Transmitter, universal asynchronous receiver-transmitter). After the controller obtains the result data returned by the self-test of each lamp bead from the bus, it parses the result data and determines the position of each lamp bead in the display frame 4 presented by the entire lamp screen according to the order of the characteristic information of each lamp bead in the result data. Thus, each lamp bead can be regarded as a light-emitting unit, which can be understood as a basic pixel. The position information of each lamp bead can be constructed as light source position distribution information. When constructing the lighting effect control data, the controller can set the corresponding light-emitting color value for each basic pixel according to the position information of each lamp bead, that is, each light-emitting unit, in the light source position distribution information according to actual needs.

[0039] In some embodiments, the controller 1 of the present application can be implemented in an independent computer device, as long as the computer device is equipped with the corresponding hardware of the controller 1, and the corresponding business logic of the controller 1, including the business logic executed by the method of the present application, is implemented as a computer program and installed in the computer device for operation. When the controller 1 is implemented in a computer device, various resources inherent to the computer device can be shared to save the overall implementation cost. The computer device referred to here can be any terminal device for use by the user, such as a smart phone, a personal computer, a laptop computer, a tablet computer, etc.

[0040] According to the product architecture and working principle of the above-mentioned light screen device, the lighting effect generation method of the present application can be implemented as a computer program product, stored in the memory of the controller of the light screen device, and called from the memory by the central processing unit and run, thereby controlling the light screen to play the corresponding lighting effects.

[0041] Any model of the present application can be deployed in a cloud server, a terminal device, or the controller of the light screen device of the present application, as long as the computing power of the deployed end is capable of meeting the operation requirements of the lighting effect generation model.

[0042] See also Figure 3A lighting effect display method of the present application, in its typical embodiment, comprises the following steps:

[0043] Step S1100, determining the lighting effect content style and lighting effect motion type according to the lighting effect generation text, and extracting a keyword set in the lighting effect generation text;

[0044] The lighting effect generation text is a description text generated for displaying the required lighting effect on the light screen device. The user provides the corresponding lighting effect generation text according to his / her own lighting effect needs, and the provision method includes that the user directly inputs the lighting effect generation text expressed in natural language through an input method, or the user inputs it in the form of voice and converts the voice into the corresponding lighting effect generation text.

[0045] In one embodiment, a style recognition model may be pre-implemented, and the style recognition model is pre-trained to a convergence state, and acquires the ability to determine the style of the lighting effect content of the lighting effect generated text. In addition, a motion recognition model may be pre-implemented, and the motion recognition model is pre-trained to a convergence state, and acquires the ability to determine the type of lighting effect motion of the lighting effect generated text. The style recognition model is used to take the lighting effect generated text as input, and the text semantic representation layer in the style recognition model extracts the deep semantic information of the lighting effect generated text, and accordingly derives a style feature vector that vectorizes the deep semantic information, and the style feature vector represents the semantic feature information related to the style in the lighting effect generated text, and then the classification layer in the style recognition model maps the style feature vector to a binary style category, and accordingly derives the classification probability of the abstract style and the classification probability of the concrete style in the binary style category, and confirms the style category with the highest classification probability as the lighting effect content style of the lighting effect generated text. A motion recognition model is adopted with the lighting effect generation text as input. The text semantic representation layer in the motion recognition model extracts the deep semantic information of the lighting effect generation text, and accordingly obtains a motion feature vector that vectorizes the deep semantic information. The motion feature vector represents the semantic feature information related to the motion state in the lighting effect generation text. Then, the classification layer in the motion recognition model maps the motion feature vector to a binary motion category, and accordingly obtains the classification probability of the static motion type and the classification probability of the dynamic motion type in the binary motion category, and confirms the style category with the highest classification probability as the lighting effect content style of the lighting effect generation text.

[0046] The text representation layer in the style recognition model can be a deep learning model suitable for text semantic representation in the NLP field, such as RoBERTa, Transfomer Encoder, MPNet, BiLSTM, GPT, etc. The classification layer in the style recognition model can be a fully connected layer, MLP, etc., and those skilled in the art can choose to implement it as needed. The text representation layer in the motion recognition model can be a deep learning model suitable for text semantic representation in the NLP field, such as RoBERTa, Transfomer Encoder, MPNet, BiLSTM, GPT, etc. The classification layer in the motion recognition model can be a fully connected layer, MLP, etc., and those skilled in the art can choose to implement it as needed. The specific training process of the style recognition model and the motion recognition model can be flexibly implemented by those skilled in the art according to the reasoning process disclosed herein.

[0047] The keyword set in the lighting effect generation text includes all the keywords in the lighting effect generation text that describe the visual characteristics of the corresponding lighting effect. For extracting the keyword set in the lighting effect generation text, specifically, firstly format the lighting effect generation text, including converting uppercase to lowercase, removing punctuation marks, stop words, etc., or any one or more of the above. Then, the formatted lighting effect generation text is segmented to obtain a corresponding segmentation sequence. The segmentation algorithm can be Jieba segmentation, n-gram segmentation, WordPiece segmentation, or CRF segmentation. In one embodiment, each segmentation in the segmentation sequence is matched with a keyword in a preset dictionary. When there is a keyword matching the segmentation in the dictionary, it is confirmed that the segmentation belongs to a keyword; otherwise, it is confirmed that the segmentation does not belong to a keyword. Thus, all word elements in the segmentation sequence that belong to the keyword constitute a keyword set. The dictionary contains multiple keywords related to the description of the lighting effect. In addition, a text set can be formed by pre-collecting multiple lighting effect generated texts, and the importance of each word in the text set can be determined by the TF-IDF algorithm. The words with importance exceeding the preset threshold are used as keywords, so as to gather all the keywords to construct the dictionary. The preset threshold can be set as needed. The matching can be an exact match based on ES (Elasticsearch), and / or a semantic match based on deep learning. In another embodiment, a vocabulary classification model such as a support vector machine (SVM) or a random forest can be pre-trained. After the model is trained to convergence, the probability of each word in the input text belonging to a keyword is determined. Therefore, the vocabulary classification model is used to take the word segmentation sequence as input, determine the probability of each word in the word segmentation sequence belonging to a keyword, confirm the words with probability exceeding the preset threshold as keywords, and gather all the keywords to form a keyword set. The preset threshold can be set as needed. The specific training process of the vocabulary classification model can be flexibly implemented by those skilled in the art according to the reasoning process disclosed herein.

[0048] Step S1200: When the lighting effect content style belongs to the target style, matching the keyword set and the lighting effect image of the lighting effect motion type from the lighting effect library;

[0049] The lighting effect library is preset with a plurality of lighting effect images containing abstract style content, as well as the image format and lighting effect description text of each lighting effect image, and the image format includes video formats such as GIF, JPG, PNG, and MP4. For example, the user's need to generate lighting effects with abstract style content can be manually imagined, and the corresponding lighting effect generation text can be set, and then the lighting effect image generation method of the present application can be used to generate a plurality of corresponding lighting effect images based on these lighting effect generation texts, and then the lighting effect images with better visual effects are manually screened out from these lighting effect images, and the lighting effect visual characteristics of each lighting effect image are described to form a corresponding lighting effect description text, and each lighting effect image is associated with its image format and lighting effect description text and stored in the lighting effect library for future use; the lighting effect image containing abstract content style can also be directly constructed by manual customization, and the lighting effect visual characteristics of each lighting effect image are described to form a corresponding lighting effect description text, and then each lighting effect image is associated with its lighting effect description text and image format and stored in the lighting effect library.

[0050] The target style is an abstract style. Therefore, when the lighting effect content style is an abstract style, it means that the lighting effect content style belongs to the target style. At this time, if the lighting effect motion type is a static motion type, all lighting effect images with image formats of JPG or PNG are obtained from the lighting effect library. Then, an open source deep learning model suitable for text semantic representation in the NLP field is used to determine the vectorized representation of the semantics of the keyword set, and the deep learning model is used to determine the vectorized representation of the semantics of each lighting effect description text in the lighting effect library. The vector similarity algorithm is used to calculate the similarity between the vectorized representation of the semantics of the keyword set and the vectorized representation of the semantics of each lighting effect description text, and the lighting effect image with the highest similarity that exceeds the preset threshold is screened out to confirm that the lighting effect image is similar to the keyword set and the lighting effect motion. The preset threshold can be set as needed; if the lighting effect motion type is a dynamic motion type, all lighting effect images with image formats of GIF or MP4 are obtained from the lighting effect library, and then, an open source deep learning model suitable for text semantic representation in the NLP field is used to determine the semantic vector representation of the keyword set, and the deep learning model is used to determine the semantic vector representation of each lighting effect description text in the lighting effect library, and a vector similarity algorithm is used to calculate the similarity between the semantic vector representation of the keyword set and the semantic vector representation of each lighting effect description text, and the lighting effect image with the highest similarity exceeding the preset threshold is screened out to confirm that the lighting effect image matches the keyword set and the lighting effect motion type, and the preset threshold can be set as needed.

[0051] Step S1300: when no lighting effect image is matched from the lighting effect library, generating a lighting effect image according to the keyword set and the lighting effect motion type;

[0052] When the lighting effect images in the lighting effect library do not match the keyword set and the lighting effect motion type, it means that no lighting effect image is matched from the lighting effect library. At this time, if the lighting effect motion type is a static motion type, in one embodiment, an open source deep learning model suitable for text semantic representation in the NLP field is used to determine the text semantic representation of the keyword set, and a Gaussian noise matrix image is constructed as a random noise image through a random function. The random noise image and the text semantic representation are combined to form the initial input data. The preset image generation model is used to take the initial input data as input, and the random noise image in the initial input data is iteratively optimized to obtain the corresponding target implicit features, and the target implicit features are decoded to reconstruct the lighting effect image. The image format of the lighting effect image can be PNG or JPG. The image generation model is pre-trained to a convergence state and acquires the ability to generate corresponding images based on the input text. The model selection can be Stable Diffusion.

[0053] In one embodiment, if the lighting effect motion type is a dynamic motion type, an open source text-based model for generating multiple image frames can be used to generate corresponding image files of multiple image frames according to the keyword set as lighting effect images. The image format of the image file composed of the multiple image frames can be any one of video formats such as GIF or MP4. The model can be any one of AnimateDiff, Gen-2, Stable Video Diffusion, Emu Video based on Stable Diffusion, etc., and technical personnel in this field can choose one to implement as needed.

[0054] In a further embodiment, if the generated lighting effect image includes a background, an open source pre-trained image semantic segmentation model is used to determine the content belonging to the background in the lighting effect image, and the color value of each pixel therein is set to the color value corresponding to black (R: 0, G: 0, B: 0), so that the corresponding light-emitting unit does not need to emit light when the background is displayed on the light screen device. The image semantic segmentation model can be Mask R-CNN, U-Net, DeepLabv3, etc.

[0055] Step S1400: Control the light screen device to play the corresponding light effect according to the light effect image, so as to display the light effect image through the light screen device.

[0056] When the resolution of the lighting effect image does not match the resolution of the light screen device, the lighting effect image can be converted into an image corresponding to the resolution of the lighting effect device, and those skilled in the art can flexibly implement it. Then, matching lighting effect control data is generated according to the converted lighting effect image, and the light screen device is controlled to play the corresponding lighting effect according to the lighting effect control data. The specific implementation here is further disclosed by the subsequent step embodiments, and this step is not discussed for the time being.

[0057] It can be seen from the above embodiments that the present application has many advantages, including but not limited to:

[0058] First of all, this application clarifies the lighting effect content style and lighting effect movement type of the lighting effect generation text by parsing the lighting effect generation text, and extracts the keyword set in the lighting effect generation text. It can be seen that the user's specific personalized requirements for the lighting effect content style and movement type are refined, achieving a high degree of consistency with user expectations.

[0059] Secondly, when the lighting effect content style belongs to the target style, the lighting effect images that match the keyword set and lighting effect movement type in the lighting effect library are directly used. Taking into account the quality uncertainty of the corresponding lighting effect images generated for the target style, the lighting effect images in the existing lighting effect library that accurately match user needs are directly used. These lighting effect images are usually pre-designed and optimized, so as to ensure a certain degree of good visual effects and high artistic components, avoid the possibility of poor image quality due to instant generation, and greatly reduce the user's confusion and time consumption in the selection process, thereby improving the user experience.

[0060] Furthermore, in the case where no lighting effect image is matched from the lighting effect library, by generating corresponding lighting effect images according to the keyword set and the lighting effect motion type, it is possible to generate lighting effect images that meet the actual needs of the user based on clear user needs.

[0061] Finally, the light screen device is controlled to play the corresponding light effect according to the light effect image, so that the light effect image is displayed on the light screen device, providing users with the required and high-quality light effect visual experience.

[0062] In a further embodiment, determining the lighting effect content style and the lighting effect motion type according to the lighting effect generated text in step S1100 includes the following steps:

[0063] Step S1110: extracting lighting effect description content in the lighting effect generation text using a large language model;

[0064] The large language model is suitable for text processing in the field of NLP. It is pre-trained with an extremely large corpus until convergence, acquires the ability to generate human language, and has a certain degree of accurate text semantic understanding and logical reasoning ability. The large language model includes OpenAI, Falcon, Chinchilla, PaLM, LLaMA 2, text-embedding-ada-002, etc. Those skilled in the art can choose one to implement as needed.

[0065] Call a preset prompt template, which includes a task description and a lighting effect generation text to be embedded. Those skilled in the art may refer to the following disclosure to flexibly set the prompt template. An exemplary prompt template is given as follows: the task description in the prompt template: "Filter the numbers and indicative descriptive verbs in the following lighting effect generation text, and retain the descriptions of the intuitively perceived objects, the visual characteristics of the required lighting effects, and the observable human behavior actions in the lighting effect generation text, and the nouns, verbs, and adjectives corresponding to these descriptions, so as to obtain the lighting effect description content as the final result without any explanation", and the lighting effect generation text to be embedded: "$query_input$".

[0066] The lighting effect generation text is embedded into a prompt template to obtain a corresponding prompt text, and the prompt text is input into a large language model to obtain the lighting effect description content in the lighting effect generation text determined by the large language model.

[0067] Step S1120: Determine the lighting effect content style and lighting effect motion type corresponding to the lighting effect generation text according to the lighting effect description content.

[0068] The preset first prompt template based on Few-shot learning is called. The first prompt template includes a task description, examples, and lighting effect description content to be embedded. Those skilled in the art may refer to the following disclosure to flexibly set the first prompt template. The first prompt template is exemplified as follows: The task description in the first prompt template: "By specifying the structure and providing some examples, determine whether the user wants an "abstract style" or a "concrete style".

[0069] 1. Judge based on keywords.

[0070] 2. Give the final result directly without any explanation. ".

[0071] Example: "<example> {'input': 'Relaxing and pleasant lighting effects', 'output': 'Abstract style'}; {'input': 'Complex and beautiful', 'output': 'Abstract style'}; {'input': 'Surprising lighting colors', 'output': 'Abstract style'}; {'input': 'Comfortable ambient light', 'output': 'Abstract style'}; {'input': 'Approaching train lighting effects', 'output': 'Figurative style'}; {'input': 'Lighting effects with a floating turtle pattern', 'output': 'Figurative style'}; {'input': 'Lighting effects with a strong sense of rhythm', 'output': 'Figurative style'}< / example>"

[0072] The content of the lighting effect description to be embedded: "<input>$The input content to be judged$< / input>"

[0073] Embed the content of the lighting effect description into the first prompt template to obtain the corresponding prompt text, and input the prompt text into the large language model to obtain the lighting effect content style of the lighting effect generation text determined by the large language model.

[0074] In addition, call the second prompt template based on Few-shot learning (few-shot learning) preset. The second prompt template includes a task description, examples, and the content of the lighting effect description to be embedded. Those skilled in the art can refer to the following disclosure to flexibly set the second prompt template. A demonstrative example of the second prompt template: The task description in the second prompt template: "Judge whether what the user wants is a'static movement type' or a 'dynamic movement type' by specifying the structure and providing some examples."

[0075] 1. Judge according to keywords.

[0076] 2. Directly give the final result without any explanation. "

[0077] Example: "<example> {'input': 'Relaxing and pleasant lighting effects', 'output':'static movement type'}; {'input': 'Complex and beautiful', 'output':'static movement type'}; {'input': 'Surprising lighting colors', 'output':'static movement type'}; {'input': 'Comfortable ambient light', 'output':'static movement type'}; {'input': 'Approaching train lighting effects', 'output': 'dynamic movement type'}; {'input': 'Lighting effects with a floating turtle pattern', 'output': 'dynamic movement type'}; {'input': 'Lighting effects with a strong sense of rhythm', 'output': 'dynamic movement type'}< / example>"

[0078] Lamp effect description content to be embedded: "<input>$Input content to be judged$< / input>"

[0079] Embed the lamp effect description content into the second prompt template to obtain the corresponding prompt text, and input the prompt text into the large language model to obtain the lamp effect movement type of the lamp effect generation text determined by the large language model.

[0080] In this embodiment, by using the large language model to extract the lamp effect description content in the lamp effect generation text, accordingly, the lamp effect content style and lamp effect movement type corresponding to the lamp effect generation text are determined, which can exclude the redundant information in the lamp effect generation text that is irrelevant to confirming its lamp effect content style and lamp effect movement type, so as to ensure the accuracy and reliability of the determined lamp effect content style and lamp effect movement type.

[0081] In a further embodiment, before determining the lamp effect content style and lamp effect movement type according to the lamp effect generation text in step S1100, the following steps are included:

[0082] Step S1000: Obtain the description text provided by the user, confirm whether there is an intention to generate a lamp effect in the description text, and when there is, use the description text as the lamp effect generation text;

[0083] In order to display the required lighting effects on the light screen device, the user can provide a corresponding description text. However, since the user may not describe the lighting effect generation intention clearly enough, that is, the lighting effect generation intention does not exist in the description text, which leads to the subsequent inability to generate the corresponding lighting effect image. Therefore, the preset intention recognition model is first called for the description text. The intention recognition model has been pre-trained to a convergence state and has learned the ability to recognize whether the input text has the intention to generate a lighting effect. Since the training process of the intention recognition model is known in the art, the technicians in this field can flexibly implement the corresponding training of the intention recognition model according to the forward reasoning process of the intention recognition model disclosed later, so that the intention recognition model after training to convergence has the said ability. In one embodiment, the intention recognition model is implemented using Bert, and the description text is segmented using a word segmentation algorithm to obtain a word segmentation sequence, and a [CLS] identifier is added at the start position of the word segmentation sequence and a [SEP] identifier is added at the end position, and then the word segmentation sequence is input into the intention recognition model, and the input embedding vectors of each identifier and word unit in the word segmentation sequence are determined through the WordPiece embedding layer in the intention recognition model, and the input embedding vectors are input into a multi-layer stacked Transformer encoder to obtain the text feature vectors of each identifier and word unit in the word segmentation sequence output by the last layer of Transformer encoder, and the text feature vector of the [CLS] identifier is linearly transformed through the feedforward neural network layer, and then a binary probability distribution is obtained through the Softmax layer, and the binary probability distribution includes the probability of the first category and the probability of the second category, the first category represents that the input text has the intention of generating light effects, and the second category represents that the input text does not have the intention of generating light effects. Determine the category corresponding to the maximum probability in the binary probability distribution, and when the category is the first category, it means that the description text has the intention of generating light effects, and at this time, the description text is used as the text of generating light effects.

[0084] Step S1010: When there is no lighting effect generation intention, sending a plurality of lighting effect images and their effect descriptions preset in the lighting effect library to the user;

[0085] When the category with the maximum probability in the binary probability distribution is the second category, it means that the description text does not have the intention of generating lighting effects. At this time, multiple preset lighting effect images are obtained from the lighting effect library, and the lighting effect description text of each lighting effect image is obtained, and these lighting effect images and their lighting effect description texts are sent to the user. The multiple lighting effect images can be lighting effect images that are manually judged to have relatively high visual beauty in the lighting effect library, or they can be lighting effect images randomly selected from all the lighting effect images in the lighting effect library. The specific number of these lighting effect images can be set as needed.

[0086] Step S1020: In response to a selection event of any one target lighting effect image among the multiple lighting effect images, recall a lighting effect image that matches the target lighting effect image in the lighting effect library.

[0087] In the recommended embodiment, a pre-constructed lighting effect playback interface is provided to the user. For the multiple lighting effect images, the user can preview any lighting effect image in the lighting effect playback interface, and the lighting effect description text of the lighting effect image previewed by the user is displayed on the interface, and a video of the lighting effect image displaying the corresponding lighting effect on the light screen device is loaded and played, so that the user can intuitively understand the visual effect of the lighting effect image displayed on the light screen device, and the user can refer to the lighting effect description text to enhance the reading and understanding of the visual effect. The lighting effect playback interface can be flexibly implemented by those skilled in the art. In addition, for each lighting effect image in the lighting effect library, those skilled in the art can pre-shoot and record the video of the lighting effect image displaying the corresponding lighting effect on the light screen device, and store the lighting effect images associated with each video in the lighting effect library for future use. Thus, the user can preview any of the multiple lighting effect images, so that the user can select any one of the multiple lighting effect images according to his own needs, and then operate the selected lighting effect image as the target lighting effect image on the lighting effect playback interface. Then, in response to the selection event, a lighting effect image corresponding to the target lighting effect image is recalled from the lighting effect library.

[0088] In this embodiment, by clarifying whether the description text provided by the user has the intention to generate lighting effects, the description text with the intention to generate lighting effects is used as the lighting effect generation text, so as to avoid using the description text without the intention to generate lighting effects for the subsequent generation of corresponding lighting effect images, which may lead to poor generation effect or generation failure, delay the user's waiting time and / or reduce the user experience. In addition, when the description text does not have the intention to generate lighting effects, multiple lighting effect images and their effect descriptions are provided to the user, and then the target lighting effect image in the user is obtained and the corresponding lighting effect image is recalled from the lighting effect library. Such a humanized design can provide the user with ready-made lighting effect images to choose from in a timely manner when the user's current lighting effect generation intention is not accurately described, greatly reducing the delay in determining the lighting effect image, and then the user can quickly see the visual effect of the corresponding lighting effect they need displayed on the light screen device, thereby improving the user experience.

[0089] In a further embodiment, extracting the keyword set in the lighting effect generation text in step S1100 includes the following steps:

[0090] Step S1101: embed the lighting effect generation text into a preset prompt template to obtain a prompt text, and use the prompt text to guide the large language model to determine a keyword set in the lighting effect generation text;

[0091] Call a preset prompt template, which includes a task description and a lighting effect generation text to be embedded. Those skilled in the art can refer to the following disclosure to flexibly set the prompt template. An exemplary prompt template is given as follows: The task description in the prompt template: "Extract all keywords in the lighting effect generation text according to the following rules, and determine the importance of each keyword relative to other keywords.

[0092] 1. Each keyword should contain at most one noun to ensure the indirectness and focus of the keyword.

[0093] 2. If the keyword contains a noun, you need to add an article before the noun to help clarify the reference and enhance the naturalness of the language expression.

[0094] 3. Give priority to using proper nouns (such as landmarks, specific names) as keywords because they are unique and recognizable.

[0095] 4. Include no more than one adjective per keyword to keep the description precise and not overly complicated.

[0096] 5. Sort all keywords according to the importance of each keyword in the lighting effect generation text to ensure that more important keywords are ranked first and are given priority when subsequently used to generate lighting effect images.

[0097] 6. Prioritize using nouns that describe real objects (except objects such as lighting equipment) as keywords to ensure that they are subsequently used to generate practical and relevant lighting effect images", and the lighting effect generation text to be embedded is: "$query_input$".

[0098] Step S1102: using the large language model to determine the keywords and their importance in the lighting effect generation text according to the prompt text, and arranging the keywords in order of importance to construct a keyword set.

[0099] The prompt text is input into the large language model to obtain all the keywords in the lighting effect generation text determined by the large language model, as well as the importance of each keyword relative to other keywords, so as to sort all the keywords in order of importance from high to low, and the sorted keywords constitute the keyword set in the lighting effect generation text.

[0100] In this embodiment, prompt text is used to guide the large language model to determine all the keywords in the lighting effect generation text and the importance of each keyword relative to other keywords, and then the keywords are arranged in order of importance to construct a keyword set in the lighting effect generation text. It can be seen that based on the prompt text with strong interpretability and editability, the large language model can quickly and accurately extract the required keyword set, and can ensure that the keywords with relatively strong importance in the keyword set are ranked relatively high, so that they are given priority when subsequently used to generate lighting effect images, thereby ensuring the generation of practical and relevant lighting effect images, meeting user needs and improving user experience.

[0101] In a further embodiment, when no lighting effect image is matched from the lighting effect library in step S1300, generating a lighting effect image according to the keyword set and the lighting effect motion type includes the following steps:

[0102] Step S1310: when the lighting effect motion type belongs to the static motion type, a text encoding module in a preset image-text matching model is used to determine the text semantic representation of the keyword set, and the text semantic representation is combined with a random noise image to obtain initial input data;

[0103] The image-text matching model is a CLIP model that has been pre-trained to a convergent state, and has learned to determine the vector representations of the input image and text that are respectively in the same semantic space, and the ability to confirm the similarity between the image and the text based on the similarity between the two vector representations.

[0104] Therefore, the text encoding module in the image-text matching model can be used to extract the deep semantic information of the keyword set with the keyword set as input, and obtain the text semantic representation of the deep semantic information vectorized. In addition, a random function is used to construct an image of a Gaussian noise matrix as a random noise image. The initial input data is obtained by combining the text semantic representation and the random noise image.

[0105] Step S1320, using an image optimization module in a preset image generation model to iteratively optimize the random noise image in the initial input data to obtain target implicit features;

[0106] The image generation model adopts Stable Diffusion, which is pre-trained to a convergence state and learns the ability to generate corresponding images according to input text.

[0107] The image optimization module in the image generation model is used for continuous iterative optimization. The U-Net network in this process is responsible for continuously optimizing the generation process, predicting the noise in the random noise image in the initial input data, and continuously injecting text semantic information according to the text semantic representation in the initial input data. In addition, the Schedule algorithm optimizes the noise predicted by U-Net each time, and the optimization process includes dynamically adjusting the predicted noise and controlling the intensity of the U-Net predicted noise, thereby coordinating the progress of the generation process. Finally, when the number of iterative optimization steps of U-Net reaches a certain number, such as 50 or 100 times, the pure noise in the random noise image is removed, and the image semantic information of the image is matched with the text semantic representation, thereby obtaining the corresponding image in the latent space as the target implicit feature.

[0108] Step S1330: The image decoder in the image generation model reconstructs a lighting effect image from the target implicit features.

[0109] The image encoder is a VAE (Variational Auto-Encoder), which converts the target implicit features from the latent space to the original pixel space, and generates a corresponding image as the lighting effect image, whose image format is PNG or JPG.

[0110] In this embodiment, a process of generating a corresponding lighting effect image according to a keyword set of the lighting effect generating text is disclosed when the lighting effect motion type of the lighting effect generating text is a static motion type. This can ensure that the lighting effect image is closely related to the keyword set, meet user needs and enhance user experience.

[0111] In a further embodiment, when no lighting effect image is matched from the lighting effect library in step S1300, generating a lighting effect image according to the keyword set and the lighting effect motion type includes the following steps:

[0112] Step S1301: when the lighting effect motion type belongs to a dynamic motion type, a preset multi-image frame generation model is used to generate multiple image frames according to the keyword set;

[0113] The lighting effect motion type is a dynamic motion type, and an open source text-based model for generating multiple image frames can be used to generate corresponding multiple image frames according to the keyword set. The image format of the lighting effect image can be any video format such as GIF or MP4, and the model can be any one of AnimateDiff, Gen-2, StableVideo Diffusion, Emu Video based on Stable Diffusion. Technical personnel in this field can choose one to implement as needed.

[0114] Step S1302: construct the lighting effect image using the multiple image frames.

[0115] The plurality of continuous image frames constitute the lighting effect image. It is not difficult to understand that, based on the lighting effect image, when playing the corresponding lighting effect, the corresponding dynamic effect can be presented.

[0116] In this embodiment, when the lighting effect motion type is a dynamic motion type, a multi-image frame generation model is used to generate multiple image frames according to a keyword set to form a lighting effect image, which can serve the needs of playing dynamic lighting effects on a light screen device and meet the visual personalized experience.

[0117] In a further embodiment, controlling the light screen device to play the corresponding light effect according to the light effect image in step S1400 so as to display the light effect image through the light screen device includes the following steps:

[0118] Step S1410: performing color enhancement processing on the lighting effect image according to the light source position distribution information of the light screen device to obtain an enhanced image, so as to reset the color of corresponding pixel points in the lighting effect image;

[0119] The color value of the pixel in the lighting effect image may be an RGB value or an HSV value. If the color value of the pixel is an RGB value, the RGB value of the pixel is converted into a corresponding HSV value.

[0120] For each pixel in the lighting effect image, in the HSV three-dimensional coordinate system, the color value of the pixel expressed in HSV can be located on the corresponding hue circle to the two-dimensional coordinates of the HSV value (the H value in the HSV value, the S value in the HSV value) as the original position of the pixel on the hue circle. The hue circle is a circular two-dimensional plane perpendicular to the V axis in the HSV three-dimensional coordinate system and corresponding to the V value in the HSV value. The V value in the circular two-dimensional plane is constant. The value range of the V value and the S value is [0,1], and the value range of the H value is [0,360].

[0121] For the hue rings corresponding to any V value in the HSV three-dimensional coordinate system, the sizes are the same and the hue rings can be divided in the same area division method. Taking a single hue ring as an example, in one embodiment, the hue ring is divided into N sectors of equal size (the central angle of each sector is N / 360), and M concentric circles of different sizes are divided (the center of each concentric circle is the center of the hue ring; the diameter of the i-th concentric circle is i / M of the diameter of the hue ring, i is an integer and i∈(0,M]). Thus, N*M in-ring areas in the hue ring are obtained, and each in-ring area is a closed space composed of the least edges. The density of lamp beads in the light screen device can be determined based on the light source position distribution information, and N and M can be set accordingly according to the density of lamp beads. It is not difficult to understand that the higher the density of lamp beads, the larger the N*M that can be set accordingly. For example, the density of lamp beads when the light screen device is a curtain lamp is larger than the density of lamp beads when the light screen device is a pixel lamp. To be smaller, accordingly, the N*M corresponding to the light screen device as a pixel light is set to be greater than the N*M corresponding to the light screen device as a curtain light. Those skilled in the art can set the specific values ​​of N and M as needed according to the disclosure herein, so that the subsequent image as a whole contains target color values ​​of multiple color types, and the color is richer in visual effect. For example, N is 6, M is 3, and the inner ring area obtained by dividing the hue ring includes 6 inner ring areas 100 of the same size, 6 inner ring areas 101 of the same size, and 6 inner ring areas 102 of the same size, totaling 3*6, i.e., 18 inner ring areas. It is not difficult to understand that the two-dimensional coordinate range corresponding to each inner ring area in the hue ring can be determined, and those skilled in the art can implement it flexibly. Thus, the two-dimensional coordinate range of a single inner ring area in the corresponding hue ring to which the pixel point belongs can be determined, thereby confirming that the original position of the pixel point belongs to the inner ring area.

[0122] Further, in one embodiment, if the inner ring area to which the color value of the pixel point belongs at the original position on the hue ring is the inner ring area of ​​the fan-shaped area in the corresponding hue ring, the distance from the original position is the distance from the original position to the center of the hue ring, and it is regarded as that the distance from the original position meets the preset condition, the center of the circle is used as the target position, the color value expressed in HSV corresponding to the center of the circle is converted into a color value expressed in RGB as the target color value, and the color value of the pixel point is replaced with the target color value; if the inner ring area to which the color value of the pixel point belongs at the original position on the hue ring is the inner ring area of ​​the non-fan-shaped area in the corresponding hue ring, the distance from the original position is the distance from the original position to the midpoint of the longest arc edge in the annular area, and it is regarded as that the distance from the original position meets the preset condition, the midpoint is used as the target position, the color value expressed in HSV corresponding to the midpoint is converted into a color value expressed in RGB as the target color value, and the color value of the pixel point is replaced with the target color value.

[0123] When the color value of each pixel in the lighting effect image is replaced with the corresponding target color value, for each different color value in the lighting effect image, the number of pixels showing the same color value is determined as the total number of pixels of the color value, and all different color values ​​are sorted in descending order according to the total number of pixels, and the top N color values ​​are selected as single main color values, respectively, and N can be set as needed. According to the density of the lamp beads, the N is set accordingly. It is not difficult to understand that the higher the density of the lamp beads, the larger the corresponding N is set. For example, the density of the lamp beads of the light screen device as a curtain light is smaller than that of the light screen device as a pixel light. Based on this, the N corresponding to the light screen device as a pixel light is set to be greater than the N corresponding to the light screen device as a curtain light. Those skilled in the art can set the N as needed according to the disclosure herein, so that the subsequent graphics as a whole contain main color values ​​of multiple color types, and the colors are richer in visual effect. For each pixel, the Euclidean distance between the color value of the pixel and each primary color value is calculated as the distance value. For example, the color value of the pixel is R: 246, B: 3, G: 255, and a primary color value is R: 255, B: 255, G: 255, and the corresponding Euclidean distance is 252.08. Then, the primary color value with the lowest distance value is selected, and the color value of the pixel is replaced with the primary color value.

[0124] When the color value of each pixel in the lighting effect image is replaced with the corresponding main color value, the main content of the lighting effect image is obtained. Specifically, the main content of the lighting effect image is usually the foreground content in the lighting effect image. Therefore, after the lighting effect image is constructed in advance, an open source pre-trained foreground recognition model can be used to determine the foreground content in the lighting effect image. When the background in the lighting effect image is a transparent background, it can be directly confirmed that the content with non-zero transparency in the lighting effect image is the main content. The foreground recognition model can be Mask R-CNN, U-Net, DeepLabv3, etc. The color value of each pixel in the main content of the current lighting effect image is traversed to confirm whether the display effect of the color value is close to black. Specifically, the Euclidean distance between the color value and the color value corresponding to black can be calculated. When the Euclidean distance is less than or equal to the preset threshold, it is confirmed that the color value is close to the color value corresponding to black, that is, it belongs to the predetermined color value. Then, the average hue value is obtained according to the hue value corresponding to the color value of each pixel in the lighting effect image, and the contrast color effect defined in color art is determined to be formed with the average hue value. The corresponding difference color range can be set as required by those skilled in the art according to the disclosure herein. Then, a color value represented by HSV is randomly set from the difference color range as the difference color value, and the color value represented by HSV is converted into a color value represented by RGB as required, and the color values ​​of the pixels belonging to the predetermined color value are replaced with the difference color value.

[0125] When the color values ​​of the pixels belonging to the predetermined color values ​​in the main content of the lighting effect image are replaced with the corresponding difference color values, the enhanced image is obtained.

[0126] Step S1420, determining whether the resolution of the enhanced image meets a preset condition according to the light source position distribution information, and if so, reconstructing the enhanced image into a target lighting effect image that matches the light source position distribution information;

[0127] The resolution of the light screen device can be determined based on the light source position distribution information. When the resolution of the enhanced image exceeds the resolution of the light screen device, it is considered that the preset condition is met. For this, the enhanced image is divided into multiple areas, and the size of each area corresponds to a single pixel of the target resolution. Specifically, the size of each area can be determined by dividing the width and height of the enhanced image by the width and height of the target resolution. The target resolution can be determined based on the resolution of the light screen device. It is only necessary to ensure that the target resolution matches the resolution of the lighting effect device. For example, the target resolution is set to be the same as the resolution of the light screen device, or the target resolution is set to 1 / N of the resolution of the light screen device. N can be set as needed.

[0128] For each region, down-sample the region. Specifically, perform maximum pooling on the colors of all pixels in the region to obtain the color with the largest number in the region, and use this color as the main color of the region.

[0129] Each region is traversed, and the main color of the region is used as the color of the pixel in the image corresponding to the target resolution. Thus, the colors of all the pixels in the image can be obtained, and the image is obtained as an enhanced image.

[0130] Step S1430, generating lighting effect control data matching the light screen device according to the target lighting effect image;

[0131] When the image format of the target lighting effect image is any one of the video formats such as GIF or MP4, the target lighting effect image can be divided into individual image frames, and each image frame can be converted into corresponding lighting effect control data one by one, and then transmitted to the light screen device, so as to control the light screen device to play the corresponding lighting effect; when the image format of the target lighting effect image is PNG or JPG, the target lighting effect image can be directly converted into corresponding lighting effect control data, and then transmitted to the light screen device, so as to control the light screen device to play the corresponding lighting effect.

[0132] Regarding converting a single image into corresponding lighting effect control data, specifically, it is not difficult to understand that, based on the relationship between the basic pixel distribution of the display frame / resolution of the light screen device and the resolution of the image, a partition mapping relationship between them can be established, so that the pixels in the image and the basic pixels of the display frame, that is, the light-emitting units, can be flexibly established in accordance with the proportional relationship of each other's pixel resolutions. Many-to-one, many-to-many, one-to-many, one-to-one and other partition mapping relationships. According to this partition mapping relationship, the color of the pixel points of each partition on the image can be mapped to the color of the pixel points of the corresponding partition in the display frame, which is expressed in the lighting effect control data, that is, the control data of each light-emitting unit in the corresponding partition. It can be seen that the corresponding lighting effect control data can be generated according to the preset rules based on the image.

[0133] In one embodiment, the resolution of the image is completely consistent with the display frame / resolution of the light screen device. In this case, the resolution of the image corresponds one-to-one to the light-emitting units in the display frame, and the control data of the corresponding light-emitting units can be directly generated according to the color of each pixel in the image, thereby achieving a one-to-one partition mapping relationship between the image and the display frame to generate lighting effect control data. In another embodiment, when the resolution of the image is inconsistent with the display frame / resolution of the light screen device, when it is determined that the image is placed in the middle of the display frame of the light screen device, the light screen device generates control data for each light-emitting unit that overlaps with the image in the middle according to the color of each pixel in the image, thereby achieving a one-to-one partition mapping relationship between the image and the middle part of the display frame to generate lighting effect control data.

[0134] Step S1440: Control the light screen device to play corresponding light effects according to the light effect control data.

[0135] When the image format of the target lighting effect image is any one of the video formats such as GIF or MP4, and multiple lighting effect control data are generated accordingly, the controller can trigger the transmission of a lighting effect control data to the light screen device at intervals according to the preset playback time slot or the frame rate of the target lighting effect image, thereby transmitting each lighting effect control data corresponding to each image frame in the target lighting effect image to the light screen device in turn; when the image format of the target lighting effect image is PNG or JPG, the lighting effect control data corresponding to the target lighting effect image can be directly transmitted to the light screen device.

[0136] Whenever the light screen device receives the lighting effect control data, it transmits it to each light-emitting unit according to the data communication protocol followed between it and each light-emitting unit, such as the serial communication protocol. After each light-emitting unit receives the lighting effect control data, it extracts or intercepts the corresponding control data, controls its own light-emitting element to emit corresponding light according to its own control data, and forwards the lighting effect control data to other light-emitting units that have not received it. Each light-emitting unit receives, analyzes, responds to and forwards the lighting effect control data according to its inherent business logic, and the entire light screen device can present the visual effect of the target lighting effect image corresponding to the lighting effect control data.

[0137] In this embodiment, a process of controlling a light screen device to play a corresponding light effect based on a light effect image is disclosed. In this process, the light source position distribution information of a light screen device associated with the light effect image is color enhanced and reconstructed to match the light source position distribution information, and then light effect control data is constructed to control the light screen device to accurately play the corresponding light effect. This ensures that the light effect played by the light screen device is more visually prominent, and is adapted to the light source position distribution information of the light screen device, thereby ensuring the integrity and accuracy of the light effect display.

[0138] See also Figure 4 , a lighting effect display device provided to meet one of the purposes of the present application is a functional embodiment of the lighting effect display method of the present application. On the other hand, the device is a lighting effect display device provided to meet one of the purposes of the present application, including a text parsing module 1100, an image matching module 1200, an image generating module 1300 and a lighting effect playing module 1400, wherein the text parsing module 1100 is used to determine the lighting effect content style and the lighting effect motion type according to the lighting effect generating text, and extract the keyword set in the lighting effect generating text; the image matching module 1200 is used to match the lighting effect image of the keyword set and the lighting effect motion type from the lighting effect library when the lighting effect content style belongs to the target style; the image generating module 1300 is used to generate a lighting effect image according to the keyword set and the lighting effect motion type when no lighting effect image is matched from the lighting effect library; the lighting effect playing module 1400 is used to control the light screen device to play the corresponding lighting effect according to the lighting effect image, so as to display the lighting effect image through the light screen device.

[0139] In a further embodiment, the text parsing module 1100 includes: a content extraction sub-module, used to extract the lighting effect description content in the lighting effect generation text using a large language model; and a style movement determination sub-module, used to determine the lighting effect content style and lighting effect movement type corresponding to the lighting effect generation text based on the lighting effect description content.

[0140] In a further embodiment, the text parsing module 1100 includes: an intention recognition submodule for obtaining a description text provided by a user, confirming whether the description text has a lighting effect generation intention, and if so, using the description text as a lighting effect generation text; a lighting effect recommendation submodule for sending a plurality of lighting effect images preset in a lighting effect library and descriptions of their effects to the user when there is no lighting effect generation intention; and an image recall submodule for responding to a selection event of any one of the plurality of lighting effect images, and recalling lighting effect images in the lighting effect library that match the target lighting effect image.

[0141] In a further embodiment, the text parsing module 1100 includes: a prompt construction submodule, used to embed the lighting effect generation text into a preset prompt template to obtain a prompt text, and use the prompt text to guide the large language model to determine the keyword set in the lighting effect generation text; a word set construction submodule, used to use the large language model to determine each keyword in the lighting effect generation text and its importance according to the prompt text, and arrange each keyword in order of importance to construct a keyword set.

[0142] In a further embodiment, the image generation module 1300 includes: a text encoding sub-module, which is used to use the text encoding module in the preset image-text matching model to determine the text semantic representation of the keyword set when the lighting effect motion type belongs to the static motion type, and combine it with the random noise image to obtain initial input data; a feature confirmation sub-module, which is used to use the image optimization module in the preset image generation model to iteratively optimize the random noise image in the initial input data to obtain target implicit features; and an image reconstruction sub-module, which is used to reconstruct the lighting effect image from the target implicit features by the image decoder in the image generation model.

[0143] In a further embodiment, the image generation module 1300 includes: a multi-image generation sub-module, used to generate multiple image frames according to the keyword set using a preset multi-image frame generation model when the lighting effect motion type belongs to a dynamic motion type; an image composition sub-module, used to construct the lighting effect image from the multiple image frames.

[0144] In a further embodiment, the lighting effect playback module 1400 includes: a color enhancement submodule, used to perform color enhancement processing on the lighting effect image according to the light source position distribution information of the light screen device to obtain an enhanced image to reset the color of the corresponding pixel points in the lighting effect image; a resolution adjustment submodule, used to determine whether the resolution of the enhanced image meets the preset conditions according to the light source position distribution information, and when it meets the conditions, reconstruct the enhanced image into a target lighting effect image that matches the light source position distribution information; a control data submodule, used to generate lighting effect control data that matches the light screen device according to the target lighting effect image; and a lighting effect control submodule, used to control the light screen device to play the corresponding lighting effect according to the lighting effect control data.

[0145] In order to solve the above technical problems, the present application also provides a computer device. Figure 5 As shown, a schematic diagram of the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. Among them, the computer-readable storage medium of the computer device stores an operating system, a database, and computer-readable instructions. The database may store a control information sequence. When the computer-readable instructions are executed by the processor, the processor can implement a lighting effect display method. The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The memory of the computer device may store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor can execute the lighting effect display method of the present application. The network interface of the computer device is used to connect and communicate with a terminal. Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0146] In this embodiment, the processor is used to execute Figure 4 The memory stores the program code and various data required to execute the above modules or submodules. The network interface is used to transmit data between user terminals or servers. The memory in this embodiment stores the program code and data required to execute all modules / submodules in the lighting effect display device of this application, and the server can call the program code and data of the server to execute the functions of all submodules.

[0147] The present application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the lighting effect display method of any embodiment of the present application.

[0148] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments of the present application can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a computer-readable storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0149] In summary, the present application can display corresponding high-quality lighting effects based on a deep and accurate understanding of user needs.

[0150] It will be understood by those skilled in the art that the various operations, methods, steps, measures, and schemes in the processes discussed in this application may be alternated, changed, combined, or deleted. Furthermore, other steps, measures, and schemes in the various operations, methods, and processes discussed in this application may also be alternated, changed, rearranged, decomposed, combined, or deleted. Furthermore, the steps, measures, and schemes in the various operations, methods, and processes in the prior art that are open source and in this application may also be alternated, changed, rearranged, decomposed, combined, or deleted.

[0151] The above description is only a partial implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A lighting effect display method, characterized in that: The steps include: Determine the lighting effect content style and lighting effect motion type according to the lighting effect generation text, and extract a keyword set in the lighting effect generation text; When the lighting effect content style belongs to the target style, matching the keyword set and the lighting effect image of the lighting effect motion type from the lighting effect library; When no lighting effect image is matched from the lighting effect library, generating a lighting effect image according to the keyword set and the lighting effect motion type; Controlling the light screen device to play the corresponding light effect according to the light effect image, so as to display the light effect image through the light screen device; The step of generating a lighting effect image according to the keyword set and the lighting effect motion type includes: When the lighting effect motion type belongs to the static motion type, a text encoding module in a preset image-text matching model is used to determine the text semantic representation of the keyword set, and the text semantic representation is combined with the random noise image to obtain initial input data; Iteratively optimizing the random noise image in the initial input data using an image optimization module in a preset image generation model to obtain target implicit features; The image decoder in the image generation model reconstructs the lighting effect image from the target implicit features; The step of generating a lighting effect image according to the keyword set and the lighting effect motion type includes: When the lighting effect motion type belongs to a dynamic motion type, a preset multi-image frame generation model is used to generate multiple image frames according to the keyword set; The lighting effect image is formed by the multiple image frames.

2. The lighting effect display method according to claim 1, characterized in that: Determining the lighting effect content style and lighting effect motion type according to the lighting effect generated text includes the following steps: A large language model is used to extract the lighting effect description content in the lighting effect generation text; The lighting effect content style and lighting effect motion type corresponding to the lighting effect generation text are determined according to the lighting effect description content.

3. The lighting effect display method according to claim 1, characterized in that: Before determining the lighting effect content style and lighting effect motion type based on the lighting effect generated text, the following steps are included: Get the description text provided by the user, and confirm whether the description text has the intention of generating lighting effects. If so, use the description text as the text for generating lighting effects. When there is no lighting effect generation intention, sending a plurality of lighting effect images preset in the lighting effect library and their effect descriptions to the user; In response to a selection event of any one target lighting effect image among the multiple lighting effect images, a lighting effect image matching the target lighting effect image is recalled from the lighting effect library.

4. The lighting effect display method according to claim 1, characterized in that: Extracting a keyword set from the lighting effect generation text includes the following steps: The lighting effect generation text is embedded into a preset prompt template to obtain a prompt text, and the prompt text is used to guide the large language model to determine a keyword set in the lighting effect generation text; The large language model is used to determine each keyword in the lighting effect generation text and its importance according to the prompt text, and each keyword is arranged in order of importance to construct a keyword set.

5. The lighting effect display method according to claim 1, characterized in that: Controlling the light screen device to play the corresponding light effect according to the light effect image includes the following steps: Performing color enhancement processing on the lighting effect image according to the light source position distribution information of the light screen device to obtain an enhanced image, so as to reset the color of corresponding pixel points in the lighting effect image; Determine whether the resolution of the enhanced image meets a preset condition according to the light source position distribution information, and if so, reconstruct the enhanced image into a target lighting effect image that matches the light source position distribution information; Generate lighting effect control data matching the light screen device according to the target lighting effect image; The light screen device is controlled to play corresponding light effects according to the light effect control data.

6. A lighting effect display device, characterized in that: include: A text parsing module, used to determine the lighting effect content style and lighting effect motion type according to the lighting effect generation text, and extract a keyword set in the lighting effect generation text; An image matching module, configured to match the keyword set and the lighting effect image of the lighting effect motion type from a lighting effect library when the lighting effect content style belongs to a target style; An image generation module, configured to generate a lighting effect image according to the keyword set and the lighting effect motion type when no lighting effect image is matched from the lighting effect library; A lighting effect playing module, used for controlling the light screen device to play the corresponding lighting effect according to the lighting effect image, so as to display the lighting effect image through the light screen device; The step of generating a lighting effect image according to the keyword set and the lighting effect motion type includes: When the lighting effect motion type belongs to the static motion type, a text encoding module in a preset image-text matching model is used to determine the text semantic representation of the keyword set, and the text semantic representation is combined with the random noise image to obtain initial input data; Iteratively optimizing the random noise image in the initial input data using an image optimization module in a preset image generation model to obtain target implicit features; The image decoder in the image generation model reconstructs the lighting effect image from the target implicit features; The step of generating a lighting effect image according to the keyword set and the lighting effect motion type includes: When the lighting effect motion type belongs to a dynamic motion type, a preset multi-image frame generation model is used to generate multiple image frames according to the keyword set; The lighting effect image is formed by the multiple image frames.

7. A computer device comprising a central processing unit and a memory, characterized in that: The central processing unit is used to call and run the computer program stored in the memory to execute the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: It stores a computer program implemented according to the method described in any one of claims 1 to 5 in the form of computer-readable instructions, and when the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

Images