Lighting effect display method and its device, equipment and medium

Abstract

The present application relates to a lighting effect display method and its device, equipment, and medium in the field of lighting control, the method comprising: obtaining a lighting effect image, using an image classification model to determine the image type of the lighting effect image; when the image type is a type without text, performing corresponding semantic segmentation on the lighting effect image according to image composition features that match the specific type of the image type, and obtaining a preliminary image containing the foreground content object in the lighting effect image; when the image type is a type with text, performing image optimization processing on the lighting effect image with the text content removed, and redrawing the text content and removing the background content therein to obtain a preliminary image; pixelating the preliminary image to obtain a target lighting effect image, and controlling the light screen device to play the corresponding lighting effect according to the target lighting effect image, so as to display the target lighting effect image through the light screen device. The present application achieves matching of high-quality lighting effects displayed on light screen devices.

Description

Technical Field

[0001] The present application relates to the field of lighting control, and in particular to a lighting effect display method and its corresponding device, computer equipment, and computer-readable storage medium. Background Art

[0002] With the improvement of people's economic level, light screen equipment is becoming more and more popular. As a kind of intelligent lamp, light screen equipment can play the role of decorating indoor space and displaying information. The characteristic of light screen equipment is that it uses lamp beads with larger visual granularity as basic pixels. It is manifested in various forms of products through different implementation methods of its light screen, such as curtain lights, pixel screens, splicing lights, etc. One of the functions of light screen equipment is to display the corresponding lighting effects according to the given lighting effect image.

[0003] In traditional technology, corresponding lighting effect images are usually generated directly according to user needs, so as to display corresponding lighting effects on the light screen device according to the lighting effect images. Since the resolution of the generated lighting effect images is too high, in order to be able to display them on the light screen device, the central part of the lighting effect image is generally captured. However, lighting effect images of different image types require filtering of non-essential visual content, which cannot be simply captured. Therefore, this approach will lose a large amount of necessary visual content, seriously affecting the lighting effect display effect, making the user's intuitive visual experience poor.

[0004] It can be seen that the traditional light screen equipment has not achieved good results in lighting effect display, which affects the promotion of light screen equipment. It is necessary to upgrade the technology of light screen equipment. Summary of the invention

[0005] The primary purpose of the present application is to solve at least one of the above problems and provide a lighting effect display method and its corresponding device, computer equipment, and computer program product.

[0006] In order to meet the various objectives of this application, this application adopts the following technical solutions:

[0007] A lighting effect display method provided to meet one of the purposes of the present application includes the following steps:

[0008] Acquire a lighting effect image, and use an image classification model to determine the image type of the lighting effect image;

[0009] When the image type is a type without text, the lighting effect image is semantically segmented according to image composition features matching the specific type of the image type to obtain a preliminary image containing a foreground content object in the lighting effect image;

[0010] When the image type is a text-containing type, performing image optimization processing on the lighting effect image with the text content removed, and redrawing the text content and removing the background content therein to obtain a preliminary image;

[0011] The preliminary image is pixelated to obtain a target lighting effect image, and the light screen device is controlled to play a corresponding lighting effect according to the target lighting effect image, so as to display the target lighting effect image through the light screen device.

[0012] On the other hand, a lighting effect display device provided to meet one of the purposes of the present application includes an image classification module, a first image processing module, a second image processing module and a lighting effect playback module, wherein the image classification module is used to obtain a lighting effect image and use an image classification model to determine the image type of the lighting effect image; the first image processing module is used to, when the image type is a type without text, perform corresponding semantic segmentation on the lighting effect image according to image composition features matching the specific type of the image type, so as to obtain a preliminary image containing a foreground content object in the lighting effect image; the second image processing module is used to, when the image type is a type with text, perform image optimization processing on the lighting effect image with the text content removed, and redraw the text content and remove the background content therein to obtain a preliminary image; the lighting effect playback module is used to pixelate the preliminary image to obtain a target lighting effect image, and control the light screen device to play the corresponding lighting effect according to the target lighting effect image, so as to display the target lighting effect image through the light screen device.

[0013] On the other hand, a computer device provided to meet one of the purposes of the present application includes a central processing unit and a memory, wherein the central processing unit is used to call and run a computer program stored in the memory to execute the steps of the lighting effect display method described in the present application.

[0014] On the other hand, a computer program product provided to meet another purpose of the present application includes a computer program / instruction, which, when executed by a processor, implements the steps of the method described in any embodiment of the present application.

[0015] The technical solution of this application has many advantages, including but not limited to the following aspects:

[0016] This application adopts different processing strategies based on identifying whether the lighting effect image contains text content, ensures targeted semantic segmentation of the lighting effect image, and then performs pixelation processing after optimization to obtain the target lighting effect image and then control the light screen device to display the image, so that the target lighting effect image can be accurately displayed on the light screen, and can present a natural and colorful visual effect while ensuring the integrity of the image content. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0018] Figure 1 It is a schematic diagram of the principle of the electrical structure of the light screen device in the embodiment of the present application;

[0019] Figure 2 The structural form of the exemplary light screen device of this application;

[0020] Figure 3 A flowchart of a typical embodiment of the lighting effect display method of the present application;

[0021] Figure 4 This is a principle block diagram of the lighting effect display device of the present application;

[0022] Figure 5 A schematic diagram of the structure of a computer device used in this application. DETAILED DESCRIPTION

[0023] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.

[0024] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0025] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0026] See also Figure 1 As can be seen from the structural schematic diagram of a lamp screen device provided in an embodiment of the present application, the lamp screen device includes a controller 1 and a lamp screen 2. The lamp screen 2 is electrically connected to the controller 1 so as to accept the control of the computer program running in the controller 1 and work together to realize the lighting effect playback.

[0027] The controller 1 generally includes a control chip, a communication component, and a bus connector. In some embodiments, the controller 1 may also be configured with a power adapter, a control panel, a display screen, etc. as required.

[0028] The power adapter is mainly used to convert AC power into DC power to power the entire light screen device. The control chip can be implemented using various embedded chips, such as Bluetooth SoC (System on Chip), WiFi SoC, MCU (Micro Controller Unit), DSP (Digital Signal Processing) and other types of chips. The control chip usually includes a central processing unit and a memory, and the memory and the central processing unit are used to store and execute program instructions respectively to achieve corresponding functions. The above various types of control chips can have their own communication components, or they can be configured with additional communication components as needed. The communication component can be used to communicate with external devices, for example, it can communicate with terminal devices such as personal computers or various smart phones, so that after the user issues various configuration instructions through his terminal device, the control chip of the controller 1 can receive the configuration instructions through the communication component to complete the basic configuration so as to control the light screen. In addition, the controller 1 can also obtain the lighting effect application package encapsulating multiple lighting effect layers through the communication component. The bus connector is mainly used to connect the light screen 2 connected to the bus to the power supply and provide the light effect playing instruction, so the corresponding pins of the power bus and the signal bus are provided. Therefore, when the light screen 2 needs to be connected to the controller 1, it can be connected to the bus connector through the corresponding connector of the light screen. The control panel usually provides one or more buttons for implementing switch control of the controller 1, selecting various preset light effect control modes, etc. The display screen can be used to display various control information so as to cooperate with the buttons in the control panel to support the realization of human-computer interaction functions. In some embodiments, the control panel and the display screen can be integrated into the same touch display screen.

[0029] The specific appearance of the light screen can be designed according to actual conditions. The more common product forms include but are not limited to curtain lights, pixel screens, splicing lights, etc.

[0030] For example, Figure 2The light screen in the embodiment is arranged in the form of a curtain light. The light screen 2 includes a plurality of light strips 21 connected to a bus. Each light strip 21 includes a plurality of lamp beads 210 connected in series. Each lamp bead 210 serves as a light emitting unit. In addition, the number of lamp beads 210 in each light strip 21 is usually the same and arranged at equal intervals. When the light screen 2 used as a curtain light is used, it is usually arranged as follows: Figure 2 The layout shown unfolds each of the light strips 21, so that all the lamp beads in all the light strips 21 are arranged in an array to form a lamp bead matrix structure. Since all the lamp beads can provide a frame effect when they emit light in coordination, the surface where the entire lamp bead matrix structure is located constitutes a display frame 4. When the lighting effects are played, a certain pattern effect can be formed within the display frame 4.

[0031] Each light strip 21 can be composed of a plurality of lamp beads 210 connected in series, each lamp bead 210 is a light-emitting unit, and each lamp bead 210 in the same light strip 21 transmits working current through the same set of cables connected to the bus. The lamp beads 210 in the same light strip 21 can be connected in parallel in terms of electrical connection. In one embodiment, each light strip 21 in the same lamp bead matrix structure can be arranged at equal intervals along the bus direction, and the lamp beads 210 in each light strip 21 are also arranged correspondingly in number and position, so that the entire display frame 4 plays a role similar to a screen when viewing its luminous effect from a distance, and can form a pattern effect in the human eye.

[0032] Each lamp bead 210 of each light strip 21 of the light screen 2 is also provided with a corresponding control chip, which can be selected according to the above disclosure, or other more economical control chips can be selected, and its main function is to extract the luminous color value corresponding to the lamp bead 210 from the lighting effect playback instruction, and control the light-emitting element in the lamp bead 210 to emit the corresponding color light. The light-emitting element can be an LED lamp.

[0033] The implementation principle of the pixel screen is basically the same as that of the curtain light, but the pixel screen provides a frame to fix each light-emitting unit, and arranges each light-emitting unit in a row and column matrix in the same rectangular frame, so as to obtain a product form effect closer to the daily display screen. The pixel screen is similar to the curtain light, and each light-emitting unit can be organized according to a structure with multiple light strips arranged side by side, and can be controlled in the same way as the curtain, so I will not go into details.

[0034] The light screen device of the present application, when powered on, the control chip of the controller can call and execute a computer program from the memory, and through the default initialization process of the computer program, the light screen is powered on and initialized, thereby completing the driver configuration of the light screen and other hardware devices.

[0035] In one embodiment, when the controller starts the light screen, it can first send a self-test instruction to the light screen to drive each light strip or each lamp bead in the light block of the light screen to return its position information in the light strip or light block. Each lamp bead is provided with a corresponding control chip for data communication with the control chip in the controller, so that the characteristic information of the lamp bead itself and the characteristic information of other lamp beads can be serially connected in sequence according to the serial communication protocol to achieve the characterization of its own position information. The serial communication protocol executed between the controller and the lamp bead can be any one of IIC (Inter-Integrated Circuit, integrated circuit bus), SPI (serial peripheral interface, serial peripheral interface), and UART (Universal Asynchronous Receiver-Transmitter, universal asynchronous receiver-transmitter). After the controller obtains the result data returned by the self-test of each lamp bead from the bus, it parses the result data and determines the position of each lamp bead in the display frame 4 presented by the entire lamp screen according to the order of the characteristic information of each lamp bead in the result data. Thus, each lamp bead can be regarded as a light-emitting unit, which can be understood as a basic pixel. The position information of each lamp bead can be constructed as light source position distribution information. When constructing the lighting effect control data, the controller can set the corresponding light-emitting color value for each basic pixel according to the position information of each lamp bead, that is, each light-emitting unit, in the light source position distribution information according to actual needs.

[0036] In some embodiments, the controller 1 of the present application can be implemented in an independent computer device, as long as the computer device is equipped with the corresponding hardware of the controller 1, and the corresponding business logic of the controller 1, including the business logic executed by the method of the present application, is implemented as a computer program and installed in the computer device for operation. When the controller 1 is implemented in a computer device, various resources inherent to the computer device can be shared to save the overall implementation cost. The computer device referred to here can be any terminal device for use by the user, such as a smart phone, a personal computer, a laptop computer, a tablet computer, etc.

[0037] According to the product architecture and working principle of the above-mentioned light screen device, the lighting effect generation method of the present application can be implemented as a computer program product, stored in the memory of the controller of the light screen device, and called from the memory by the central processing unit and run, thereby controlling the light screen to play the corresponding lighting effects.

[0038] Any model of the present application can be deployed in a cloud server, a terminal device, or the controller of the light screen device of the present application, as long as the computing power of the deployed end is capable of meeting the operation requirements of the lighting effect generation model.

[0039] Please refer to Figure 3 , in a typical embodiment of a lighting effect display method of the present application, it includes the following steps:

[0040] Step S1100, obtain a lighting effect image, and use an image classification model to determine the image type of the lighting effect image;

[0041] In one embodiment, the user can provide an image corresponding to the desired lighting effect to be displayed on the light screen device as the lighting effect image.

[0042] In another embodiment, the user can provide corresponding lighting effect generation text according to their own lighting effect requirements. The lighting effect generation text is a description text for generating the desired lighting effect to be displayed on the light screen device. The provided method can include that the user directly inputs the lighting effect generation text expressed in natural language through an input method, or the user inputs it in the form of voice and converts the voice into the corresponding lighting effect generation text.

[0043] Call a preset first prompt template based on Few-shot learning. The first prompt template includes a task description, examples, and content of the lighting effect description to be embedded. Those skilled in the art can flexibly set the first prompt template with reference to the following disclosure. As an exemplary example of the first prompt template: The task description in the first prompt template: "Based on the specified structure and some provided examples, determine whether what the user wants is an 'abstract style' or a 'figurative style'."

[0044] 1. Judge according to keywords.

[0045] 2. Directly give the final result without any explanation.

[0046] Examples: "<example>{'input': 'Relaxing and pleasant lighting effect', 'output': 'Abstract style'}, {'input': 'Complex and beautiful', 'output': 'Abstract style'}, {'input': 'Surprising lighting colors', 'output': 'Abstract style'}, {'input': 'Comfortable ambient light', 'output': 'Abstract style'}, {'input': 'Approaching train lighting effect', 'output': 'Figurative style'}, {'input': 'Floating turtle pattern lighting effect', 'output': 'Figurative style'}, {'input': 'Rhythmic lighting effect', 'output': 'Figurative style'}< / example>"

[0047] Content of the lighting effect description to be embedded: "<input>$Input content to be judged$< / input>"

[0048] Embed the light effect description content into the first prompt template to obtain the corresponding prompt text, and input the prompt text into the large language model to obtain the light effect content style of the light effect generation text determined by the large language model.

[0049] In addition, call a preset second prompt template based on Few-shot learning. The second prompt template includes a task description, examples, and the light effect description content to be embedded. Those skilled in the art can refer to the following disclosure to flexibly set the second prompt template. As an exemplary example of the second prompt template: The task description in the second prompt template: "Based on the specified structure and some provided examples, determine whether the user wants a'static motion type' or a 'dynamic motion type'."

[0050] 1. Judge according to the keywords.

[0051] 2. Directly give the final result without any explanation.

[0052] Examples: "<example>{"input": "Relaxing and pleasant light effect", "output": "Static motion type"}, {"input": "Complex and beautiful", "output": "Static motion type"}, {"input": "Surprising light color", "output": "Static motion type"}, {"input": "Comfortable ambient light", "output": "Static motion type"}, {"input": "Approaching train light effect", "output": "Dynamic motion type"}, {"input": "Floating turtle pattern light effect", "output": "Dynamic motion type"}, {"input": "Light effect with strong rhythm", "output": "Dynamic motion type"}< / example>"

[0053] The light effect description content to be embedded: "<input>$The input content to be judged$< / input>"

[0054] Embed the light effect generation text into the second prompt template to obtain the corresponding prompt text, and input the prompt text into the large language model to obtain the light effect motion type of the light effect generation text determined by the large language model.

[0055] If the lighting effect motion type is a static motion type, in one embodiment, an open source text-based image generation model can be used to generate a corresponding image as a lighting effect image according to the keyword set. The image format of the lighting effect image can be PNG or JPG. The model can be any one of the DELL·E series model, ERNIE-ViLG series model, Midjourney, Stable Diffusion, eDiff-I, ModelScope, AltDiffusion, layerDiffusion, etc., and those skilled in the art can select one to implement as needed.

[0056] If the lighting effect motion type is a dynamic motion type, in one embodiment, an open source text-based model for generating multiple image frames can be used to generate an image file consisting of corresponding multiple image frames according to the keyword set as a lighting effect image. The image format of the lighting effect image can be any one of the video formats such as GIF or MP4, and the model can be any one of AnimateDiff, Gen-2, Stable Video Diffusion, Emu Video based on Stable Diffusion, etc., and technical personnel in this field can choose one to implement as needed.

[0057] The image classification model includes an image feature representation module and a classifier connected thereto. The image feature representation module is suitable for extracting and representing the image semantic features of the input image. The specific selection can be any readily available model such as the Resnet series, VGG, Mobilenet series, etc. In the recommended embodiment, the image feature representation module is a MobilenetV2 model, which has the advantages of being lightweight and highly accurate, so it is easy to deploy quickly, and can quickly and accurately extract the image semantic features of the image, and the corresponding required computing resources are relatively small. Of course, those skilled in the art can also build an image feature representation module by themselves, as long as it can extract the image semantic features of the input image. The classifier can be implemented using a fully connected layer or MLP, etc., and the fully connected layer can be a global convolution with convolution kernels h and w, where h and w are the height and width of the global convolution result, respectively. The image classification model is pre-trained to convergence and acquires the ability to determine the image type of the input image. Those skilled in the art can flexibly implement corresponding training of the image classification model according to the following reasoning process of the image classification model, so that it can acquire the ability after training to a convergence state.

[0058] In one embodiment, the image feature representation module in the image classification model adopts the MobilenetV2 model, which takes the lighting effect image as input, and first expands the low-dimensional compressed representation of the lighting effect image to a high-dimensional representation through the inverted residual structure of the linear bottleneck, and uses lightweight deep convolution (DW) to extract the image semantic features, and finally uses linear convolution to project the image semantic features back to the low-dimensional representation to obtain the corresponding image feature vector. The classifier in the image classification model performs multi-classification mapping on the image feature vector, and linearly maps the image feature vector to multiple classification spaces, each of which represents an image type, including portrait type, landscape type, text type, and other types, and obtains the classification probability corresponding to the mapping to each classification space, and determines the image type represented by the classification space with the largest classification probability as the image type of the lighting effect image.

[0059] Step S1200: When the image type is a type without text, semantic segmentation is performed on the lighting effect image according to image composition features matching the specific type of the image type to obtain a preliminary image containing a foreground content object in the lighting effect image;

[0060] The image composition feature refers to the content feature of the lighting effect image that constitutes the corresponding image type. When the image type is any one of the portrait type, the landscape type, and the other type, it means that the image type at this time belongs to the type without text.

[0061] In a further embodiment, any one or more of the following steps are included:

[0062] Step S1210: If the specific type of the image type is a portrait type, semantic segmentation of human body parts is performed on the lighting effect image according to human body composition features matching the portrait type, to obtain a preliminary image containing various human body parts in the lighting effect image;

[0063] If the image type is a portrait type, the image composition features of the lighting effect image matching the portrait type are the various parts of the human body, thereby performing semantic segmentation of the human body parts on the lighting effect image. In one embodiment, the Deeplabv3+ResNet34 model pre-trained to convergence is used to implement the semantic segmentation. Based on the 17 human body parts of the portrait in the lighting effect image segmented as foreground content objects and the background content in the lighting effect image, the corresponding mask image is inferred. The mask image is composed of color areas of the 17 human body parts and the background content, each color area contains the same color, and different color areas contain different colors. The color is usually represented by the numerical value of the three channels of R, G, and B. The mask image contains 18 colors, and the color corresponding to each color area can be set as needed by those skilled in the art according to the disclosure herein. The color of the color area of ​​the background content in the mask image is set to black (R: 0, G: 0, B: 0), so that the corresponding light-emitting unit does not need to emit light when the background content is subsequently displayed on the light screen device, which has the visual effect of removing the background content. The mask image after color setting is used as the preliminary image. The 17 human body parts include body, right hand, left hand, left foot, right foot, right thigh, right thigh, right calf, left calf, left arm, left arm, left forearm, right forearm, face, left eye, right eye and mouth.

[0064] Step S1220: If the specific type of the image type is a landscape type, semantic segmentation of landscape elements is performed on the lighting effect image according to landscape composition features matching the landscape type to obtain a preliminary image containing the landscape elements in the lighting effect image.

[0065] If the image type is a landscape type, the image composition features of the lighting effect image matching the landscape type are the various elements that make up the landscape, such as stars, mountains and rivers, people, flowers, plants, trees, etc., thereby semantically segmenting the various elements that make up the landscape for the lighting effect image. In one embodiment, the semantic segmentation is implemented by using a CropFormer model that has been pre-trained to a convergent state based on an open source EntitySeg data set. Based on the segmentation of the various elements that make up the landscape in the lighting effect image as foreground content objects and the background content in the lighting effect image, a corresponding mask image is inferred. The mask image is composed of the color regions of the various elements that make up the landscape and the background content. Each color region contains the same color, and different color regions contain different colors. The color is usually represented by the numerical values ​​of the three channels of R, G, and B. The mask image contains a corresponding plurality of colors, and the color corresponding to each color region can be flexibly set by those skilled in the art. The color of the area color of the background content in the mask image is set to black (R: 0, G: 0, B: 0), so that the corresponding light-emitting unit does not need to emit light when the background content is subsequently displayed on the light screen device, thereby achieving a visual effect of removing the background content. The mask image after color setting is used as the preliminary image.

[0066] If the image type is other types, the image constituent elements that match other types of lighting effect images are the foreground content objects and background content that constitute the composition. The foreground content object can be the content object other than the background content in the lighting effect image, or the content object obtained by further subdividing it on the basis of the foreground content object, thereby performing semantic segmentation of the foreground and background of the lighting effect image. In one embodiment, the IS-Net model pre-trained to convergence is used to implement the semantic segmentation. Based on the segmented background content in the lighting effect image and the foreground content objects other than the background content, the color of the area of ​​the background content in the lighting effect image is set to black (R: 0, G: 0, B: 0), so that the corresponding light-emitting unit does not need to emit light when the background content is subsequently displayed on the light screen device, achieving the visual effect of removing the background content. The lighting effect image after color setting is used as the preliminary image.

[0067] Step S1300: when the image type is a text-containing type, performing image optimization processing on the lighting effect image with the text content removed, and redrawing the text content and removing the background content therein to obtain a preliminary image;

[0068] At this time, in one embodiment, an open source text erasing model, such as EnsNet, EraseNet, MTRNet, and self-supervised text erasing, can be used to remove the text content in the lighting effect image. In the recommended embodiment, the EraseNet model is used as the text erasing model. The coarse removal network in the text erasing model uses the lighting effect image as input to quickly remove most of the text in the image, but may leave some residual text and / or unnatural background to obtain a first-stage erased image. The optimized subnetwork in the text erasing model receives the first-stage erased image output by the coarse removal network, removes the remaining text residue, and repairs the background, so that the area after the text is removed is more coordinated with the image content around it, to obtain a second-stage erased image, i.e., the lighting effect image with the text content removed.

[0069] Then, for the lighting effect image with the text content removed, an image quality scoring model is used, including an image feature representation module and a classifier connected thereto. The image feature representation module is suitable for extracting and representing the image semantic features of the input image. The specific selection can be any readily available model such as the Resnet series, VGG, Mobilenet series, etc. In the recommended embodiment, the image feature representation module is a MobilenetV2 model, which has the advantages of being lightweight and highly accurate, so it is easy to deploy quickly, and can quickly and accurately extract the image semantic features of the image, and the corresponding required computing resources are relatively small. Of course, those skilled in the art can also build an image feature representation module by themselves, as long as it can extract the image semantic features of the input image. The classifier can be implemented using a fully connected layer or MLP, etc. The fully connected layer can be a global convolution with convolution kernels h and w, where h and w are the height and width of the global convolution result, respectively. The image quality model is pre-trained to convergence and acquires the ability to determine the image quality score of the input image. Those skilled in the art can flexibly and adaptably implement corresponding training of the image quality scoring model according to the following reasoning process of the image quality scoring model, so that the image quality scoring model can acquire the aforementioned capability after being trained to a convergent state.

[0070] In one embodiment, the image feature representation module in the image quality scoring model adopts the MobilenetV2 model, which takes the lighting effect image as input, and first expands the low-dimensional compressed representation of the lighting effect image to a high-dimensional representation through the inverted residual structure of the linear bottleneck, and uses lightweight deep convolution (DW) to extract the image semantic features, and finally uses linear convolution to project the image semantic features back to the low-dimensional representation to obtain the corresponding image feature vector. The classifier in the image quality scoring model performs multi-classification mapping on the image feature vector, and linearly maps the image feature vector to two classification spaces, including a positive class space representing the highest image quality score and a negative class space representing the lowest image quality score, and obtains the classification probability corresponding to the mapping to each classification space, and determines the classification probability mapped to the positive class space as the image quality score of the lighting effect image.

[0071] When the image quality score of the lighting effect image is lower than the preset threshold, it means that the image quality of the lighting effect image is too low. The lighting effect image is optimized to improve the image quality of the lighting effect image. The processed lighting effect image is semantically segmented into foreground and background using an open source pre-trained image semantic segmentation model. Based on the segmented background content in the lighting effect image and the foreground content objects other than the background content, the color of the area of ​​the background content in the lighting effect image is set to black (R: 0, G: 0, B: 0), so that the corresponding light-emitting unit does not need to emit light when the background content is subsequently displayed on the light screen device, thereby achieving the visual effect of removing the background content. The lighting effect image after color setting is used as the preliminary image. The image optimization processing can be implemented using a super-resolution reconstruction algorithm, such as SRCNN (Super-Resolution Convolutional NeuralNetwork), ESRGAN (Enhanced Super-Resolution Generative Adversarial Networks), and any of the algorithms suitable for improving image quality. Those skilled in the art can flexibly implement it. The preset threshold can be set as needed by those skilled in the art according to the disclosure herein. The image semantic segmentation model can be MaskR-CNN, U-Net, DeepLabv3, etc.

[0072] Step S1400, pixelate the preliminary image to obtain a target lighting effect image, and control the light screen device to play a corresponding lighting effect according to the target lighting effect image, so as to display the target lighting effect image through the light screen device.

[0073] In one embodiment, the resolution of the light screen device can be determined according to the light source position distribution information, and the preliminary image can be divided into regions, the size of each region corresponds to a single pixel of the resolution, and for each region, the total number of pixels presenting each color is determined according to the color of each pixel in the region, and the color with the largest number of pixels is selected as the main color of the region. A pure white image of the resolution is constructed, and the color of each pixel in the pure white image is set as the main color of the region corresponding to the pixel, and pixelization is completed to obtain a target light effect image. When the image format of the target light effect image is any one of the video formats such as GIF or MP4, each image frame in the target light effect image can be framed, and each image frame can be converted into corresponding light effect control data one by one, and transmitted to the light screen device, so that the light screen device can be controlled to play the corresponding light effect; when the image format of the target light effect image is PNG or JPG, the target light effect image can be directly converted into corresponding light effect control data, and transmitted to the light screen device, so that the light screen device can be controlled to play the corresponding light effect.

[0074] For converting a single image into corresponding lighting effect control data, specifically, according to the relationship between the basic pixel distribution of the display frame / resolution of the light screen device and the resolution of the image, a partition mapping relationship between them can be established, so that the pixel points in the image and the basic pixel points of the display frame, that is, the light-emitting units, can be flexibly established in accordance with the proportional relationship of the pixel resolutions of each other. Many-to-one, many-to-many, one-to-many, one-to-one and other partition mapping relationships. According to this partition mapping relationship, the color of the pixel points of each partition on the image can be mapped to the color of the pixel points of the corresponding partition in the display frame, which is expressed in the lighting effect control data, that is, the control data of each light-emitting unit in the corresponding partition. It can be seen that the corresponding lighting effect control data can be generated according to the preset rules based on the image. If the resolution of the image is completely consistent with the display frame / resolution of the light screen device, in this case, the resolution of the image corresponds to the light-emitting unit in the display frame one-to-one, and the control data of the corresponding light-emitting unit can be directly generated according to the color of each pixel point in the image, thereby realizing the generation of lighting effect control data between the image and the display frame in a one-to-one partition mapping relationship. Whenever the light screen device receives the lighting effect control data, it transmits it to each light-emitting unit according to the data communication protocol followed between it and each light-emitting unit, such as the serial communication protocol. After each light-emitting unit receives the lighting effect control data, it extracts or intercepts the corresponding control data, controls its own light-emitting element to emit corresponding light according to its own control data, and forwards the lighting effect control data to other light-emitting units that have not received it. Each light-emitting unit receives, analyzes, responds to and forwards the lighting effect control data according to its inherent business logic, and the entire light screen device can present the visual effect of the target lighting effect image corresponding to the lighting effect control data.

[0075] It can be seen from the above embodiments that the present application has many advantages, including but not limited to:

[0076] This application adopts different processing strategies based on identifying whether the lighting effect image contains text content, ensures targeted semantic segmentation of the lighting effect image, and then performs pixelation processing after optimization to obtain the target lighting effect image and then control the light screen device to display the image, so that the target lighting effect image can be accurately displayed on the light screen, and can present a natural and colorful visual effect while ensuring the integrity of the image content.

[0077] In a further embodiment, before performing image optimization processing on the lighting effect image with the text content removed in step S1300, the following steps are included:

[0078] Step S2300: using a text mask prediction module in a preset text removal model to predict a text mask of the lighting effect image;

[0079] The text removal model is pre-trained to convergence and acquires the ability to perform end-to-end text erasure on the input image. Those skilled in the art can implement the training of the model to convergence and acquire this ability according to the reasoning process of the model disclosed later. The text mask prediction module is a full convolution structure of encoding and decoding, which encodes the input image by three downsampling convolution layers and four residual blocks, and decodes the feature map from the residual block by three upsampling transposed convolution layers.

[0080] The text removal model takes the lighting effect image as input, and the text mask prediction module in the model extracts the deep semantic information of the lighting effect image, obtains the image feature vector that vectorizes the deep semantic information, detects the position of the text content in the lighting effect image based on the image feature vector, and predicts the corresponding text mask.

[0081] Step S2310: Fill and repair the text mask by the mask background repair module in the text removal model to remove the text content in the lighting effect image and repair the background content in the corresponding text area.

[0082] The mask background restoration module includes an encoder of three down-sampling partial convolutional layers, a self-attention block, and three up-sampling partial convolutional layers, which are connected to the text mask prediction module.

[0083] The mask background repair module first convolves the text mask predicted by the text mask prediction module through three downsampling partial convolution layers, then adds a self-attention block, and finally repairs the background content lost by removing the text content in the image through an encoder of three upsampling partial convolution layers, so that more texture information is retained while removing the text area where the text content is located, and the background texture other than the text in the area is repaired.

[0084] In this embodiment, the process of removing text content from a lighting effect image is disclosed, ensuring the complete removal of the text content and guaranteeing a natural transition between the corresponding background content and other image content, thereby guaranteeing the overall aesthetics and visual continuity of the lighting effect image with the text content removed.

[0085] In a further embodiment, the step S1300 of performing image optimization processing on the lighting effect image with the text content removed includes the following steps:

[0086] Step S1310: obtaining text content and text area information in the lighting effect image, and adjusting the text style of the text content according to preset text adjustment parameters to obtain text content after the text style adjustment;

[0087] Before removing the text content in the lighting effect image, the open source pre-trained DBNet model and CRNN model are used to detect the text in the lighting effect image and recognize the text content, and the coordinate information of the text content in the lighting effect image and the text box in which it is located is obtained, including the maximum horizontal coordinate, maximum vertical coordinate, minimum horizontal coordinate, and minimum vertical coordinate of the corresponding text content in the box. The coordinate information of the text box is used as the text area information.

[0088] The preset text adjustment parameters include any one or more of the parameters related to the style of the edited text, such as character spacing, single character width, single character height, contrast color set, font, bold, underline, etc. The technicians in this field can flexibly and flexibly adjust the text style of the text content according to the text adjustment parameters. In an exemplary embodiment, the preset text adjustment parameters include character spacing, single character width, and contrast color set. The style of the text content is adjusted accordingly according to the text adjustment parameters. In the adjusted text content, the spacing between every two adjacent characters is the character spacing, the width of each character is the single character width, and the height of each character is the original height of the character multiplied by the single character width and then divided by the original width. For each character, a different contrast color from the previous one is selected from the contrast color set each time as the color of the character, so that the colors of every two adjacent characters are different. The original height is the value obtained by subtracting the minimum ordinate from the maximum ordinate in the text area information.

[0089] The specific values ​​of the character spacing and the single character width can be set by those skilled in the art as required.

[0090] For the preset contrast color set, specifically, the Kmeans algorithm is used as a clustering algorithm to perform color clustering on the lighting effect image with the text content removed, and all colors represented by R, G, and B in the lighting effect image constitute the input data set of the clustering algorithm. The center points of three theme clusters are randomly determined (not necessarily the layout feature vectors in the data set), and each color in the data set is assigned to a theme cluster. Specifically, taking color as an example, the Euclidean distance between the color and each center point is calculated, and the color is assigned to the theme cluster corresponding to the center point with the closest distance. Accordingly, after each color is assigned to a corresponding theme cluster, the center point of each theme cluster is updated to the average value of all colors in the theme cluster, and the above process is repeated until all colors in the data set are closest to their corresponding center points, and clustering is completed. The colors closest to the center point of each cluster are selected from the three clusters, and each of these colors is used as a core color, and all core colors are combined to form a core color set. In one embodiment, each core color in the core color set is converted to be represented by H, S, and V. For each core color, a color that is one hundred eighty degrees away from the core color in the H, S, and V hue space is used as a contrast color of the core color, and the three contrast colors thus obtained constitute a contrast color set.

[0091] Step S1320, scaling the lighting effect image with the text content removed accordingly according to the corresponding scaling ratios before and after the text content is adjusted to obtain a scaled lighting effect image;

[0092] The scaling ratio can be obtained by dividing the width of the text content after adjustment by the width before adjustment. It can be understood that the width of the text content after adjustment is the sum of the width of each character in the text content plus the sum of the character spacing between every two adjacent characters, and the width of the text content before adjustment is the value obtained by subtracting the minimum horizontal coordinate from the maximum horizontal coordinate in the text area information.

[0093] The width of the lighting effect image with the text content removed is scaled to a value obtained by multiplying the width by a scaling ratio, and the height of the lighting effect image is scaled to a value obtained by multiplying the height by the scaling ratio, thereby obtaining a scaled lighting effect image.

[0094] Step S1330: determining text area information of the text content in the scaled lighting effect image according to the scaling ratio;

[0095] The maximum horizontal coordinate, the maximum vertical coordinate, the minimum horizontal coordinate, and the minimum vertical coordinate in the coordinate information in the text area information are multiplied by the scaling ratio respectively, and the corresponding product results are set to form the text area information of the text content in the scaled lighting effect image.

[0096] Step S1340: according to the text area information of the text content in the scaled lighting effect image, draw the text content after the text style is adjusted into the scaled lighting effect image to obtain a preliminary image.

[0097] According to the text area information of the text content in the scaled lighting effect image, the specific position of the text content after the text style adjustment in the scaled lighting effect image can be determined, and the adjusted text content can be redrawn in this specific position to obtain a preliminary image.

[0098] In this embodiment, by first removing the text content in the lighting effect image, then adjusting the text style of the text content, and then redrawing the adjusted text content, the fusion of the text content and the lighting effect image is made more natural and coordinated, while maintaining the readability of the text and enhancing the information transmission capability of the lighting effect image.

[0099] In a further embodiment, before the text content is adjusted in text style according to preset text adjustment parameters in step S1310 and the text content after the text style adjustment is obtained, the following steps are included:

[0100] Step S2300, determining that a plurality of core colors in the lighting effect image excluding text content constitute a core color set;

[0101] The Kmeans algorithm is used as a clustering algorithm to perform color clustering on the lighting effect image with the text content removed. All colors represented by R, G, and B in the lighting effect image constitute the input data set of the clustering algorithm. The center points of three theme clusters are randomly determined (not necessarily the layout feature vectors in the data set), and each color in the data set is assigned to a theme cluster. Specifically, taking color as an example, the Euclidean distance between the color and each center point is calculated, and the color is assigned to the theme cluster corresponding to the center point with the closest distance. Accordingly, after each color is assigned to a corresponding theme cluster, the center point of each theme cluster is updated to the average value of all colors in the theme cluster. The above process is repeated until all colors in the data set are closest to their corresponding center points, and clustering is completed. The colors closest to the center point of each cluster are selected from the three clusters, and each of these colors is used as a core color. All core colors are combined to form a core color set.

[0102] Step S2310: Determine a standard color from a preset standard color set, the standard color having a perceived difference from any core color in the core color set that meets a preset condition, use the standard color as a contrast color, and form a contrast color set with all contrast colors;

[0103] In one embodiment, seven standard colors are preset, namely red: R: 255, G: 0, B: 0, orange: R: 255, G: 125, B: 0, yellow: R: 255, G: 255, B: 0, green: R: 0, G: 255, B: 0, cyan: R: 0, G: 255, B: 255, blue: R: 0, G: 0, B: 255, purple: R: 255, G: 0, B: 255, and all standard colors are combined to form a standard color set.

[0104] The core color set and the standard color set are nested and traversed. Further, each core color in the core color set and each standard color in the standard color set are converted to L, a, and b, and the CIEDE2000 color difference formula is used to calculate the perceived difference between each core color in the core color set represented by L, a, and b and each standard color in the standard color set represented by L, a, and b. From the standard color set, standard colors whose perceived difference with each core color in the core color set is greater than or equal to a preset threshold are selected as contrast colors, and at least four such contrast colors are combined to form a contrast color set. The preset threshold can be set as required by those skilled in the art.

[0105] Step S2320: Using the contrast color set as a text adjustment parameter to adjust the font color of the text content.

[0106] In one embodiment, each contrast color in the contrast color set can be traversed in a loop, and the contrast color traversed each time can be set as the font color of each character in the text content, thereby adjusting the font color of the text content so that the font colors of adjacent characters in the adjusted text content are different.

[0107] In this embodiment, the construction process of the contrast color set is disclosed to ensure that each contrast color therein has an obvious difference in visual perception from each core color in the lighting effect image with the text content removed.

[0108] In a further embodiment, pixelating the preliminary image to obtain the target lighting effect image in step S1400 includes the following steps:

[0109] Step S1410, dividing the preliminary image into regions with a preset color block size as a pixel unit, and determining a plurality of regions to be pixelated;

[0110] The width of the preliminary image is divided by the width of the resolution of the light screen device to obtain the width of the preset color block size, and the height of the preliminary image is divided by the height of the resolution of the light screen device to obtain the height of the preset color block size, thereby obtaining the preset color block size.

[0111] It is easy to understand that the preliminary image can be divided into a plurality of regions of the preset color block size, and each region is used as a region to be pixelated.

[0112] Step S1420, determining the main color of each area to be pixelated in the preliminary image, calling a preset color block image matching the main color from a preset color block image set and placing it in the area to be pixelated;

[0113] The preset color block atlas includes preset color blocks of multiple preset color block sizes, each preset color block has a different color, and each preset color block is a pure color image of its color. Those skilled in the art can flexibly pre-construct the preset color block atlas, or implement it according to the subsequent embodiments. In one embodiment, a variable K is set to be initially 0, and the variable K is controlled to increase by increment A each time until it exceeds 255, and the variable K initially 0 and the variable K after each increase are obtained as elements in the array to construct the array, and then the array is used to copy two identical arrays, and the three arrays are nested and traversed, and each time one of the values ​​is obtained from each of the three arrays, and a group of RGB corresponding to the specific values ​​of R, G, and B is obtained, and finally each group of R, G, and B is exhaustively enumerated, and for each group of R, G, and B The color represented by the color, constructs a pure white image of the preset color block size, and the color of all pixels of the image is set to the color represented by the group of R, G, and B, and the pure color image of the color is obtained as a single preset color block image. All the preset color block images are collected to form a preset color block atlas. For ease of understanding, an exemplary example is given. For example, if the increment A is 25, the values ​​of the three identical numerical values ​​are [0, 51, 102, 153, 204, 255], and 125 groups of R, G, and B can be enumerated.

[0114] For each area to be pixelated, the total number of pixels presenting each color is determined according to the color of each pixel in the area to be pixelated, and the color with the largest number of pixels is screened out as the main color of the area to be pixelated. The Euler distance between the main color and the color of each preset color block image in the preset color block image set is calculated, and the preset color block with the highest Euler distance is screened out and placed in the area to be pixelated.

[0115] Step S1430: Combine the preset color block images placed in each area to be pixelated to obtain a target lighting effect image.

[0116] It can be understood that by keeping the preset color block images placed in each area to be pixelated in place and directly splicing them together, an image with the same resolution as the light screen device can be obtained, and the image can be used as the target lighting effect image.

[0117] In this embodiment, it is disclosed that after the preliminary image is pixelated, the target lighting effect image displayed on the light screen device is obtained, which simplifies the complexity of the preliminary image while maintaining the main color features of the image, so that the target lighting effect image is displayed relatively clearly and naturally on the light screen device.

[0118] In a further embodiment, pixelating the preliminary image to obtain the target lighting effect image in step S1400 includes the following steps:

[0119] Step S2400, obtaining a pixelated preliminary image and a standard color set, and determining that a plurality of main colors in the preliminary image constitute a main color set;

[0120] The Kmeans algorithm is used as a clustering algorithm to perform color clustering on the pixelated lighting effect image. All colors represented by R, G, and B in the lighting effect image constitute the input data set of the clustering algorithm. The center points of five theme clusters are randomly determined (not necessarily the layout feature vectors in the data set), and each color in the data set is assigned to a theme cluster. Specifically, taking color as an example, the Euclidean distance between the color and each center point is calculated, and the color is assigned to the theme cluster corresponding to the center point with the closest distance. Accordingly, after each color is assigned to a corresponding theme cluster, the center point of each theme cluster is updated to the average value of all colors in the theme cluster. The above process is repeated until all colors in the data set are closest to their corresponding center points, and clustering is completed. The colors closest to the center point of each cluster are selected from the five clusters, and each of these colors is used as the main color, and all the main colors are combined to form a main color set.

[0121] Seven standard colors are preset, namely red: R:255,G:0,B:0, orange: R:255,G:125,B:0, yellow: R:255,G:255, B:0, green: R:0,G:255, B:0, cyan: R: 0,G:255, B:255, blue: R:0,G:0, B:255, purple: R:255,G:0,B:255. All standard colors constitute a standard color set.

[0122] Step S2410: Determine, from the standard color set, a standard color that matches any main color in the main color set, and form an enhanced color set with all unmatched standard colors;

[0123] By calculating the Euler distance between each main color in the main color set and each standard color in the standard color set, it is determined that the main color and the standard color corresponding to the Euler distance exceeding the preset threshold match each other. The preset threshold can be set by those skilled in the art as needed.

[0124] Step S2420: When a preset target color exists in the main color set, a region of the preliminary image containing the target color is determined as a target region;

[0125] The target color is the maximum value minus the minimum value among the corresponding R, G, and B, which exceeds the preset threshold. Based on this, it is calculated whether the maximum value minus the minimum value among R, G, and B corresponding to each main color in the main color set exceeds the preset threshold. When there is a main color in the main color set that exceeds the preset threshold, such a main color is regarded as the target color, and the color of each pixel in the preliminary image is traversed, and the area composed of all pixels with the color of the target color is screened out and confirmed as the target area.

[0126] The preset threshold can be set by those skilled in the art as needed, for example, 3.

[0127] Step S2430: When the target color in the target area matches any standard color in the enhanced color set, the target color in the target area is reset to the standard color to obtain a target lighting effect image.

[0128] The difference between the sum of R, G, and B of the target color in the target area and the sum of R, G, and B of each standard color in the enhanced color set is calculated. When the difference is less than a preset threshold, it is confirmed that the corresponding target color matches the standard color, and the target color in the target area is reset to the standard color, thereby obtaining a target lighting effect image.

[0129] In this embodiment, the target area containing the target color in the pixelated preliminary image is further replaced with a color different from all the main colors in the image, which can highlight the visual effect of the target area and enhance the attractiveness of the lighting effect image.

[0130] In a further embodiment, pixelating the preliminary image to obtain the target lighting effect image in step S1400 includes the following steps:

[0131] Step S3400, obtaining a pixelated preliminary image, and when there is an area smaller than a preset area in the preliminary image, taking the area as a target area;

[0132] In one embodiment, the Kmeans algorithm is used as a clustering algorithm to perform color clustering on the pixelated lighting effect image, and all colors represented by R, G, and B in the lighting effect image constitute the input data set of the clustering algorithm. The center points of K theme clusters are randomly determined (not necessarily the layout feature vectors in the data set), and each color in the data set is assigned to a theme cluster. Specifically, taking color as an example, the Euclidean distance between the color and each center point is calculated, and the color is assigned to the theme cluster corresponding to the center point with the closest distance. Accordingly, after each color is assigned to a corresponding theme cluster, the center point of each theme cluster is updated to the average value of all colors in the theme cluster, and the above process is repeated until all colors in the data set are closest to the corresponding center point, and clustering is completed. The color closest to the center point of each cluster is selected from the K clusters, and each of these colors is used as the main color, and all the main colors are combined to form a main color set. The K value is determined by the elbow analysis method, so that the number of clustered theme clusters is more reliable, thereby ensuring the accuracy of clustering. For each theme cluster, the color of each pixel in the preliminary image is traversed, and the area formed by all pixels whose colors belong to the color in the theme cluster is determined as the target area, and the colors of all pixels in the outermost contour of the target area. The outermost contour of the target area can be determined according to the position of each pixel in the target area.

[0133] In another embodiment, the Sobel algorithm is used to detect the edges of each region in the preliminary image, and the area of ​​each region is obtained after connected region analysis. When there is an area smaller than the area of ​​a preset region, the area is taken as the target area, and the colors of all pixel points of the outermost contour of the target area are obtained.

[0134] Step S3410: determine the edge color of the target area, reset all colors in the target area to the edge color, and obtain a target lighting effect image.

[0135] According to the colors of all pixels of the outermost contour of the target area, the total number of pixels displaying each color is determined, the color of the pixel with the largest total number of pixels is screened out as the edge color, all colors in the target area are reset to the edge color, and the target lighting effect image is obtained.

[0136] In this embodiment, the edge colors of smaller areas in the pixelated preliminary image are further unified, which naturally transitions the non-focus content of the image, making the target lighting effect image more concise and prominent when displayed on the light screen, thereby enhancing the visual experience.

[0137] In a further embodiment, pixelating the preliminary image to obtain the target lighting effect image in step S1400 includes the following steps:

[0138] Step S4400, obtaining a pixelated preliminary image and a standard color set, and determining that a plurality of main colors in the preliminary image constitute a main color set;

[0139] The Kmeans algorithm is used as a clustering algorithm to perform color clustering on the pixelated lighting effect image. All colors represented by R, G, and B in the lighting effect image constitute the input data set of the clustering algorithm. The center points of five theme clusters are randomly determined (not necessarily the layout feature vectors in the data set), and each color in the data set is assigned to a theme cluster. Specifically, taking color as an example, the Euclidean distance between the color and each center point is calculated, and the color is assigned to the theme cluster corresponding to the center point with the closest distance. Accordingly, after each color is assigned to a corresponding theme cluster, the center point of each theme cluster is updated to the average value of all colors in the theme cluster. The above process is repeated until all colors in the data set are closest to their corresponding center points, and clustering is completed. The colors closest to the center point of each cluster are selected from the five clusters, and each of these colors is used as the main color, and all the main colors are combined to form a main color set.

[0140] Seven standard colors are preset, namely red: R:255,G:0,B:0, orange: R:255,G:125,B:0, yellow: R:255,G:255, B:0, green: R:0,G:255, B:0, cyan: R: 0,G:255, B:255, blue: R:0,G:0, B:255, purple: R:255,G:0,B:255. All standard colors constitute a standard color set.

[0141] Step S4410: Determine a standard color that matches any main color in the main color set from the standard color set, and form an enhanced color set with all unmatched standard colors;

[0142] By calculating the Euler distance between each main color in the main color set and each standard color in the standard color set, it is determined that the main color and the standard color corresponding to the Euler distance exceeding the preset threshold match each other. The preset threshold can be set by those skilled in the art as needed.

[0143] Step S4420, determining the area in the preliminary image that contains any of the main colors, and taking the remaining area as the target area;

[0144] The color of each pixel in the preliminary image is traversed, and the area formed by all pixels of non-main colors is screened out to be confirmed as the target area.

[0145] Step S4430: reset all colors in the target area to any standard color in the enhanced color set to obtain a target lighting effect image.

[0146] For a single target area, a single standard color from the enhanced color set is randomly selected, and the target color in the target area is reset to the standard color, thereby obtaining a target lighting effect image; for multiple target areas, for each target area, a single standard color different from the previous one can be selected from the enhanced color set each time, and the target color in the target area can be reset to the standard color.

[0147] In this embodiment, by further replacing the areas containing non-main colors in the pixelated preliminary image with colors different from all the main colors in the image, the non-main color areas can also be highlighted, greatly enhancing the expressiveness and visual appeal of the lighting effect image.

[0148] See also Figure 4 , a lighting effect display device provided to meet one of the purposes of the present application is a functional embodiment of the lighting effect display method of the present application. On the other hand, the device is a lighting effect display device provided to meet one of the purposes of the present application, including an image classification module 1100, a first image processing module 1200, a second image processing module 1300 and a lighting effect playback module 1400, wherein the image classification module 1100 is used to obtain a lighting effect image and use an image classification model to determine the image type of the lighting effect image; the first image processing module 1200 is used to, when the image type belongs to a type without text, according to the specific information related to the image type. The lighting effect image is semantically segmented according to the image composition features that match the type, and a preliminary image containing the foreground content object in the lighting effect image is obtained; a second image processing module 1300 is used for, when the image type belongs to a text-containing type, performing image optimization processing on the lighting effect image with the text content removed, and redrawing the text content and removing the background content therein to obtain a preliminary image; a lighting effect playback module 1400 is used for pixelating the preliminary image to obtain a target lighting effect image, and controlling the light screen device to play the corresponding lighting effect according to the target lighting effect image, so as to display the target lighting effect image through the light screen device.

[0149] In a further embodiment, the second image processing module 1300 includes: a mask prediction submodule, which is used to predict the text mask of the lighting effect image by using a text mask prediction module in a preset text removal model; and a text removal submodule, which is used to fill and repair the text mask by a mask background repair module in the text removal model to remove the text content in the lighting effect image and repair the background content in the corresponding text area.

[0150] In a further embodiment, the second image processing module 1300 includes: a text area information updating submodule, used to obtain the text content and its text area information in the lighting effect image, and adjust the text style of the text content according to preset text adjustment parameters to obtain the text content after the text style adjustment; an image scaling submodule, used to scale the lighting effect image with the text content removed accordingly according to the corresponding scaling ratio before and after the text content adjustment to obtain the scaled lighting effect image; a text area information determining submodule, used to determine the text area information of the text content in the scaled lighting effect image according to the scaling ratio; a text redrawing submodule, used to draw the text content after the text style adjustment into the scaled lighting effect image according to the text area information of the text content in the scaled lighting effect image to obtain a preliminary image.

[0151] In a further embodiment, the lighting effect playback module 1400 includes: an area determination submodule, used to divide the preliminary image into regions with a preset color block size as a pixel unit, and determine a plurality of regions to be pixelated; a color block filling submodule, used to determine the main color of each region to be pixelated in the preliminary image, and call a preset color block image matching the main color from a preset color block image set and place it in the region to be pixelated; a color block image combination submodule, used to combine the preset color block images placed in each region to be pixelated to obtain a target lighting effect image.

[0152] In a further embodiment, the lighting effect playback module 1400 includes: a main color set composition submodule, used to obtain a pixelated preliminary image and a standard color set, and determine that multiple main colors in the preliminary image constitute a main color set; an enhanced color set composition submodule, used to determine, from the standard color set, a standard color that matches any main color in the main color set, and to form an enhanced color set with all unmatched standard colors; a first target area determination submodule, used to determine, when there is a preset target color in the main color set, an area in the preliminary image containing the target color as a target area; a first color reset submodule, used to reset the target color in the target area to the standard color when the target color in the target area matches any standard color in the enhanced color set, so as to obtain a target lighting effect image.

[0153] In a further embodiment, the lighting effect playback module 1400 includes: a second target area determination submodule, used to obtain a preliminary image after pixelation, when there is an area smaller than a preset area in the preliminary image, the area is used as the target area; a second color reset submodule, used to determine the edge color of the target area, reset all colors in the target area to the edge color, and obtain a target lighting effect image.

[0154] In a further embodiment, the lighting effect playback module 1400 includes: a main color set composition submodule, used to obtain a pixelated preliminary image and a standard color set, and determine that multiple main colors in the preliminary image constitute a main color set; an enhanced color set composition submodule, which determines, from the standard color set, a standard color that matches any main color in the main color set, and forms an enhanced color set with all unmatched standard colors; a third target area determination submodule, used to determine an area in the preliminary image that contains any of the main colors, and uses the remaining area as a target area; a third color reset submodule, used to reset all colors in the target area to any standard color in the enhanced color set, and obtain a target lighting effect image.

[0155] In order to solve the above technical problems, the present application also provides a computer device. Figure 5 As shown, a schematic diagram of the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. Among them, the computer-readable storage medium of the computer device stores an operating system, a database, and computer-readable instructions. The database may store a control information sequence. When the computer-readable instructions are executed by the processor, the processor can implement a lighting effect display method. The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The memory of the computer device may store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor can execute the lighting effect display method of the present application. The network interface of the computer device is used to connect and communicate with a terminal. Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0156] In this embodiment, the processor is used to execute Figure 4 The memory stores the program code and various data required to execute the above modules or submodules. The network interface is used to transmit data between user terminals or servers. The memory in this embodiment stores the program code and data required to execute all modules / submodules in the lighting effect display device of this application, and the server can call the program code and data of the server to execute the functions of all submodules.

[0157] The present application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the lighting effect display method of any embodiment of the present application.

[0158] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments of the present application can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a computer-readable storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0159] In summary, the present application achieves high-quality lighting effects displayed on matching light screen devices.

[0160] It will be understood by those skilled in the art that the various operations, methods, steps, measures, and schemes in the processes discussed in this application may be alternated, changed, combined, or deleted. Furthermore, other steps, measures, and schemes in the various operations, methods, and processes discussed in this application may also be alternated, changed, rearranged, decomposed, combined, or deleted. Furthermore, the steps, measures, and schemes in the various operations, methods, and processes in the prior art that are open source and in this application may also be alternated, changed, rearranged, decomposed, combined, or deleted.

[0161] The above description is only a partial implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A lighting effect display method, characterized in that: The steps include: Acquire a lighting effect image, and use an image classification model to determine the image type of the lighting effect image; When the image type is a type without text, the lighting effect image is semantically segmented according to image composition features matching the specific type of the image type to obtain a preliminary image containing a foreground content object in the lighting effect image; When the image type is a text-containing type, performing image optimization processing on the lighting effect image with the text content removed, and redrawing the text content and removing the background content therein to obtain a preliminary image; Pixelating the preliminary image to obtain a target lighting effect image, and controlling the light screen device to play a corresponding lighting effect according to the target lighting effect image, so as to display the target lighting effect image through the light screen device; The step of performing image optimization processing on the lighting effect image with the text content removed includes: Acquire text content and text area information in the lighting effect image, perform text style adjustment on the text content according to preset text adjustment parameters, and obtain text content after text style adjustment; According to the corresponding scaling ratios before and after the text content is adjusted, the lighting effect image with the text content removed is scaled accordingly to obtain a scaled lighting effect image; Determining text area information of the text content in the scaled lighting effect image according to the scaling ratio; According to the text area information of the text content in the scaled lighting effect image, the text content after the text style is adjusted is drawn into the scaled lighting effect image to obtain a preliminary image.

2. The lighting effect display method according to claim 1, characterized in that: According to the image composition features matching the specific type of the image type, the lighting effect image is semantically segmented accordingly to obtain a preliminary image containing the foreground content object in the lighting effect image, including any one or more of the following steps: If the specific type of the image type is a portrait type, semantic segmentation of human body parts is performed on the lighting effect image according to human body composition features matching the portrait type, to obtain a preliminary image containing various human body parts in the lighting effect image; If the specific type of the image type is a landscape type, semantic segmentation of landscape elements is performed on the lighting effect image according to landscape composition features matching the landscape type to obtain a preliminary image containing the landscape elements in the lighting effect image.

3. The lighting effect display method according to claim 1, characterized in that: Before performing image optimization processing on the lighting effect image with the text content removed, the following steps are included: Using a text mask prediction module in a preset text removal model to predict a text mask of the lighting effect image; The text mask is filled and repaired by the mask background repair module in the text removal model to remove the text content in the lighting effect image and repair the background content in the corresponding text area.

4. The lighting effect display method according to claim 1, characterized in that: Before adjusting the text style of the text content according to the preset text adjustment parameters and obtaining the text content after the text style adjustment, the following steps are included: Determine that multiple core colors in the lighting effect image excluding text content constitute a core color set; From a preset standard color set, determine a standard color whose perceived difference from any core color in the core color set meets a preset condition, use the standard color as a contrast color, and form a contrast color set with all contrast colors; The contrast color set is used as a text adjustment parameter to adjust the font color of the text content.

5. The lighting effect display method according to claim 1, characterized in that: Pixelating the preliminary image to obtain a target lighting effect image comprises the following steps: Taking a preset color block size as a pixel unit, dividing the preliminary image into regions to determine a plurality of regions to be pixelated; Determine the main color of each area to be pixelated in the preliminary image, and call a preset color block image matching the main color from a preset color block image set and place it in the area to be pixelated; The preset color block images placed in each area to be pixelated are combined to obtain the target lighting effect image.

6. The lighting effect display method according to claim 1, characterized in that: Pixelating the preliminary image to obtain a target lighting effect image comprises the following steps: Obtaining a pixelated preliminary image and a standard color set, and determining that a plurality of main colors in the preliminary image constitute a main color set; From the standard color set, determine a standard color that matches any primary color in the primary color set, and form an enhanced color set with all unmatched standard colors; When a preset target color exists in the main color set, determining a region in the preliminary image containing the target color as a target region; When the target color in the target area matches any standard color in the enhanced color set, the target color in the target area is reset to the standard color to obtain a target lighting effect image.

7. The lighting effect display method according to claim 1, characterized in that: Pixelating the preliminary image to obtain a target lighting effect image comprises the following steps: Acquire a pixelated preliminary image, and when there is an area smaller than a preset area in the preliminary image, take the area as a target area; The edge color of the target area is determined, and all colors in the target area are reset to the edge color to obtain a target lighting effect image.

8. The lighting effect display method according to claim 1, characterized in that: Pixelating the preliminary image to obtain a target lighting effect image comprises the following steps: Obtaining a pixelated preliminary image and a standard color set, and determining that a plurality of main colors in the preliminary image constitute a main color set; From the standard color set, determine a standard color that matches any primary color in the primary color set, and form an enhanced color set with all unmatched standard colors; Determine the area in the preliminary image that contains any of the main colors, and use the remaining area as the target area; All colors in the target area are reset to any standard color in the enhanced color set to obtain a target lighting effect image.

9. A lighting effect display device, characterized in that: include: An image classification module is used to obtain a lighting effect image and determine the image type of the lighting effect image using an image classification model; A first image processing module is used for, when the image type is a type without text, performing corresponding semantic segmentation on the lighting effect image according to image composition features matching a specific type of the image type, to obtain a preliminary image containing a foreground content object in the lighting effect image; A second image processing module is used for, when the image type is a text-containing type, performing image optimization processing on the lighting effect image with the text content removed, and redrawing the text content and removing the background content therein to obtain a preliminary image; A lighting effect playing module, used for pixelating the preliminary image to obtain a target lighting effect image, and controlling the light screen device to play a corresponding lighting effect according to the target lighting effect image, so as to display the target lighting effect image through the light screen device; The step of performing image optimization processing on the lighting effect image with the text content removed includes: Acquire text content and text area information in the lighting effect image, perform text style adjustment on the text content according to preset text adjustment parameters, and obtain text content after text style adjustment; According to the corresponding scaling ratios before and after the text content is adjusted, the lighting effect image with the text content removed is scaled accordingly to obtain a scaled lighting effect image; Determining text area information of the text content in the scaled lighting effect image according to the scaling ratio; According to the text area information of the text content in the scaled lighting effect image, the text content after the text style is adjusted is drawn into the scaled lighting effect image to obtain a preliminary image.

Images