Light effect control method and apparatus, device, medium, product

Abstract

The application relates to a lamp effect control method and device, equipment, medium and product, the method comprising: acquiring a displayed interface image; querying a pre-stored image feature corresponding lamp effect type of the interface image from an image lamp effect library, the image feature being matched with a to-be-matched image feature semantic; calling a lamp effect control instruction corresponding to the lamp effect type to control a lamp effect display unit connected with a current embedded unit to play a corresponding lamp effect. The application can determine a corresponding lamp effect type according to the semantic of the image feature of the interface image based on the embedded unit, thereby controlling the lamp effect display unit to play a corresponding semantic, making the atmosphere rendered by the lamp effect consistent with the picture atmosphere of the interface image, and being controllable in cost and convenient to implement.

Description

Technical Field

[0001] This application relates to the field of lighting technology, and in particular to a lighting effect control method, device, equipment, medium, and product thereof. Background Technology

[0002] As artificial intelligence (AI) matures and deep learning algorithms and NPU chips continue to evolve, its applications are increasingly reaching the consumer market. For example, lighting equipment is beginning to explore various AI technologies to achieve more intelligent functionality. Lighting equipment serves multiple purposes, including displaying information and creating ambiance, and its applications are wide-ranging. Its intelligence is also increasing, and its functions are constantly evolving to meet diverse needs.

[0003] Some of the latest lighting devices in the industry use cameras to capture colors generated by images on the screen, partitioning the screen image to capture colors and mapping them onto light strips to achieve ambient lighting. The drawback of these devices is that they only consider the colors of the image and not other information such as the image's content and style. This makes it difficult for the implemented lighting effects to form an effective semantic connection with the image's content and style, often failing to create a realistic atmosphere. For example, when the image presents a sad scene, because it contains red areas, current technology might map the red to warm-toned lighting effects such as pink or orange, causing the actual atmosphere to be out of sync with the image's atmosphere, achieving the opposite effect.

[0004] Another technical obstacle hindering the generation of lighting effects and the content and style of images is the limitation of computing power, cost, and storage space of embedded chips such as NPUs. The various computing resources of such chips are relatively limited, and if related intelligent technologies are over-introduced, it will lead to a sharp increase in implementation costs, thereby raising the threshold for improvement.

[0005] Therefore, it is evident that, faced with multiple challenges such as hardware constraints and the need for intelligentization, how to achieve technological upgrades in lighting control equipment is a problem that those skilled in the art need to overcome. Summary of the Invention

[0006] The purpose of this application is to solve the above-mentioned problems by providing a lighting effect control method and corresponding devices, equipment, non-volatile readable storage media, and computer program products.

[0007] According to one aspect of this application, a lighting effect control method is provided, comprising the following steps:

[0008] Get the image displayed on the interface;

[0009] The lighting effect type that matches the semantic meaning of the image features to be matched in the interface image is retrieved from the image lighting effect library;

[0010] The corresponding lighting effect control command for the lighting effect type is invoked to control the lighting effect display unit connected to the currently embedded unit to play the corresponding lighting effect.

[0011] Optionally, the lighting effect type mapped to a pre-stored image feature that matches the semantic meaning of the interface image to be matched can be retrieved from the image lighting effect library, including:

[0012] The image features of the interface image are extracted using an image coding model as the image features to be matched.

[0013] Calculate the semantic similarity between the image features to be matched and each pre-stored image feature in the image lighting effect library;

[0014] Determine whether the semantic similarity of the pre-stored image feature corresponding to the highest semantic similarity exceeds a preset similarity threshold. If it exceeds the similarity threshold, select the lighting effect type that maps to the pre-stored image feature with the highest semantic similarity.

[0015] Optionally, the semantic similarity between the image features to be matched and each pre-stored image feature in the image lighting effect library is calculated, including:

[0016] The feature matrix formed by the vector representations of each pre-stored image feature in the image lighting effect library is converted into its transpose matrix.

[0017] The matrix product between the vector representation of the image feature to be matched in the interface image and the transpose matrix is ​​used as the semantic similarity of the corresponding pre-stored image feature.

[0018] The semantic similarity of each pre-stored image feature is normalized to a unified numerical range.

[0019] Optionally, obtaining the interface image before displaying it includes:

[0020] Acquire a sample image and set the lighting effect type of the sample image;

[0021] Extract the image features of the sample image, and store the image features as pre-stored image features mapped to their corresponding lighting effect types in the image lighting effect library;

[0022] Initialize and set the similarity threshold for each pre-stored image feature in the image lighting effect library.

[0023] Optionally, in the step of initializing and setting the similarity threshold of each pre-stored image feature in the image lighting effect library,

[0024] The similarity threshold is determined in a personalized manner based on the source image of the pre-stored image features, or...

[0025] The similarity threshold corresponds to the lighting effect type mapped to the source image of the pre-stored image features, which is then personalized.

[0026] The similarity threshold is determined individually based on the source type of the interface image.

[0027] Optionally, before determining whether the semantic similarity of the pre-stored image feature corresponding to the highest semantic similarity exceeds a preset similarity threshold, and before selecting the lighting effect type mapped to the pre-stored image feature with the highest semantic similarity when the similarity threshold is exceeded, the following steps are taken:

[0028] Identify the source type of the source interface of the interface image, and call a preset similarity threshold corresponding to the source type to determine whether the semantic similarity of the pre-stored image features corresponding to the highest semantic similarity exceeds the preset similarity threshold.

[0029] Optionally, before calling the lighting effect control command corresponding to the lighting effect type to control the lighting effect display unit connected to the currently embedded unit to play the corresponding lighting effect, the following steps are included:

[0030] Determine whether the lighting effect types corresponding to multiple consecutive interface images are consistent. If they are consistent, execute the subsequent steps to switch the lighting effect; otherwise, skip the subsequent steps to maintain the original lighting effect.

[0031] or,

[0032] Determine whether the semantic similarity of the features of the images to be matched in multiple consecutive interface images is lower than a preset inter-frame threshold. If they are all lower than the inter-frame threshold, execute the subsequent steps to switch the lighting effect; otherwise, skip the subsequent steps to maintain the original lighting effect.

[0033] According to another aspect of this application, a lighting effect control device is provided, comprising:

[0034] The image acquisition module is configured to acquire images displayed on the interface.

[0035] The lighting effect matching module is configured to query the image lighting effect library to find the lighting effect type that matches the semantic features of the interface image to be matched and maps it to a pre-stored image feature.

[0036] The lighting effect playback module is configured to call the lighting effect control command corresponding to the lighting effect type to control the lighting effect display unit connected to the currently embedded unit to play the corresponding lighting effect.

[0037] According to another aspect of this application, a lighting effect control device is provided, including an embedded unit and a sound pickup unit and a lighting effect display unit connected to the embedded unit. The embedded unit includes a central processing unit and a memory. The central processing unit is used to call and run a computer program stored in the memory to execute the steps of the lighting effect control method described in this application.

[0038] According to another aspect of this application, a non-volatile readable storage medium is provided, which stores a computer program implemented according to the lighting effect control method in the form of computer-readable instructions, wherein the computer program is invoked by a computer to execute the steps included in the method.

[0039] According to another aspect of this application, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the method described in any embodiment of this application.

[0040] Compared to existing technologies, this application realizes lighting effect control within an embedded unit. By representing multiple pre-stored image features and their corresponding lighting effect types in an image lighting effect library, when it is necessary to determine the lighting effect type corresponding to an interface image, the image features of the interface image are used as the image features to be matched. Based on semantic matching, the lighting effect type of the pre-stored image features that are semantically similar to the image features to be matched is retrieved from the image lighting effect library. Then, the lighting effect display unit is controlled to play the corresponding lighting effect according to this lighting effect type. This enables the lighting effect implemented by the lighting effect display unit to establish a semantic association between the pre-stored image features and the image features to be matched, thereby enabling the rendered lighting effect atmosphere to semantically correspond to the atmosphere displayed by the interface image. This intelligently improves the consistency of the lighting effect with the content and style of the interface image. At the same time, it also balances the hardware conditions and the level of intelligence of the embedded unit, achieving economies of scale. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a schematic diagram of the principle of a lighting effect control device according to this application;

[0043] Figure 2 This is a flowchart illustrating one embodiment of the lighting effect control method of this application;

[0044] Figure 3This is a flowchart illustrating the process of determining the lighting effect type corresponding to the interface image in an embodiment of this application.

[0045] Figure 4 This is a flowchart illustrating the process of calculating semantic similarity in an embodiment of this application;

[0046] Figure 5 This is a schematic diagram illustrating the process of constructing the image lighting effect library in an embodiment of this application;

[0047] Figure 6 This is a schematic block diagram of the lighting effect control device of this application;

[0048] Figure 7 This is a schematic diagram of the structure of a computer device that can be used as a lighting effect control device, as used in this application. Detailed Implementation

[0049] Please see Figure 1 In the schematic diagram of a lighting effect control device provided by this application, the lighting effect control device includes an embedded unit, a camera unit, and a lighting effect display unit, wherein:

[0050] The embedded unit can be implemented using various embedded chips, such as Bluetooth SoC, WiFi SoC, MCU, DSP, NPU, etc. The chip typically includes a central processing unit and a memory, mainly used to store and execute program instructions to achieve corresponding functions, such as running a computer program product implemented according to the lighting effect control method of this application, or running an image feature extraction model that may be used in this application, as well as other related control logic, etc.

[0051] The camera unit refers to a camera device suitable for capturing environmental images. This camera unit is an optional accessory for the lighting effect control device of this application. Besides using the camera unit to acquire interface images displayed by other smart devices, the lighting effect control device of this application can also acquire the interface images displayed by the smart devices through various other communication protocols, such as wired connections provided by the HDMI protocol or wireless transmission methods such as WiFi.

[0052] The lighting effect display unit can be single or multiple, and is used to execute lighting effect control commands to play corresponding lighting effects, achieving effects such as information display and ambiance decoration. The lighting effects can be preset by those skilled in the art, and their corresponding lighting effect control commands can be compiled and pre-stored in the embedded unit and / or lighting effect display unit, and can be invoked and executed to play the corresponding lighting effects when needed.

[0053] When the lighting effect control device is running, the embedded unit can acquire an interface image from any source, then obtain its image features based on the interface image, perform semantic matching between the image features and pre-stored image features in the image lighting effect library to determine the corresponding lighting effect type, and then use the lighting effect control command corresponding to the lighting effect type to control one or more of the lighting effect display units to play the corresponding lighting effect.

[0054] Based on the above explanation of principles, please refer to Figure 2 According to a lighting effect control method provided in this application, in one embodiment, the method includes the following steps:

[0055] Step S1100: Obtain the displayed interface image;

[0056] The interface images refer to images displayed in the graphical user interface of any smart device. These images can be images output during video streaming, such as image frames output from a live video stream in a web streaming program, or image frames output from other media streams; they can also be content images generated by applications and displayed in the graphical user interface. These applications typically provide content images in a way that features changing visuals, such as game programs providing content images corresponding to constantly changing game visuals; they can also be other static images presented in the graphical user interface, such as dynamic or static screensaver images.

[0057] Since the method of this application is mainly used in lighting effect control devices, when acquiring the interface image, it is usually the interface image displayed in the smart device that is acquired. Other images that are from the same source as the interface image but are not displayed on the graphical user interface are generally only processed when they are displayed on the graphical user interface.

[0058] There are several ways to obtain the image displayed on the interface, such as:

[0059] In one embodiment, the lighting effect control device of this application can be connected to the smart device for displaying the interface image via a wired connection using the interfaces provided by various wired display interface protocols, such as HDMI, DVI, DP, VGA, etc., so that the embedded unit of the lighting effect control device of this application can obtain the interface image displayed on the display device of the smart device via a wired connection.

[0060] In another embodiment, based on various wireless communication protocols, such as WiFi and Bluetooth, the interface image displayed on the smart device can be wirelessly transmitted to the embedded unit of the lighting effect control device of this application, provided that both the smart device and the embedded unit of the lighting effect control device of this application comply with the same media data exchange protocol.

[0061] In another embodiment, a camera unit can be configured in the lighting effect control device of this application. The camera unit captures interface images on the display device of the smart device in real time, and then provides them to the embedded unit in the lighting effect control device for processing. For the interface images recorded in real time by the camera unit, each image frame can be retrieved from the corresponding image space and then subjected to corresponding image optimization processing. For example, image correction, smoothing, cropping, scaling, etc., can be performed as needed to adjust them into images of a specific resolution suitable for subsequent processing.

[0062] Step S1200: Query the image lighting effect library to find the lighting effect type that matches the semantics of the image features to be matched of the interface image and maps to the pre-stored image features.

[0063] This application provides an image lighting effect library, which stores the image features of various preset images as pre-stored image features. For each pre-stored image feature, a lighting effect type corresponding to the style is set according to the content and style of its source image.

[0064] The lighting effect types are pre-designed, and their corresponding lighting effect control instructions are compiled and stored in the memory of the embedded unit or other memory accessible by the embedded unit. This allows the embedded unit to invoke the corresponding lighting effect control instructions based on the lighting effect instructions. There are multiple lighting effect types, each corresponding to a different atmosphere style. For example, atmosphere styles can be set according to emotional types, identifying atmosphere styles corresponding to emotions such as sadness, peace, joy, excitement, thrill, and depression, and assigning a corresponding lighting effect type to each atmosphere style. Therefore, based on the style of the source image, the corresponding pre-stored image features can be set as the lighting effect type of the appropriate style.

[0065] The mapping relationship between lighting effect type and pre-stored image features of source images can be one-to-one or one-to-many. For example, multiple pre-stored image features corresponding to multiple source images can be labeled as belonging to the same lighting effect type.

[0066] When it is necessary to match the corresponding lighting effect type for the interface image, similarly, it is necessary to first use an image encoding model to extract the image features of the interface image as the image features to be matched, so that semantic matching operations can be performed between the interface image and each source image in the image lighting effect library based on the image features.

[0067] When performing semantic matching, the semantic similarity between the image features to be matched of the interface image and each pre-stored image feature in the image lighting effect library can be calculated first. Then, a pre-stored image feature is selected based on the semantic similarity as the target image feature to achieve semantic matching with the image features to be matched. This indicates that the interface image and the source image of the target image feature are similar in content and style. Then, the target lighting effect type mapped to this target image feature is determined as the lighting effect type corresponding to the interface image.

[0068] In one embodiment, when selecting and determining the target image features, the pre-stored image features corresponding to the highest semantic similarity can be selected as the target image features to pursue the most similar result. In another embodiment, to prevent individual image features from being incorrect due to overfitting of the corresponding neural network model, the pre-stored image features corresponding to the second highest semantic similarity can also be selected as the target image features.

[0069] In one embodiment, the image features to be matched in the interface image and the pre-stored image features of each source image can both be extracted using the same image encoding model. The image encoding model can be selected from basic models suitable for performing convolutional operations on images to extract their deep semantic features, such as Convolutional Neural Networks (CNN), Residual Networks (ResNet), and VGG. The image encoding model can be a pre-trained model.

[0070] In another embodiment, the image coding model is pre-trained to convergence before being put into use. During training, a classifier is attached to the image coding model, and a large number of training samples and their supervision labels are used. The training samples can be various sample images representing different atmosphere styles, and the corresponding supervision labels are label information corresponding to the atmosphere style. During training, a training sample is input each time. The image coding model performs feature representation on the training sample, predicts its corresponding atmosphere style, and then calculates the classification loss value of the predicted atmosphere style using the supervision label corresponding to the training sample. The image coding model is then updated using gradients based on the classification loss value. When the classification loss value reaches a preset target threshold, it indicates that the image coding model has converged, and training can be terminated, the attached classifier removed, and the model used to extract the image features.

[0071] Since the image coding model is trained under supervised supervision according to the atmosphere style, the image features obtained by the image coding model in representing the image can represent the deep semantics corresponding to the atmosphere style. When performing image matching based on deep semantics of the same nature, the accurate correspondence of the indirectly determined lighting effect type can be ensured. It can also be seen that the atmosphere style represented by the supervised labels provided during the training of the image coding model is actually the atmosphere style adopted by the lighting effect type of this application.

[0072] In this application, the operation of performing similarity matching based on the image features of the interface image and the source image to indirectly determine the lighting effect type does not directly use a classifier to predict the lighting effect type corresponding to the image features of the interface image. This can take into account the relatively limited computing resources of the embedding unit. That is, the pre-stored image features in the image lighting effect library can theoretically be obtained by running the image encoding model outside the embedding unit in advance. As for the various operations when matching the interface image with the pre-stored image features of each source image, their computational load is relatively low, and therefore they are more suitable for deployment in the embedding unit. It can be seen that this arrangement has its advantages, can achieve better computing efficiency, and helps to control the overall implementation cost of the entire lighting effect control device.

[0073] Step S1300: Call the lighting effect control command corresponding to the lighting effect type to control the lighting effect display unit connected to the current embedded unit to play the corresponding lighting effect.

[0074] The embedded unit pre-stores mapping data between lighting effect types and their corresponding lighting effect control commands. Alternatively, this mapping data can be stored in the image lighting effect library. Therefore, once the lighting effect type corresponding to the interface image is determined, the lighting effect control command mapped to that type can be obtained from the mapping data. This command is then transmitted to the lighting effect display unit connected to the embedded unit, thereby controlling the display unit to play the corresponding lighting effect.

[0075] In one embodiment, multiple lighting effect display units can cooperate to display the same lighting effect content. Accordingly, when the embedded unit transmits the lighting effect control command to the multiple lighting effect display units, each lighting effect display unit plays its respective lighting effect display process according to the lighting effect control command, and all lighting effect display units jointly display the lighting effect content.

[0076] In another embodiment, after a lighting effect control command of a certain type, such as classical lighting effect, is transmitted to the lighting effect display unit, the lighting effect display unit controls the lighting effect to, for example, flash according to a predefined protocol. If the subsequently determined lighting effect type does not change, there is no need to transmit the same lighting effect control command to the lighting effect display unit. Once a different lighting effect type is subsequently generated, for example, a new lighting effect type of rock lighting effect, the corresponding lighting effect control command is transmitted, and the lighting effect display unit is controlled to scroll the lighting effect content according to a predefined protocol. In some embodiments, during the switching display of two different lighting effect types, a smoothing process can be performed, allowing the brightness and color values ​​of the light-emitting elements to change gradually to achieve a natural transition.

[0077] As can be seen from the above embodiments, this application realizes the implementation of lighting effect control in the embedded unit. By representing the mapping relationship between multiple pre-stored image features and corresponding lighting effect types in the image lighting effect library, when it is necessary to determine the lighting effect type corresponding to the interface image, the image features of the interface image are used as the image features to be matched. Based on semantic matching, the lighting effect type of the pre-stored image features that are semantically similar to the image features to be matched is queried from the image lighting effect library. Then, the lighting effect display unit is controlled to play the corresponding lighting effect according to this lighting effect type, so that the lighting effect implemented by the lighting effect display unit can establish an image semantic association with the pre-stored image features and the image features to be matched. In this way, the rendered lighting effect atmosphere can correspond semantically with the atmosphere displayed by the interface image, intelligently improving the consistency of the lighting effect with the content, style, etc. of the interface image. At the same time, it also takes into account the balance between the hardware conditions and the degree of intelligence of the embedded unit, and can achieve economies of scale.

[0078] Based on any embodiment of this application, please refer to Figure 3 The lighting effect type mapped to the pre-stored image features that match the semantic meaning of the interface image to be matched is retrieved from the image lighting effect library, including:

[0079] Step S1210: Use an image coding model to extract the image features of the interface image as the image features to be matched;

[0080] In this embodiment, the code information and weight parameters of the image coding model trained in advance can be installed into the embedding unit, and the corresponding image coding service can be enabled. When the interface image is passed into the embedding unit, the image coding service running in the embedding unit calls the image coding model to perform feature representation on the interface image.

[0081] After the interface image is processed successively through convolutional layers, pooling layers, and batch processing layers in the image coding model, corresponding image features, i.e., image features to be matched, are obtained. To facilitate efficient subsequent computation, in one embodiment, the extracted image features can be unfolded into a high-dimensional vector in the image coding model. Similarly, each pre-stored image feature in this application can also be represented as a high-dimensional vector.

[0082] In one embodiment, the image encoding model required for extracting image features employs a VGG16 convolutional neural network. The VGG16 model consists of 5 convolutional layers, 3 fully connected layers, and a softmax output layer. Max pooling is used to separate the layers, and the activation units of all hidden layers use the ReLU function. During online inference, only the convolutional and pooling layers of this model are used to construct the image encoding model, pruning the fully connected and softmax layers used during model training. This image encoding model can then be used to represent the features of the input image and obtain the corresponding image features. Of course, this is just one example of image encoding model selection; those skilled in the art can choose other basic models as needed.

[0083] Step S1220: Calculate the semantic similarity between the image features to be matched and each pre-stored image feature in the image lighting effect library;

[0084] To determine the similarity between the image feature to be matched and each pre-stored image feature in the image lighting effect library, which is essentially a procedure to determine the similarity in image content and style between the interface image and each source image in the image lighting effect library, any data distance algorithm can be used to calculate the data distance between the image feature to be matched and each pre-stored image feature, and each data distance is converted into a representation of semantic similarity, so that the higher the semantic similarity, the more similar the two image features are in image content and style.

[0085] In this application, the data distance algorithm can be implemented using any of the following algorithms: matrix inner product, cosine similarity, Pearson correlation coefficient, Euclidean distance, Jaccard distance, etc. Various data distance algorithms can effectively represent the semantic similarity between two image features; therefore, their calculation results can be used as a measure of semantic similarity.

[0086] In one embodiment, cosine similarity is preferred for similarity calculation due to its simplicity and effectiveness. The cosine similarity algorithm measures the similarity between two vectors by measuring the cosine of the angle between them. The cosine of a 0-degree angle is 1, and the cosine of any other angle is no greater than 1; its minimum value is -1. Therefore, the cosine of the angle between two vectors determines whether they point approximately in the same direction. When two vectors point in the same direction, the cosine similarity is 1; when the angle between them is 90°, the cosine similarity is 0; and when they point in completely opposite directions, the cosine similarity is -1. This result is independent of the vector length and only depends on the direction in which the vectors point. Cosine similarity is typically used in positive space, so the given value is between -1 and 1. Negative cosine similarity is not considered, and the range is 0-1, which can be converted to a percentage of 0%-100% to represent image similarity.

[0087] Furthermore, the cosine similarity calculation formula can be optimized to improve calculation speed by calculating the numerator and denominator separately:

[0088] (1) The numerator x1*x2+y1*y2 is calculated using the NPU matrix multiplication method. Only one matrix multiplication is needed to calculate the product of the image features of the input interface image and each image feature in the image lighting effect library, which greatly improves the calculation speed.

[0089] (2) The denominator is calculated separately for the length of the feature vector in the rectangular coordinate system. It only needs to be calculated once during feature extraction, thus saving computing power.

[0090] (3) After finding the numerator and denominator, the cosine distance can be calculated according to the cosine distance formula. Assuming that the image library has 1,000 images, this calculation method can be 1,000 times faster than the traditional calculation method.

[0091] Step S1230: Determine whether the semantic similarity of the pre-stored image feature corresponding to the highest semantic similarity exceeds a preset similarity threshold. When it exceeds the similarity threshold, select the lighting effect type that maps to the pre-stored image feature with the highest semantic similarity.

[0092] Considering that the lighting effect display unit can only play one lighting effect at a time, in this embodiment, the target lighting effect type corresponding to the interface image is determined only based on the highest semantic similarity. To this end, the pre-stored image features in the image lighting effect library are first sorted in reverse order according to semantic similarity, thus determining the first pre-stored image feature in the sort as the one with the highest semantic similarity. Then, a preset similarity threshold is used to compare this highest semantic similarity with the highest semantic similarity. When the highest semantic similarity exceeds the similarity threshold, it indicates that the source image corresponding to the highest semantic similarity is highly similar to the interface image in content and style, and the two match. Therefore, the lighting effect type mapped to the pre-stored image feature corresponding to the highest semantic similarity can be determined as the target lighting effect type matching the interface image.

[0093] In one embodiment, when the highest semantic similarity does not exceed the preset threshold, the interface image can be color sampled to determine its corresponding average color gamut, and then its corresponding target lighting effect type can be determined based on the average color gamut. Thus, even if the current interface image fails to determine its corresponding lighting effect type through semantic matching, technical relief can be achieved based on the average color gamut of the interface image, and the corresponding target lighting effect type can be determined to call the corresponding lighting effect control command to implement the control of the ambient lighting effect.

[0094] As can be seen from the above embodiments, based on semantic matching, pre-stored image features that match the interface image in terms of content and style can be accurately determined. In fact, the source image corresponding to the pre-stored image features is determined, and then the lighting effect type mapped to the pre-stored image features is determined in order to obtain the corresponding lighting effect control instructions. The lighting effect type is precisely set according to the style of the source image. Therefore, the interface image and the matched lighting effect type can ensure accurate correspondence in terms of atmosphere and style, so that the resulting lighting effect atmosphere can maintain a high degree of coordination and consistency with the atmosphere presented by the interface image.

[0095] Based on any embodiment of this application, please refer to Figure 4 Calculate the semantic similarity between the image features to be matched and each pre-stored image feature in the image lighting effect library, including:

[0096] Step S1221: Convert the feature matrix formed by the vector representations of each pre-stored image feature in the image lighting effect library into its transpose matrix;

[0097] In this embodiment, each pre-stored image feature in the image lighting effect library is first represented as a high-dimensional vector, so that each pre-stored image feature is actually a high-dimensional image feature vector. Thus, when there are M pre-stored image features, and each pre-stored image feature has K-dimensional values, then each pre-stored image feature can be represented as an M*K feature matrix.

[0098] To perform similarity operations based on matrix operations, the feature matrix can be transposed to become a K*M transpose matrix.

[0099] Step S1222: Calculate the matrix product between the vector representation of the image feature to be matched of the interface image and the transpose matrix as the semantic similarity of the corresponding pre-stored image feature;

[0100] Similarly, the vector representation of the image feature to be matched in the interface image is a single-row vector, i.e., a 1*K vector. In this embodiment, considering the computational efficiency of the embedding unit, the semantic similarity between the image feature to be matched and each pre-stored image feature is calculated based on the matrix inner product method. The matrix inner product algorithm is essentially a simplified algorithm of the cosine similarity algorithm and achieves the same effect. Therefore, based on the principle of matrix operation, the 1*K image feature to be matched is multiplied by the K*M transpose matrix to obtain a 1*M product matrix. In this product matrix, the values ​​of the M elements represent the semantic similarity values ​​corresponding to the M pre-stored image features in the image lighting effect library.

[0101] Step S1223: Normalize the semantic similarity of each pre-stored image feature to a unified numerical range.

[0102] To make the results more intuitive, the product matrix can be further normalized to match the semantic similarity values ​​to a preset range, such as [0,1]. When performing normalization, maximum value normalization can be used, which involves dividing the semantic similarity of each pre-stored image feature by the highest semantic similarity value to obtain the final semantic similarity value.

[0103] As can be seen from the above embodiments, in the embedding unit, matrix multiplication is performed on the features of the image to be matched and each pre-stored image feature in the image lighting effect library to determine the semantic similarity corresponding to each pre-stored image feature, based on matrix operations. Instead of directly performing a large amount of computational classification mapping operation on the features of the image to be matched in the embedding unit, this processing method has lower system overhead, higher computational efficiency, and is more suitable for the specific scenario device of the embedding unit.

[0104] Based on any embodiment of this application, please refer to Figure 5Before obtaining the displayed interface image, the following are included:

[0105] Step S2100: Obtain a sample image and set the lighting effect type of the sample image;

[0106] The image lighting effect library can be pre-constructed. To do this, a batch of sample images can be prepared, each with different content and style, to define different atmosphere styles. After preparing the sample images, the lighting effect types for each sample image are further obtained through manual annotation. These lighting effect types belong to a member type of a type set. Generally, each sample image can only belong to one lighting effect type, while one lighting effect type can correspond to multiple sample images.

[0107] Step S2200: Extract the image features of the sample image, and store the image features as pre-stored image features and their corresponding lighting effect types in the image lighting effect library;

[0108] Furthermore, the image coding model described above can be run on any computer device to extract the corresponding image features for each prepared sample image. These image features can also be converted into corresponding vector representations. Then, the image features of each sample image are used as pre-stored image features, and a mapping relationship is constructed between them and their corresponding lighting effect types, storing this mapping relationship in an image lighting effect library. Ultimately, this image lighting effect library will be migrated to the embedded unit of the lighting effect control device of this application for storage.

[0109] In one embodiment, the image lighting effect library can convert all pre-stored image features into a feature matrix representation, where each row is a high-dimensional vector representation of a pre-stored image feature. Alternatively, the feature matrix can be further converted into its transpose for storage. As for the index item of each pre-stored image feature, a mapping relationship can be established with its corresponding lighting effect type to achieve the association mapping between each pre-stored image feature and its corresponding lighting effect type. Similarly, according to actual needs, a corresponding mapping relationship can also be established between the lighting effect type and its corresponding lighting effect control command, and stored together in the image lighting effect library.

[0110] Step S2300: Initialize and set the similarity threshold of each pre-stored image feature in the image lighting effect library.

[0111] In the subsequent step of determining whether the pre-stored image features with the highest semantic similarity are suitable for determining the lighting effect type of the interface image, a preset similarity threshold is used to judge whether the highest semantic similarity is valid. In this case, the similarity threshold can be initialized here. There are several different implementations of the initialization method, for example:

[0112] In one embodiment, the similarity threshold is personalized based on the source image of the pre-stored image feature. Specifically, the source images of each pre-stored image feature differ in content and style, and the stability of the displayed images also varies. Therefore, the similarity threshold corresponding to each source image can also be different, thereby achieving personalized definition. For example, source images with thunder and lightning scenes will have relatively unstable image performance, while source images with sunny scenes will have relatively stable image quality. Therefore, different similarity thresholds can be assigned to different situations, so that the pre-stored image features correspond one-to-one with the similarity thresholds. When a subsequent pre-stored image feature obtains the highest semantic similarity, the similarity threshold corresponding to that pre-stored image feature is called to determine the highest semantic similarity.

[0113] In another embodiment, the similarity threshold is individually determined according to the lighting effect type mapped to the source image of the pre-stored image feature. Specifically, the similarity threshold can be different for each lighting effect type. For example, for a cheerful lighting effect type, since its possible scenarios are relatively rich, the similarity threshold for this lighting effect type can be set to a relatively low value. Conversely, for a thrilling lighting effect type, if its possible scenarios are relatively few, its similarity threshold can be set to a relatively high value. Thus, by setting the similarity threshold according to the lighting effect type, when a pre-stored image feature needs to be compared to obtain the highest semantic similarity, the lighting effect type to which the pre-stored image feature belongs is first determined, and then the corresponding similarity threshold is called according to the lighting effect type for comparison.

[0114] In another embodiment, the similarity threshold is individually determined based on the source type of the interface image. Specifically, considering that the image quality may vary depending on the source of the interface image, for example, when acquiring an interface image transmitted from a smart device via wired or wireless means, the image quality is usually more stable. In this case, a relatively high similarity threshold can be set for interface images from this source type. When using a camera unit to capture an interface image displayed on the smart device's display device, considering factors such as image distortion, pixel grid, and reflection caused by the capture, the image quality of the obtained interface image may be relatively average. In this case, a relatively low similarity threshold can be set for this source type. Therefore, when a pre-stored image feature needs to be compared using a similarity threshold to obtain the highest semantic similarity, the source type of the corresponding interface image is first determined, and then the corresponding similarity threshold is called for comparison based on that source type.

[0115] In other embodiments, one or two of the above embodiments for setting similarity thresholds can be combined. For example, a basic threshold can be set based on each lighting effect type, and a threshold ratio can be set based on the source type of the interface image. When it is necessary to obtain a similarity threshold, the basic threshold of the lighting effect type corresponding to the highest semantic similarity is first determined, and then the threshold ratio corresponding to the source type of the interface image is determined. The product of the basic threshold and the threshold ratio is used to determine the corresponding similarity threshold, which is used to make a judgment on the highest semantic similarity. Such modifications can be made based on the above embodiments, without departing from the scope of the inventive spirit of this application.

[0116] As can be seen from the above embodiments, in the process of constructing the image lighting effect library of this application, the pre-stored image features are first determined based on the sample image, and a mapping relationship is established between the pre-stored image features and the lighting effect type labeled on the sample image. Then, the threshold used to determine the highest semantic similarity is initialized. After the image lighting effect library is migrated to the embedding unit, the embedding unit can perform efficient calculations based on the image lighting effect library, thereby quickly matching the lighting effect type corresponding to the interface image. Since the similarity thresholds of each pre-stored image feature have been initialized in advance, when the image lighting effect library is deployed in the lighting effect control device, the standardization of the lighting effect control device product can be realized, the accuracy of playing the lighting effect corresponding to the atmosphere based on the interface image can be ensured, and the economies of scale corresponding to the large-scale production of the lighting effect control device can be obtained.

[0117] Based on any embodiment of this application, before determining whether the semantic similarity of the pre-stored image feature corresponding to the highest semantic similarity exceeds a preset similarity threshold, and before selecting the lighting effect type mapped to the pre-stored image feature with the highest semantic similarity when the similarity threshold is exceeded, the process includes:

[0118] Identify the source type of the source interface of the interface image, and call a preset similarity threshold corresponding to the source type to determine whether the semantic similarity of the pre-stored image features corresponding to the highest semantic similarity exceeds the preset similarity threshold.

[0119] In this embodiment, before determining whether the highest semantic similarity exceeds a preset similarity threshold, the embedded device performs an identification step. Specifically, the source type of the source interface can be determined by the source interface through which the interface image is transmitted. For example, the interface image acquired by the camera unit and the interface image acquired via HDMI typically use different function interfaces. In this case, identifying this function interface can determine the corresponding source type. After determining the source type, the preset similarity threshold corresponding to that source type can be called. This similarity threshold is used to determine whether the highest semantic similarity is valid. When it is valid, the lighting effect type mapped to its corresponding pre-stored image features is called as the target lighting effect type corresponding to the interface image.

[0120] As can be seen from the above embodiments, interface images transmitted from different source interfaces typically have different levels of image quality. Before implementing the highest semantic similarity judgment, the source interface is first identified to determine the corresponding source type, and then the similarity threshold is determined accordingly. This allows for personalized adjustment of the highest semantic similarity filtering conditions based on different source interfaces, thereby making the determination of the lighting effect type related to the image quality of the interface image and improving the intelligence level of the lighting effect control device of this application.

[0121] Based on any embodiment of this application, before calling the lighting effect control instruction corresponding to the lighting effect type to control the lighting effect display unit connected to the current embedded unit to play the corresponding lighting effect, in one embodiment, it includes: determining whether the lighting effect types corresponding to multiple consecutive interface images are consistent; if they are consistent, executing subsequent steps to implement lighting effect switching; otherwise, skipping subsequent steps to maintain the original lighting effect.

[0122] In this application, steps S1100 and S1200 can be run concurrently with step S1300. Before executing step S1300, the determined lighting effect types corresponding to multiple consecutive interface images can be obtained to examine the stability of the content of multiple consecutive interface images.

[0123] Specifically, when multiple consecutive interface images correspond to the same lighting effect type, it usually indicates that these interface images belong to the same content. In this case, the generated lighting effect type is relatively stable. Therefore, step S1300 can be executed to use the lighting effect type as the target lighting effect type to call the corresponding lighting effect control instruction to control the lighting effect display unit. Otherwise, step S1300 does not need to be executed. Instead, the detection continues until the above conditions are met before step S1300 is executed.

[0124] In another embodiment, before calling the lighting effect control instruction corresponding to the lighting effect type to control the lighting effect display unit connected to the current embedded unit to play the corresponding lighting effect, the method includes: determining whether the semantic similarity of the image features to be matched of multiple consecutive interface images is lower than a preset inter-frame threshold. If they are all lower than the inter-frame threshold, the subsequent steps are executed to implement the lighting effect switching; otherwise, the subsequent steps are skipped to maintain the original lighting effect.

[0125] Unlike the previous embodiment, in this embodiment, the image features of the consecutive multiple interface images, i.e., the image features to be matched, are directly used to calculate the inter-frame similarity. That is, for every two consecutive interface images, the semantic similarity between their image features is calculated. The specific algorithm can be the same as described above in this application. Then, each semantic similarity is compared with a preset inter-frame threshold. The inter-frame threshold can be preset as needed. When the semantic similarity of each of the consecutive multiple interface images is lower than the inter-frame threshold, it indicates that the content of each interface image has changed significantly. At this time, the lighting effect type corresponding to the last interface image should be valid. Only then is the lighting effect type corresponding to the last interface image the target lighting effect type, and step S1300 is continued to be executed, calling the corresponding lighting effect control instruction to control the lighting effect display unit. Otherwise, step S1300 is not required, but the detection continues until the above conditions are met before step S1300 is executed.

[0126] In the two embodiments above, the number of consecutive interface images is typically measured in image frames, with the total duration of all image frames preferably not exceeding 120 milliseconds. For example, when the frame rate is 25fps, each image frame corresponds to 40 milliseconds. In this case, the lighting effect type can be compared using three consecutive image frames as a unit to determine if they are consistent. This processing serves as a buffer to determine the lighting effect type, making the determined lighting effect type smoother. Changes in individual image frames will not cause a switch in the lighting effect type, and this buffering process is basically controlled within a range that is imperceptible to the human eye.

[0127] As can be seen from the above embodiments, this application can examine the stability of lighting effect types by utilizing the changing relationships of multiple consecutive interface images without being perceived by the human eye. It uses a strategy to maintain stability as much as possible to control the switching of lighting effects, so that the switching of lighting effects does not occur frequently due to short-term changes in individual interface images, thereby ensuring a good user experience.

[0128] As can be seen from the various embodiments disclosed above, this application uses artificial intelligence deep learning algorithms and embedded processors such as NPUs to capture screen display content and obtain interface images through camera units or other channels. It then performs artificial intelligence training and inference to accurately identify the similarity between the screen-displayed image and images in the image lighting effect library. Finally, through corresponding algorithms, it determines the corresponding lighting effect type based on semantic similarity for lighting effect display and atmosphere fusion. For example, when playing the movie Titanic, the camera captures screen content, and combined with the NPU chip and AI model, it performs inference to derive the features of the screen-displayed content. After obtaining the features of the current image, it compares them with the image features in the image lighting effect library to find the image with the highest similarity to the library and outputs a similarity value. For instance, if the image lighting effect library contains the classic image of Jake and Rose "flying" at the bow of the Titanic, when the movie plays this scene, the device will capture the scene and compare it with the image in the image lighting effect library, outputting a similarity result. If the similarity requirement is met, the device will control the lighting equipment to play a predefined Titanic "flying" lighting effect, enhancing the user's viewing experience. As can be seen, this application focuses on the recognition of the entire image. That is, it will only output a result when the content of the image being played on the screen is highly similar to or completely identical to an image in the image lighting effect library. This is significantly different from the existing technology, which only recognizes a specific feature of the screen and then controls the lighting effect, and can greatly improve the user experience.

[0129] Please see Figure 6 According to one aspect of this application, a lighting effect control device includes an image acquisition module 1100, a lighting effect matching module 1200, and a lighting effect playback module 1300. The image acquisition module 1100 is configured to acquire a displayed interface image; the lighting effect matching module 1200 is configured to query a pre-stored image feature mapping that matches the semantic features of the interface image from an image lighting effect library; and the lighting effect playback module 1300 is configured to call a lighting effect control instruction corresponding to the lighting effect type to control a lighting effect display unit connected to the currently embedded unit to play the corresponding lighting effect.

[0130] Based on any embodiment of this application, the lighting effect matching module 1200 includes: a feature extraction unit, configured to extract image features of the interface image as image features to be matched using an image coding model; a similarity calculation unit, configured to calculate the semantic similarity between the image features to be matched and each pre-stored image feature in the image lighting effect library; and a type determination unit, configured to determine whether the semantic similarity of the pre-stored image feature corresponding to the highest semantic similarity exceeds a preset similarity threshold, and when the similarity threshold is exceeded, to select the lighting effect type mapped to the pre-stored image feature with the highest semantic similarity.

[0131] Based on any embodiment of this application, the similarity operation unit includes: a transpose processing unit, configured to convert the feature matrix formed by the vector representations of each pre-stored image feature in the image lighting effect library into its transpose matrix; a similarity calculation subunit, configured to calculate the matrix product between the vector representation of the image feature to be matched of the interface image and the transpose matrix as the semantic similarity of the corresponding pre-stored image feature; and a normalization processing subunit, configured to normalize the semantic similarity of each pre-stored image feature to a unified numerical range.

[0132] Based on any embodiment of this application, the lighting effect control device of this application includes: a sample acquisition module, configured to acquire a sample image and set the lighting effect type of the sample image; a mapping construction module, configured to extract image features of the sample image and map the image features as pre-stored image features to their corresponding lighting effect types and store them in the image lighting effect library; and a threshold setting module, configured to initialize and set the similarity threshold of each pre-stored image feature in the image lighting effect library.

[0133] Based on any embodiment of this application, in the threshold setting module, the similarity threshold is personalized to the source image of the pre-stored image feature, or the similarity threshold is personalized to the lighting effect type mapped to the source image of the pre-stored image feature, or the similarity threshold is personalized to the source type of the interface image.

[0134] Based on any embodiment of this application, the lighting effect control device of this application includes: a threshold calling unit, configured to identify the source type of the source interface of the interface image, and call a preset similarity threshold corresponding to the source type according to the source type, for determining whether the semantic similarity of the pre-stored image feature corresponding to the highest semantic similarity exceeds the preset similarity threshold.

[0135] Based on any embodiment of this application, the lighting effect control device of this application includes: a first buffer module, configured to determine whether the lighting effect types corresponding to multiple consecutive interface images are consistent; if consistent, execute subsequent steps to implement lighting effect switching; otherwise, skip subsequent steps to maintain the original lighting effect; or, includes: a second buffer module, configured to determine whether the semantic similarity of the image features to be matched of multiple consecutive interface images is lower than a preset inter-frame threshold; if lower than the inter-frame threshold, execute subsequent steps to implement lighting effect switching; otherwise, skip subsequent steps to maintain the original lighting effect.

[0136] Another embodiment of this application provides a lighting effect control device, which is implemented using computer equipment. For example... Figure 7The diagram shows the internal structure of the computer device. The computer device includes a processor, a camera unit, a computer-readable storage medium, a memory, and a network interface connected via a system bus. The computer-readable, non-volatile storage medium stores an operating system, a database, and computer-readable instructions. The database may store information sequences. When the computer-readable instructions are executed by the processor, the processor can implement the lighting effect control method of this application.

[0137] The processor of this computer device provides computing and control capabilities to support the operation of the entire computer device. The memory of this computer device can store computer-readable instructions, which, when executed by the processor, cause the processor to perform the lighting effect control method of this application. The network interface of this computer device is used for communication with a terminal.

[0138] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0139] In this embodiment, the processor executes the specific functions of each module implemented according to each step of the lighting effect control method of this application, and the memory stores the program code and various types of data required to execute the above modules. The non-volatile readable storage medium in this embodiment stores the program code and data of the computer program product implemented according to the lighting effect control method of this application.

[0140] This application also provides a non-volatile readable storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the lighting effect control method of any embodiment of this application.

[0141] This application also provides a computer program product, including a computer program / instructions that, when executed by one or more processors, implement the steps of the lighting effect control method described in any embodiment of this application.

[0142] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The aforementioned storage medium can be a computer-readable storage medium such as a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM).

[0143] In summary, this application can determine the corresponding lighting effect type based on the semantics of the image features of the interface image using the embedded unit, thereby controlling the lighting effect display unit to play the corresponding semantics, so that the atmosphere rendered by the lighting effect is consistent with the atmosphere of the interface image, with controllable cost and easy implementation.

Claims

1. A lighting effect control method, characterized in that, An embedded unit for use in a lighting effect control device, the lighting effect control device including the embedded unit and a lighting effect display unit, the method comprising: Acquire the interface image displayed by the external smart device; The lighting effect type mapped to a pre-stored image feature that semantically matches the interface image to be matched is retrieved from the image lighting effect library. The pre-stored image features in the image lighting effect library are obtained by pre-running an image coding model outside the embedding unit. This includes: using the image coding model to extract image features of the interface image as image features to be matched; calculating the semantic similarity between the image features to be matched and each pre-stored image feature in the image lighting effect library based on matrix inner product, and determining the pre-stored image feature with the highest semantic similarity; identifying the source type of the source interface of the interface image, and calling a preset similarity threshold corresponding to the source type; when the highest semantic similarity exceeds the similarity threshold, selecting the lighting effect type mapped to the pre-stored image feature with the highest semantic similarity. The corresponding lighting effect control command for the lighting effect type is invoked to control the lighting effect display unit connected to the currently embedded unit to play the corresponding lighting effect.

2. The lighting effect control method according to claim 1, characterized in that, Calculating the semantic similarity between the image features to be matched and each pre-stored image feature in the image lighting effect library includes: The feature matrix formed by the vector representations of each pre-stored image feature in the image lighting effect library is converted into its transpose matrix. The semantic similarity of each pre-stored image feature is obtained by calculating the matrix product between the vector representation of the image feature to be matched in the interface image and the transpose matrix. The semantic similarity of each pre-stored image feature is normalized to a unified numerical range.

3. The lighting effect control method according to claim 1 or 2, characterized in that, Before invoking the lighting effect control command corresponding to the lighting effect type to control the lighting effect display unit connected to the currently embedded unit to play the corresponding lighting effect, the process includes: Determine whether the lighting effect types corresponding to multiple consecutive interface images are consistent. If they are consistent, execute the subsequent steps to switch the lighting effect; otherwise, skip the subsequent steps to maintain the original lighting effect. or, Determine whether the semantic similarity of the features of the images to be matched in multiple consecutive interface images is lower than a preset inter-frame threshold. If they are all lower than the inter-frame threshold, execute the subsequent steps to switch the lighting effect; otherwise, skip the subsequent steps to maintain the original lighting effect.

4. A lighting effect control device, characterized in that, The device includes: The image acquisition module is configured to acquire images of the external smart device's displayed interface. The lighting effect matching module is configured to query the image lighting effect library to find the lighting effect type that matches the semantics of the image features to be matched in the interface image. The pre-stored image features in the image lighting effect library are obtained by running an image coding model in advance outside the embedding unit. The lighting effect playback module is configured to call the lighting effect control command corresponding to the lighting effect type to control the lighting effect display unit connected to the current embedded unit to play the corresponding lighting effect; The lighting effect matching module includes: The feature extraction unit is configured to use an image coding model to extract image features of the interface image as image features to be matched; The similarity operation unit is configured to calculate the semantic similarity between the image feature to be matched and each pre-stored image feature in the image lighting effect library based on the matrix inner product method, and determine the pre-stored image feature with the highest semantic similarity. The threshold invocation unit is configured to identify the source type of the source interface of the interface image and invoke a preset similarity threshold corresponding to the source type according to the source type; The type determination unit is configured to select a lighting effect type that maps to the pre-stored image features with the highest semantic similarity when the highest semantic similarity exceeds the similarity threshold.

5. The lighting effect control device according to claim 4, characterized in that, The similarity operation unit includes: The transpose processing subunit is configured to convert the feature matrix formed by the vector representations of each pre-stored image feature in the image lighting effect library into its transpose matrix. The similarity calculation subunit is configured to calculate the matrix product between the vector representation of the image feature to be matched of the interface image and the transpose matrix as the semantic similarity of the corresponding pre-stored image feature; The normalization processing subunit is configured to normalize the semantic similarity of each pre-stored image feature to a uniform numerical range.

6. The lighting effect control device according to claim 4 or 5, characterized in that, The lighting effect playback module includes: The first buffer module is set to determine whether the lighting effect types corresponding to multiple consecutive interface images are consistent. If they are consistent, the subsequent steps are executed to switch the lighting effect; otherwise, the subsequent steps are skipped to maintain the original lighting effect. Or, including: The second buffer module is configured to determine whether the semantic similarity of the features of the images to be matched in multiple consecutive interface images is lower than a preset inter-frame threshold. If they are all lower than the inter-frame threshold, the subsequent steps are executed to switch the lighting effect; otherwise, the subsequent steps are skipped to maintain the original lighting effect.

7. A lighting effect control device, comprising an embedded unit and a lighting effect display unit connected to the embedded unit, wherein the embedded unit includes a central processing unit and a memory, characterized in that, The central processing unit is used to invoke and run a computer program stored in the memory to perform the steps of the method as described in any one of claims 1 to 3.

8. A non-volatile readable storage medium, characterized in that, It stores, in the form of computer-readable instructions, a computer program implemented according to any one of claims 1 to 3, which, when invoked by a computer, executes the steps included in the corresponding method.

Images