Control method, system, device and storage medium of advertising display screen

Abstract

The present invention relates to the technical field of advertising display screens, and discloses a control method, system, device and storage medium for an advertising display screen. The method comprises acquiring audio data of a customer; performing text extraction based on the audio data to obtain text data; performing keyword matching based on the text data to obtain key audio data; matching a corresponding image acquisition sensor based on the key audio data, and performing posture extraction to obtain a head angle; performing head angle judgment based on the head angle to generate a first brightness adjustment instruction or a second brightness adjustment instruction; sending the first brightness adjustment instruction or the second brightness adjustment instruction to an advertising display screen, so that the advertising display screen adjusts the brightness according to the first brightness adjustment instruction or the second brightness adjustment instruction. The method can reduce unnecessary energy consumption while improving the effectiveness of advertising.

Description

Technical Field

[0001] The present invention relates to the technical field of advertising display screens, and in particular to a control method, system, device and storage medium for an advertising display screen. Background Art

[0002] An advertising display screen is an electronic display device used to display advertising content. It uses digital technology to play images, videos, texts or other multimedia content to achieve the purpose of advertising. Advertising displays can take many forms. Among them, advertising displays use light-emitting diodes (LEDs) as display units and can be indoor displays in shopping malls, office buildings, and subway stations.

[0003] At present, the existing energy-saving methods by adjusting the brightness of advertising display screens mainly focus on the internal energy saving of the display screen, such as: using low-power LED chips, by using low-power LED chips, the energy consumption of the display screen can be reduced; using efficient power management systems, by optimizing power management, energy waste can be reduced. However, these internal energy-saving methods lack an intelligent control method. If the brightness is adjusted too low, it will be detrimental to the viewing experience of passers-by; on the contrary, if the brightness is adjusted too high, energy consumption will increase; if the brightness remains unchanged, the energy utilization efficiency is also unsatisfactory.

[0004] In summary, the existing energy-saving methods for advertising display screens cannot achieve intelligent control, resulting in low energy utilization. Summary of the invention

[0005] The present invention provides a control method, system, device and storage medium for an advertising display screen, which can reduce unnecessary energy consumption while improving the effectiveness of advertisements.

[0006] In a first aspect, in order to solve the above technical problems, the present invention provides a method for controlling an advertising display screen, comprising:

[0007] Get audio data from customers;

[0008] Extracting text based on the audio data to obtain text data;

[0009] According to the text data, keyword matching is performed to obtain key audio data;

[0010] According to the key audio data, the corresponding image acquisition sensor is matched, and posture extraction is performed to obtain the head angle;

[0011] According to the head angle, head angle judgment is performed to generate a first brightness adjustment instruction or a second brightness adjustment instruction;

[0012] The first brightness adjustment instruction or the second brightness adjustment instruction is sent to the advertising display screen, so that the advertising display screen adjusts the brightness according to the first brightness adjustment instruction or the second brightness adjustment instruction.

[0013] Preferably, performing keyword matching according to the text data to obtain key audio data includes:

[0014] Converting the text data into a text vector using a universal sentence encoder;

[0015] Calculate the cosine similarity between the text vector and the existing advertisement keyword vector, and rank them to obtain a key vector; wherein the existing advertisement keyword vector is a vector converted by a sentence encoder of the advertisement-related text, and the key vector is a text vector corresponding to the cosine similarity ranked the highest among the cosine similarities;

[0016] According to the key vector, reversely infer the corresponding highest cosine similarity, and determine whether the highest cosine similarity is greater than a first preset threshold;

[0017] When it is determined that the highest cosine similarity is greater than the first preset threshold, the audio data corresponding to the text data is used as key audio data.

[0018] Preferably, the converting the text data into a text vector using a universal sentence encoder comprises:

[0019] Extract words based on the text data to obtain words in the sentences of the text data;

[0020] According to the words, word embedding processing is performed using the universal sentence encoder to obtain a word vector;

[0021] The word vectors are summed to obtain a sentence vector;

[0022] The sentence vector is divided by the square root of the length of the sentence vector to obtain a text vector.

[0023] Preferably, the cosine similarity calculation formula is:

[0024]

[0025] In the formula, is the text vector, is the existing advertisement keyword vector, express The inner product of is a vector , Model.

[0026] Preferably, the step of matching the corresponding image acquisition sensor according to the key audio data and performing posture extraction to obtain the head angle includes:

[0027] According to the key audio data, matching the audio acquisition sensor that acquires the key audio data to obtain the key audio sensor;

[0028] According to the key audio sensor, the facial image acquisition sensor of the corresponding area is matched to obtain the key video sensor;

[0029] According to the key video sensor, posture extraction is performed to obtain the head angle.

[0030] Preferably, performing posture extraction according to the key video sensor to obtain the head angle includes:

[0031] According to the key video sensor, video acquisition is performed to obtain video data;

[0032] According to the video data, the VIBE model is used to extract posture and obtain the head angle.

[0033] Preferably, the step of determining the head angle according to the head angle and generating the first brightness adjustment instruction or the second brightness adjustment instruction includes:

[0034] Determine whether the head angle is within a second preset threshold, and if so, generate a first brightness adjustment instruction; if not, generate a second brightness adjustment instruction.

[0035] In a second aspect, the present invention provides a control system for an advertising display screen, comprising:

[0036] A data collection module, used to obtain audio data of customers;

[0037] A text extraction module, used to extract text based on the audio data to obtain text data;

[0038] A word matching module, used to perform keyword matching based on the text data to obtain key audio data;

[0039] A posture extraction module, used to match the corresponding image acquisition sensor according to the key audio data, and perform posture extraction to obtain the head angle;

[0040] An angle judgment module, used to judge the head angle according to the head angle, and generate a first brightness adjustment instruction or a second brightness adjustment instruction;

[0041] The instruction sending module is used to send the first brightness adjustment instruction or the second brightness adjustment instruction to the advertising display screen, so that the advertising display screen adjusts the brightness according to the first brightness adjustment instruction or the second brightness adjustment instruction.

[0042] In a third aspect, the present invention further provides an electronic device comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements any one of the above-described methods for controlling an advertising display screen when executing the computer program.

[0043] In a fourth aspect, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned methods for controlling an advertising display screen.

[0044] Compared with the prior art, the present invention has the following beneficial effects: the embodiment of the present invention provides a control method, system, device and storage medium for an advertising display screen. The method includes: obtaining audio data of a customer; performing text extraction based on the audio data to obtain text data; performing keyword matching based on the text data to obtain key audio data; matching the corresponding image acquisition sensor based on the key audio data, and performing posture extraction to obtain the head angle; performing head angle judgment based on the head angle to generate a first brightness adjustment instruction or a second brightness adjustment instruction; sending the first brightness adjustment instruction or the second brightness adjustment instruction to the advertising display screen, so that the advertising display screen adjusts the brightness according to the first brightness adjustment instruction or the second brightness adjustment instruction.

[0045] In the present invention, the method can collect audio data through an audio collection sensor installed indoors, and extract keywords that are repeated in the audio data and related to existing advertising texts. When keywords related to advertising texts in the audio data are extracted, the image collection sensor of the corresponding area can be matched, and the image collection sensor can be used to extract features of the head posture of passers-by in a specific area to determine whether the passers-by are looking at the advertising display screen, and then adjust the brightness of the advertising display screen by determining whether the passers-by are looking at the advertising display screen. The method first collects audio information of passers-by, and when it is determined that passers-by in a certain area are interested in a certain keyword in the advertisement, the head posture information of passers-by in the specific area is collected, and then the brightness of the advertising display screen in the area is adjusted, which can reduce unnecessary energy consumption while improving the effectiveness of advertising. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1is a flow chart of a method for controlling an advertising display screen provided by a first embodiment of the present invention;

[0047] Figure 2 It is a schematic diagram of the structure of a control system of an advertising display screen provided by a second embodiment of the present invention. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] Reference Figure 1 The first embodiment of the present invention provides a method for controlling an advertising display screen, comprising the following steps:

[0050] S11, obtaining audio data of the customer;

[0051] S12, performing text extraction based on the audio data to obtain text data;

[0052] S13, performing keyword matching according to the text data to obtain key audio data;

[0053] S14, matching the corresponding image acquisition sensor according to the key audio data, and performing posture extraction to obtain the head angle;

[0054] S15, performing head angle determination according to the head angle, and generating a first brightness adjustment instruction or a second brightness adjustment instruction;

[0055] S16: Send the first brightness adjustment instruction or the second brightness adjustment instruction to the advertising display screen, so that the advertising display screen adjusts the brightness according to the first brightness adjustment instruction or the second brightness adjustment instruction.

[0056] It is worth noting that an advertising display screen is an electronic display device used to display advertising content. It uses digital technology to play images, videos, texts or other multimedia content to achieve the purpose of advertising. Advertising display screens can take many forms, among which advertising display screens use light-emitting diodes (LEDs) as display units and can be indoor display screens in shopping malls, office buildings, and subway stations.

[0057] In order to facilitate the understanding of the present invention, some preferred embodiments of the present invention are further described below.

[0058] In step S11, it is worth noting that the customers mainly refer to passers-by who can see the indoor advertising display screen, and can be passers-by in shopping malls, office buildings, and subway stations. Specifically, the method for acquiring the audio data can be: using audio acquisition sensors installed in shopping malls, office buildings, and subway stations to collect the audio data of customers in real time. The sensors can be installed near the advertising display screen or at a certain distance to ensure that the conversations of passers-by or the sounds in the surrounding environment can be collected.

[0059] Specifically, an audio sensor is a microphone, which is a device used to capture sound signals and convert them into electrical signals. The types of audio sensors can be: electret microphone, a small microphone; condenser microphone, which uses the change in capacitance between the plates to detect sound, which can provide better frequency response and sound clarity; array microphone, which consists of multiple microphones and can perform beamforming and sound source localization, and is suitable for complex environments that require sound directionality or noise suppression.

[0060] It is worth noting that before the collected audio data is subjected to text extraction, that is, before step S12, the collected original audio data may contain noise and irrelevant information, and needs to be preprocessed by filtering and noise reduction technology to improve the accuracy of subsequent processing. Among them, the audio denoising technology can be: spectral subtraction, which estimates the power spectrum of the noise and subtracts the estimated value of the noise from the power spectrum of the signal to achieve denoising; Wiener filtering, which is a statistical method that attempts to minimize the mean square error between the output signal and the expected signal; wavelet transform, which decomposes the signal into sub-bands of different frequencies and then processes the signal and noise independently in each sub-band to achieve denoising.

[0061] In step S12, text extraction is performed according to the audio data to obtain text data. Specifically, the method for text extraction can be: using speech recognition technology (ASR), which is a technology for converting speech signals into text, and converting audio data into text data through machine learning models such as deep neural networks (DNN) or convolutional neural networks (CNN); using traditional methods based on feature extraction, first extracting a series of features from the audio, and then matching speech to text through statistical models, such as Mel-frequency cepstral coefficients and Gaussian mixture models-hidden Markov models; using various large language models, such as Generative Pre-trained Transformer (GPT) developed by OpenAI. In addition, a suitable deep learning model can be selected to perform feature extraction and pattern recognition on speech signals to achieve high-precision speech-to-text conversion.

[0062] It is worth noting that before the obtained text data is matched with keywords, that is, before performing step S13, natural language processing technology can be used to clean, segment, remove stop words, etc., to obtain more semantic information, wherein the natural language processing technology can be: BERT model, which preprocesses text and extracts keywords through a deep learning model, and is suitable for extracting useful information from large amounts of text; TF-IDF model, a statistical method used to measure the importance of a word in a text, commonly used in text extraction and keyword recognition.

[0063] In step S13, keyword matching is performed based on the text data to obtain key audio data, including:

[0064] Converting the text data into a text vector using a universal sentence encoder;

[0065] Calculate the cosine similarity between the text vector and the existing advertisement keyword vector, and rank them to obtain a key vector; wherein the existing advertisement keyword vector is a vector converted by a sentence encoder of the advertisement-related text, and the key vector is a text vector corresponding to the cosine similarity ranked the highest among the cosine similarities;

[0066] According to the key vector, reversely infer the corresponding highest cosine similarity, and determine whether the highest cosine similarity is greater than a first preset threshold;

[0067] When it is determined that the highest cosine similarity is greater than the first preset threshold, the audio data corresponding to the text data is used as key audio data.

[0068] Specifically, the cosine similarity calculation formula is:

[0069]

[0070] In the formula, is the text vector, is the existing advertisement keyword vector, express The inner product of is a vector , Model.

[0071] It is worth noting that the sentence encoder is a natural language processing (NLP) technology tool that is used to convert an entire sentence or paragraph into a vector representation of a fixed dimension. This vector representation can capture the semantic information of the sentence and make it usable for various tasks in machine learning models, such as text classification, similarity calculation, sentiment analysis, etc. In the present invention, the role of the sentence encoder is to convert the extracted text data into a text vector, and then perform keyword matching and similarity calculation. Specifically, the sentence encoder in the present invention converts natural language text into a fixed-dimensional vector, so that the text data can be calculated with the existing advertising keyword vector, thereby finding the most relevant key audio data.

[0072] Specifically, pre-trained models for sentence encoders can include: Universal Sentence Encoder, which can provide universal sentence embeddings across multiple languages; Sent2Vec, which uses the average or weighted average of word vectors to obtain sentence vectors; GPT (Generative Pre-trained Transformer), a model developed by OpenAI, which can generate sentence vectors from text.

[0073] It is worth noting that cosine similarity is a method for measuring the similarity between two vectors. It evaluates their similarity by calculating the cosine value of the angle between two vectors in the vector space. The closer the cosine value is to 1, the smaller the angle between the two vectors is and the higher the similarity is; conversely, the closer the cosine value is to -1, the larger the angle between the two vectors is and the lower the similarity is. In the advertising display screen control method of the present invention, cosine similarity is used to calculate the similarity between a text vector and an existing advertising keyword vector. First, the text data is converted into a text vector by a universal sentence encoder, and the advertising keywords are also converted into corresponding advertising keyword vectors. Then, the cosine similarity calculation formula is used to calculate the cosine similarity between the text vector and each advertising keyword vector, and the ranking is performed. The text vector corresponding to the top cosine similarity is considered to be the most relevant to the advertising content, that is, the key vector. Finally, according to the comparison result of the highest cosine similarity of the key vector and the preset threshold, it is determined whether the text data is used as key audio data. If the highest cosine similarity is greater than the preset threshold, the audio data corresponding to the text data is regarded as key audio data for subsequent processing steps. The role of cosine similarity is to measure the similarity between two vectors (i.e., text vector and ad keyword vector), thereby helping the system find the voice data most relevant to the ad content. Specifically, cosine similarity is used to compare the similarity between the text vector converted from the customer audio and the ad keyword vector pre-stored in the system, in order to facilitate keyword matching.

[0074] For example, suppose there is an advertisement text about "smart watches" in the existing advertisement text. At this time, based on steps S11-S13, the text A converted from the audio data is obtained: "I am looking for a watch with health monitoring function"; the existing advertisement text B in the system is: "The latest smart watch with health monitoring function". Assume that the vector of text A obtained by the sentence encoder is , then the module length is ; The vector of advertisement text B is , then the module length is ; The cosine similarity is calculated to be 0.82.

[0075] Specifically, since there are multiple passers-by in environments such as shopping malls, office buildings, and subway stations, each passer-by may generate audio data. Therefore, the audio data acquisition sensor will capture multiple audio clips, and each audio clip will generate corresponding text data after text extraction. For each text data, the system will convert it into a text vector and perform cosine similarity calculations with the advertising keyword vectors pre-stored in the system one by one. Since each text data will perform cosine similarity calculations with multiple advertising keyword vectors, multiple cosine similarity values ​​will be generated. The system will sort these values ​​and find the largest cosine similarity value. The text data corresponding to this value is considered to be the most matched to a certain advertising keyword, that is, the customer's audio data clearly mentions this advertising keyword, indicating that the customer is talking about the content in the advertisement.

[0076] Specifically, the preset threshold is used to determine whether the cosine similarity is high enough, so as to determine whether the user voice data matches the advertising keywords sufficiently. The method for determining the preset threshold can be: setting it according to experience, interviewing customers about whether their consumption of the advertising product is based on the advertising display screen to determine whether the set threshold is reasonable; determining it according to the complexity of semantic similarity, in some scenarios, it may be necessary to introduce more semantic factors, for example, some keywords may have ambiguity or contextual relevance, in this case, different thresholds can be set for these keywords, where a higher threshold can be set for highly relevant and semantically clear keywords, and a lower threshold can be set for words with a wider semantic range; determining it according to industry standards and reference values, in the field of natural language processing, about 0.7 is often used as the basic threshold of text similarity to determine whether two texts are semantically related, and in specific practical application scenarios, this value can be slightly increased or decreased. When it is determined that the highest cosine similarity is greater than the first preset threshold, it means that the customer is talking about content related to the advertisement, and the corresponding audio data is used as key audio data.

[0077] In one embodiment, the converting the text data into a text vector using a universal sentence encoder includes:

[0078] Extract words based on the text data to obtain words in the sentences of the text data;

[0079] According to the words, word embedding processing is performed using the universal sentence encoder to obtain a word vector;

[0080] The word vectors are summed to obtain a sentence vector;

[0081] The sentence vector is divided by the square root of the length of the sentence vector to obtain a text vector.

[0082] It is worth noting that word embedding is a technique in natural language processing (NLP) for mapping words or phrases in a vocabulary to vectors in a high-dimensional space. This mapping enables similar words to have a close distance in the vector space, thereby capturing the semantic relationship between words. In the embodiment, first, all words are extracted from the text data. This usually involves a word segmentation process, that is, dividing the text into a sequence of words or phrases. Secondly, the extracted words are word embedded using a universal sentence encoder, which learns the co-occurrence relationship, contextual relationship, etc. between words by training a large amount of text data, thereby generating a vector representation of each word. In the word embedding process, each word is mapped to a vector in a high-dimensional space. The dimensions of these vectors are usually pre-set, such as 100 dimensions, 200 dimensions or higher. Then, the vectors of the words are summed to obtain a sentence vector, and the sentence vector is divided by the square root of the length of the sentence vector. This representation method is called sentence embedding. The semantic similarity between words and sentences can be evaluated by calculating the similarity (cosine similarity) between vectors.

[0083] Specifically, the word vectors are summed to obtain a sentence vector; and the sentence vector is then divided by the square root of the sentence vector length to obtain a text vector. The purpose of obtaining the text vector is to reduce the influence of the sentence length in the audio data on the word vector and obtain a text vector of appropriate length to facilitate the subsequent calculation of cosine similarity.

[0084] In step S14, the corresponding image acquisition sensor is matched according to the key audio data, and posture extraction is performed to obtain the head angle, including:

[0085] According to the key audio data, matching the audio acquisition sensor that acquires the key audio data to obtain the key audio sensor;

[0086] According to the key audio sensor, the facial image acquisition sensor of the corresponding area is matched to obtain the key video sensor;

[0087] According to the key video sensor, posture extraction is performed to obtain the head angle.

[0088] Specifically, image acquisition sensors, like audio acquisition sensors, are pre-installed in shopping malls, office buildings, and subway stations. It is worth noting that the image acquisition sensors should have a preset mapping relationship with the audio acquisition sensors. For example, a mapping relationship is preset between audio acquisition sensor A and image acquisition sensor A. At this time, if audio acquisition sensor A obtains key audio data, the matching facial image acquisition sensor is image acquisition sensor A, where image acquisition sensor A should be located near audio acquisition sensor A to facilitate capturing the source of the audio data, i.e., the customer.

[0089] It is worth noting that the image acquisition sensor should cover the same area or position as the audio acquisition sensor to ensure that when the audio acquisition sensor captures the customer's voice, the image acquisition sensor can capture the image of the same customer. At the same time, the distance between the two should not be too large to avoid time delay or asynchrony between sound and image acquisition, which will affect the system's accurate judgment of customer behavior. In addition, the installation location of the sensor should avoid interference from other sound sources or visual obstacles to avoid affecting the accuracy of audio and image data. Obviously, the key video sensor and the key audio sensor also correspond one to one.

[0090] In one implementation, performing posture extraction according to the key video sensor to obtain the head angle includes:

[0091] According to the key video sensor, video acquisition is performed to obtain video data;

[0092] According to the video data, the VIBE model is used to extract posture and obtain the head angle.

[0093] Specifically, the VIBE (Video Inference for Human Body Pose Estimation) model is a deep learning model for human body posture estimation, which can predict the position of each joint of the human body by analyzing the human body posture in the video frame. The VIBE model usually uses a convolutional neural network (CNN) to extract image features, and combines a recurrent neural network (RNN) to process the timing information in the video sequence, so as to accurately capture the human body movements in the video. In the present invention, the role of the VIBE model is: to perform posture extraction, using the video data collected by the key video sensor, the VIBE model can identify and extract the head posture of the characters in the video, especially the angle of the head; the acquisition of the head angle, through the VIBE model analysis, can obtain the head angle of the customer watching the advertising display screen.

[0094] It is worth noting that before the collected video data is used for subsequent posture extraction, some preprocessing operations may be required, such as denoising, contrast enhancement, image resizing, etc., to improve the accuracy and efficiency of subsequent processing.

[0095] The purpose of step S14 is to match the video sensor corresponding to the audio sensor after obtaining the key audio data, and further determine whether they are paying attention to the advertising display screen by extracting their head posture.

[0096] In step S15, a head angle judgment is performed according to the head angle, and a first brightness adjustment instruction or a second brightness adjustment instruction is generated.

[0097] In one implementation, the step of determining the head angle according to the head angle and generating the first brightness adjustment instruction or the second brightness adjustment instruction includes:

[0098] Determine whether the head angle is within a second preset threshold, and if so, generate a first brightness adjustment instruction; if not, generate a second brightness adjustment instruction.

[0099] It is worth noting that the brightness adjustment trend corresponding to the first brightness adjustment instruction is from bright to dark, and the brightness adjustment trend corresponding to the second brightness adjustment instruction is from dark to bright. Obviously, the reason for setting the adjustment trend is: when the image acquisition sensor detects that the customer's attention is on the screen (that is, the head angle is within the preset threshold), the purpose of saving energy can be achieved by reducing the brightness; when the image acquisition sensor detects that the customer is not watching the advertisement (that is, the head angle is not within the preset threshold), increasing the brightness helps to attract the customer's attention and improve the effectiveness of the advertisement.

[0100] Specifically, determining whether the head angle is within a preset threshold is an important basis for judging whether the customer is watching the advertising display screen, wherein the method for determining the preset threshold can be: based on experience setting, the preset threshold can be set according to historical experience, based on a large amount of experimental data or observations in actual applications, and a more reasonable angle range is obtained through statistical analysis; based on customer behavior analysis, the preset threshold can be determined by analyzing the customer's viewing behavior, for example, observing at what head angle the customer can clearly see the content on the advertising display screen, and the correlation between this angle and the customer's actual viewing behavior; considering environmental factors, the determination of the preset threshold also needs to consider environmental factors, such as the installation position, height, tilt angle, etc. of the advertising display screen, these factors will affect the best head angle when the customer views the advertising display screen; industry standards and reference values, refer to the industry head angle standards to determine whether the head angle within a certain threshold can be determined as viewing or not viewing.

[0101] It is worth noting that the brightness adjustment trend should be determined according to the actual situation to ensure that it can optimize energy consumption and improve the viewing experience and advertising effect of the advertising display screen: determined according to customer response, by analyzing the audience's viewing behavior and response, to determine the size of the adjustment trend; determined according to the indoor ambient light, the ambient light conditions have a significant impact on the visibility of the display screen. In a dark environment, the display screen may require a higher brightness to be clearly visible; in a well-lit environment, reducing the brightness can reduce light pollution and save energy; determined according to the position of the display screen, the brightness adjustment trend will change with the different positions of the display screen; determined according to the display content, as the advertising content changes, the brightness adjustment trend of the display screen will also change accordingly. It is worth noting that the above adjustment methods can be adjusted through models or algorithms such as machine learning models, deep learning models, reinforcement learning, and fuzzy logic control.

[0102] In step S16, after obtaining the first brightness adjustment instruction or the second brightness adjustment instruction, the first brightness adjustment instruction or the second brightness adjustment instruction is sent to the advertising display screen, so that the advertising display screen adjusts the brightness according to the first brightness adjustment instruction or the second brightness adjustment instruction.

[0103] The following describes the working process of the present invention by taking a relatively common scenario as an example. The working process is as follows:

[0104] First, multiple audio collection sensors are installed in the mall. These sensors are distributed near the advertising display screens or key traffic areas to capture customers' conversations and the sounds of the surrounding environment in real time. These audio sensors can be electret microphones, condenser microphones or array microphones. The most suitable type is selected according to the specific environment to ensure the clarity and accuracy of sound collection.

[0105] When customers walk or stop in the mall, their conversations or the sounds of the surrounding environment are captured by audio collection sensors and converted into electrical signals. Subsequently, these audio data are processed by automatic speech recognition technology (ASR) to convert the voice signals into text data. ASR technology uses machine learning models such as deep neural networks (DNN) or convolutional neural networks (CNN) to perform high-precision speech-to-text conversion on audio data. At the same time, audio data will also be processed for noise reduction to improve the accuracy of text conversion.

[0106] After obtaining the text data, the system will use a universal sentence encoder to convert the text data into a text vector. This process includes steps such as word extraction, word embedding processing, word vector summation, and normalization, and finally obtains a text vector that can represent the text content.

[0107] Next, the system uses these text vectors to calculate cosine similarity with the preset ad keyword vectors. The ad keyword vector is a vector obtained by converting the ad-related text through a sentence encoder, representing the keywords of the ad content. By calculating the cosine similarity between the text vector and the ad keyword vector and ranking them, the system can identify customer discussions related to the ad content. When the highest cosine similarity is greater than the preset threshold, the corresponding audio data is considered key audio data, indicating that the customer is interested in a keyword in the ad.

[0108] Once key audio data is detected, the system activates the image acquisition sensors corresponding to the audio acquisition area. These image acquisition sensors capture the customer's facial image and perform posture extraction, especially head angle extraction, through a posture estimation model (such as the VIBE model). Before extracting the head angle, the system may pre-process the video data, including denoising, contrast enhancement, image resizing, etc., to improve the accuracy of posture extraction.

[0109] The VIBE model predicts the customer's head angle by analyzing the human posture, especially the head posture, in the video frame. The system compares this head angle with the preset threshold to determine whether the customer is watching the advertising display. If the head angle is within the preset threshold, the system believes that the customer is paying attention to the advertising display and generates a first brightness adjustment instruction to reduce the brightness of the advertising display to save energy. On the contrary, if the head angle is not within the preset threshold, the system believes that the customer is not paying attention to the advertising display and generates a second brightness adjustment instruction to increase the brightness of the advertising display to attract the customer's attention.

[0110] Finally, the system sends these brightness adjustment instructions to the advertising display screen, and the advertising display screen adjusts its brightness according to the received instructions. In this way, the present invention can reduce unnecessary energy consumption while improving the effectiveness of advertising, and realize a more intelligent and environmentally friendly advertising display method.

[0111] In summary, the present invention provides an operation control method for an inverter, including: acquiring audio data of a customer; performing text extraction based on the audio data to obtain text data; performing keyword matching based on the text data to obtain key audio data; matching a corresponding image acquisition sensor based on the key audio data, and performing posture extraction to obtain a head angle; performing head angle judgment based on the head angle to generate a first brightness adjustment instruction or a second brightness adjustment instruction; sending the first brightness adjustment instruction or the second brightness adjustment instruction to an advertising display screen, so that the advertising display screen adjusts the brightness according to the first brightness adjustment instruction or the second brightness adjustment instruction.

[0112] In the present invention, the method can collect audio data through an audio collection sensor installed indoors, and extract keywords that are repeated in the audio data and related to existing advertising texts. When keywords related to advertising texts in the audio data are extracted, the image collection sensor of the corresponding area can be matched, and the image collection sensor can be used to extract features of the head posture of passers-by in a specific area to determine whether the passers-by are looking at the advertising display screen, and then adjust the brightness of the advertising display screen by determining whether the passers-by are looking at the advertising display screen. The method first collects audio information of passers-by, and when it is determined that passers-by in a certain area are interested in a certain keyword in the advertisement, the head posture information of passers-by in the specific area is collected, and then the brightness of the advertising display screen in the area is adjusted, which can reduce unnecessary energy consumption while improving the effectiveness of advertising.

[0113] Reference Figure 2 The second embodiment of the present invention provides a control system for an advertising display screen, comprising:

[0114] A data collection module, used to obtain audio data of customers;

[0115] A text extraction module, used to extract text based on the audio data to obtain text data;

[0116] A word matching module, used to perform keyword matching based on the text data to obtain key audio data;

[0117] A posture extraction module, used to match the corresponding image acquisition sensor according to the key audio data, and perform posture extraction to obtain the head angle;

[0118] An angle judgment module, used to judge the head angle according to the head angle, and generate a first brightness adjustment instruction or a second brightness adjustment instruction;

[0119] The instruction sending module is used to send the first brightness adjustment instruction or the second brightness adjustment instruction to the advertising display screen, so that the advertising display screen adjusts the brightness according to the first brightness adjustment instruction or the second brightness adjustment instruction.

[0120] In an optional implementation, the term matching module is specifically used to:

[0121] Converting the text data into a text vector using a universal sentence encoder;

[0122] Calculate the cosine similarity between the text vector and the existing advertisement keyword vector, and rank them to obtain a key vector; wherein the existing advertisement keyword vector is a vector converted by a sentence encoder of the advertisement-related text, and the key vector is a text vector corresponding to the cosine similarity ranked the highest among the cosine similarities;

[0123] According to the key vector, reversely infer the corresponding highest cosine similarity, and determine whether the highest cosine similarity is greater than a first preset threshold;

[0124] When it is determined that the highest cosine similarity is greater than the first preset threshold, the audio data corresponding to the text data is used as key audio data.

[0125] In an optional implementation, the term matching module is specifically used to:

[0126] Extract words based on the text data to obtain words in the sentences of the text data;

[0127] According to the words, word embedding processing is performed using the universal sentence encoder to obtain a word vector;

[0128] The word vectors are summed to obtain a sentence vector;

[0129] The sentence vector is divided by the square root of the length of the sentence vector to obtain a text vector.

[0130] In an optional implementation, the posture extraction module is specifically used for:

[0131] According to the key audio data, matching the audio acquisition sensor that acquires the key audio data to obtain the key audio sensor;

[0132] According to the key audio sensor, the facial image acquisition sensor of the corresponding area is matched to obtain the key video sensor;

[0133] According to the key video sensor, posture extraction is performed to obtain the head angle.

[0134] In an optional implementation, the posture extraction module is specifically used for:

[0135] According to the key video sensor, video acquisition is performed to obtain video data;

[0136] According to the video data, the VIBE model is used to extract posture and obtain the head angle.

[0137] In an optional implementation, the angle determination module is specifically used for:

[0138] Determine whether the head angle is within a second preset threshold, and if so, generate a first brightness adjustment instruction; if not, generate a second brightness adjustment instruction.

[0139] It should be noted that the control system of an advertising display screen provided in an embodiment of the present invention is used to execute all the process steps of a control method of an advertising display screen in the above embodiment, and the working principles and beneficial effects of the two correspond one to one, so they will not be repeated here.

[0140] The embodiment of the present invention further provides an electronic device. The electronic device includes: a processor, a memory, and a computer program stored in the memory and executable on the processor, such as a control program for an advertising display screen. When the processor executes the computer program, the steps in the above-mentioned embodiments of the control method for an advertising display screen are implemented, such as Figure 1 Alternatively, when the processor executes the computer program, the functions of the modules / units in the above-mentioned system embodiments are realized, such as the angle determination module.

[0141] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program in the electronic device.

[0142] The electronic device may be a computing device such as a desktop computer, a notebook, a PDA, and a smart tablet. The electronic device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the above components are merely examples of electronic devices and do not constitute a limitation on the electronic device. The electronic device may include more or fewer components than the above components, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0143] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device.

[0144] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the electronic device by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0145] Wherein, if the module / unit integrated in the electronic device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or system capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0146] It should be noted that the system embodiment described above is merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the system embodiment provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.

[0147] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for controlling an advertising display screen, characterized in that: include: Get audio data from customers; Extracting text based on the audio data to obtain text data; According to the text data, keyword matching is performed to obtain key audio data; According to the key audio data, the corresponding image acquisition sensor is matched, and posture extraction is performed to obtain the head angle; According to the head angle, head angle judgment is performed to generate a first brightness adjustment instruction or a second brightness adjustment instruction; Sending the first brightness adjustment instruction or the second brightness adjustment instruction to the advertising display screen, so that the advertising display screen adjusts the brightness according to the first brightness adjustment instruction or the second brightness adjustment instruction; The step of matching the corresponding image acquisition sensor according to the key audio data and performing posture extraction to obtain the head angle includes: According to the key audio data, matching the audio acquisition sensor that acquires the key audio data to obtain the key audio sensor; According to the key audio sensor, the facial image acquisition sensor of the corresponding area is matched to obtain the key video sensor; Perform posture extraction according to the key video sensor to obtain the head angle; The step of performing posture extraction according to the key video sensor to obtain the head angle includes: According to the key video sensor, video acquisition is performed to obtain video data; According to the video data, posture extraction is performed using the VIBE model to obtain the head angle; The step of determining the head angle according to the head angle and generating a first brightness adjustment instruction or a second brightness adjustment instruction includes: Determine whether the head angle is within a second preset threshold, and if so, generate a first brightness adjustment instruction; if not, generate a second brightness adjustment instruction; The brightness adjustment trend corresponding to the first brightness adjustment instruction is from bright to dark, and the brightness adjustment trend corresponding to the second brightness adjustment instruction is from dark to bright.

2. The control method of the advertising display screen according to claim 1, characterized in that: The step of performing keyword matching according to the text data to obtain key audio data includes: Converting the text data into a text vector using a universal sentence encoder; Calculate the cosine similarity between the text vector and the existing advertisement keyword vector, and rank them to obtain a key vector; wherein the existing advertisement keyword vector is a vector converted by a sentence encoder of the advertisement-related text, and the key vector is a text vector corresponding to the cosine similarity ranked the highest among the cosine similarities; According to the key vector, reversely infer the corresponding highest cosine similarity, and determine whether the highest cosine similarity is greater than a first preset threshold; When it is determined that the highest cosine similarity is greater than the first preset threshold, the audio data corresponding to the text data is used as key audio data.

3. The control method of the advertising display screen according to claim 2, characterized in that: The converting the text data into a text vector using a universal sentence encoder comprises: Extract words based on the text data to obtain words in the sentences of the text data; According to the words, word embedding processing is performed using the universal sentence encoder to obtain a word vector; The word vectors are summed to obtain a sentence vector; The sentence vector is divided by the square root of the length of the sentence vector to obtain a text vector.

4. The control method of the advertising display screen according to claim 2, characterized in that: The cosine similarity calculation formula is: In the formula, is the text vector, is the existing advertisement keyword vector, express The inner product of is a vector , Model.

5. A control system for an advertising display screen, characterized in that: A method for controlling an advertising display screen according to any one of claims 1 to 4, comprising: A data collection module, used to obtain audio data of customers; A text extraction module, used to extract text based on the audio data to obtain text data; A word matching module, used to perform keyword matching based on the text data to obtain key audio data; A posture extraction module, used to match the corresponding image acquisition sensor according to the key audio data, and perform posture extraction to obtain the head angle; An angle judgment module, used to judge the head angle according to the head angle, and generate a first brightness adjustment instruction or a second brightness adjustment instruction; The instruction sending module is used to send the first brightness adjustment instruction or the second brightness adjustment instruction to the advertising display screen, so that the advertising display screen adjusts the brightness according to the first brightness adjustment instruction or the second brightness adjustment instruction.

6. An electronic device, characterized in that: The invention comprises a processor, a memory and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the control method of the advertising display screen according to any one of claims 1 to 4 when executing the computer program.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the method for controlling an advertising display screen according to any one of claims 1 to 4.

Images