A method and system for placing and correcting public transportation advertisements based on large models and machine vision

Through large models and machine vision technology, combined with adaptive sparrow search, hornet whiskers and simulated annealing algorithm, personalized generation and real-time correction of public transportation advertisements are achieved, solving the problem of poor public transportation advertisement delivery results, and improving advertising quality and revenue .

CN118333696BActive Publication Date: 2024-11-15NAN JING INTELLIGENT TRANSPORTATION INFORMATION CO LTD
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202410749633.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-06-12
Publication Date
2024-11-15
Estimated Expiration
2044-06-12

AI Technical Summary

Technical Problem

During the public transportation advertisement, corresponding advertisements cannot be generated based on the passenger's age, gender and height, and cannot be corrected in time, resulting in poor advertising effectiveness and reducing the advertising revenue of public transportation companies.

Method used

Using a method based on big model and machine vision, RBF neural network-decision-level fusion algorithm optimized by adaptive sparrow search algorithm to detect passenger age, BP neural network optimized by Tianniu Xu algorithm to detect passenger height, and add mixed frogs that simulated annealing The jump algorithm classifies and detects passenger gender, and combines the promotional product library to generate advertising copy and posters, verify attention level in real time, and automatically updates the poorly effective advertising content.

Benefits of technology

It realizes accurate detection of passenger-related characteristics and personalized generation of advertising content, improves advertising quality and update efficiency, improves passenger travel experience, and reduces labor costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118333696B_ABST
    Figure CN118333696B_ABST
Patent Text Reader

Abstract

The present invention discloses a method and system for placing and correcting bus advertisements based on a large model and machine vision. Firstly, a RBF neural network-decision-level fusion algorithm optimized based on an adaptive sparrow search algorithm is used to detect the age group of passengers and obtain the detection result of the age group of passengers. Then, a BP neural network algorithm optimized based on a longicorn beard algorithm is used to detect the height of passengers and obtain the detection result of the height of passengers. The present invention realizes the detection of relevant characteristics of passengers and uses different algorithms to detect the age group, gender and height. It also realizes the automatic generation of advertisement copy and poster according to passenger characteristics and product information by using large model technology, and also realizes the real-time verification of the advertisement copy and poster effect by the degree of attention. The whole process does not require manual detection, judgment and update, thereby improving the advertisement quality and update feedback efficiency, and is suitable for being widely promoted and used.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of public transportation advertisement placement, and in particular to a public transportation advertisement placement and correction method and system based on a large model and machine vision. Background Art

[0002] Public transportation advertising refers to a marketing method of placing advertisements on buses or at bus stops. This form of advertising has a wide coverage and high exposure rate, because buses are one of the main means of transportation for urban residents. When placing public transportation advertising, it is necessary to fully consider the target audience, advertising form, delivery time and budget factors to improve advertising effect and return rate.

[0003] At present, most of the bus advertisements are placed in a unified manner according to the design sequence, resulting in poor viewing of bus advertisements. Not only can the corresponding advertisements not be generated according to the age, gender, height and promotional product library of passengers in the bus, but timely corrections cannot be made during the advertisement placement process, which seriously affects the effect of advertisement placement and reduces the advertising revenue of bus companies. Therefore, it is necessary to design a bus advertisement placement and correction method and system based on large models and machine vision. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the prior art and to better and effectively solve the problem that most of the current bus advertisements are placed in a unified manner according to the design sequence, resulting in poor viewing of bus advertisements. Not only can the corresponding advertisements not be generated according to the age, gender, height and promotional product library of passengers in the bus, but timely corrections cannot be made during the advertisement placement process, which seriously affects the effect of advertisement placement, which also reduces the advertising revenue of the bus company. A bus advertisement placement and correction method and system based on a large model and machine vision are provided, which realizes the detection of passenger-related characteristics and uses different algorithms to detect age, gender and height, and also realizes the automatic generation of advertising copy and posters according to passenger characteristics and product information using large model technology, and also realizes real-time verification of the effects of advertising copy and posters through the degree of attention, and advertising copy and posters with poor effects will be automatically updated. The entire process does not require manual detection, judgment and updating, thereby improving the advertising quality and update feedback efficiency, and improving the travel experience of passengers.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is:

[0006] A method for placing and correcting bus advertisements based on a large model and machine vision comprises the following steps:

[0007] Step A, using the RBF neural network-decision-level fusion algorithm optimized by the adaptive sparrow search algorithm to detect the age group of passengers and obtain the detection results of the age group of passengers;

[0008] Step B, using a BP neural network algorithm optimized based on a beetle whisker algorithm to detect the height of the passenger, and obtaining a passenger height detection result;

[0009] Step C, using the SVM algorithm optimized by the hybrid frog leaping algorithm with simulated annealing to classify and detect the gender of the passengers, and obtaining the passenger gender classification detection result;

[0010] Step D, generating advertising copy and poster keywords based on the passenger age group detection results, the passenger height detection results and the passenger gender classification detection results in combination with the promotional product library, and then making a promotional poster from the advertising copy and poster keywords and displaying it on the bus display;

[0011] Step E, detecting the passenger's head posture and obtaining the passenger's head posture detection result;

[0012] Step F, setting a threshold based on the passenger head posture detection result and determining the passenger's concentration level on the promotional poster, and then modifying the display content of the promotional poster according to the passenger's concentration level on the promotional poster.

[0013] The aforementioned method for placing and correcting public transportation advertisements based on a large model and machine vision, step A, uses an RBF neural network-decision-level fusion algorithm optimized based on an adaptive sparrow search algorithm to detect the age group of passengers, and obtains the detection results of the age group of passengers, wherein the input variable is the feature data after the face data is reduced in dimension using LGBP, and the adaptive crossover operation and mutation operation in the adaptive sparrow search algorithm are used to optimize the center value, width value, weight and age group accuracy of the RBF neural network, and the adaptive crossover operation and mutation operation in the adaptive sparrow search algorithm are specifically to sort the fitness of individual sparrows in the process of sparrow population finding food and following reconnaissance and early warning, and select the sparrow individual with the best fitness at present for crossover operation and mutation operation, and the specific steps are as follows,

[0014] Step A1, crossover operation, select sparrow individuals in the population and randomly pair them with each other, then cross the paired sparrow individuals through the crossover operation probability A crossover operation occurs, where the two sparrows in the crossover operation and The crossover formula is shown in formula (1).

[0015]

[0016] in, Represents sparrow Location, Represents sparrow Location, Represents sparrow Position after crossover operation, Represents sparrow Position after crossover operation, Indicates credibility, Represents a random number between the interval (0, 1);

[0017] Step A2: mutation operation, the position of the sparrow individual is mutated with probability Mutation occurs, set the individual optimal value No. The dimension variable is , then Applying random perturbation variables to perform mutation operations is shown in formula (2):

[0018]

[0019] in, is a random disturbance variable and obeys a normal distribution with a mean of 0 and a variance of 1;

[0020] Step A3, adaptive crossover operation and mutation operation, using adaptive crossover operation and mutation operation to make the crossover operation probability and mutation operation probability Can be adjusted automatically, as shown in formula (3) and formula (4),

[0021]

[0022]

[0023] in, and is a constant, is the average fitness of the entire population, is the maximum fitness of the entire population, is the minimum fitness of the entire population, is the global optimal fitness;

[0024] Step A4, compare the fitness of offspring with that of the parent generation and eliminate individuals with poor fitness;

[0025] Step A5, using the RBF neural network to establish an age group determination model, and then using the adaptive crossover operation and mutation operation in the adaptive sparrow search algorithm to optimize the center value, width value, and weight value of the RBF neural network;

[0026] Step A6, use the KPCA algorithm to reduce the feature dimension of the original face image data, and then perform decision-level fusion on the reduced-dimensional data based on the confidence of the result obtained by the SVM algorithm to obtain the passenger age group detection result. The specific steps are as follows:

[0027] Step A61: Use KPCA to extract features from face data Use SVM algorithm for classification and get the usage features Belong to The probability of the class is , and then the probability The probabilities of sorting by size and getting the top three are , and ,in , and They are the top three categories in terms of probability, and the credibility As shown in formula (5),

[0028]

[0029] Step A62: Use LGBP to extract features from face data Use adaptive sparrow search algorithm and RBF neural network for classification and obtain credibility ;

[0030] Step A63, credibility and credibility The weights of the weighted fusion algorithm are calculated and obtained as shown in formula (6):

[0031]

[0032] in, and They represent the weights calculated based on the SVM algorithm and the weights obtained based on the fusion of the adaptive sparrow search algorithm and the RBF neural network respectively;

[0033] Step A64, calculate the fused image belongs to Class Probability As shown in formula (7), the passenger age group detection result is In The value represents the age group.

[0034] .

[0035] The aforementioned method for placing and correcting public transportation advertisements based on a large model and machine vision, step B, uses a BP neural network algorithm optimized based on the beetle whisker algorithm to detect the height of the passenger to obtain the passenger height detection result, wherein the optimized BP neural network algorithm uses a two-dimensional image and a three-dimensional world coordinate to fit the key points detected by the human skeleton points, thereby detecting the three-dimensional visual coordinates of the key skeleton points, and then calculating the length of each section of the body skeleton to estimate the passenger height. The specific steps are as follows,

[0036] Step B1, set the random vector of the beetle's whiskers and define the spatial dimension , and then set the model structure to , where M is the number of neurons in the input layer, S is the number of neurons in the hidden layer, 3 is the number of neurons in the output layer, and the search space dimension is ;

[0037] Step B2, set the step size factor , where the step size factor The regional search capability used to control the longhorn beetle is shown in formula (8):

[0038]

[0039] Among them, eta is a number close to 1 between [0, 1];

[0040] Step B3, using the root mean square error MSE of the test data as a fitness evaluation function, and the fitness evaluation function is used to promote the search of the spatial area, as shown in formula (9),

[0041]

[0042] in, Represents fitness. When the algorithm stops iterating, the location where the fitness function has the smallest value is the optimal solution. is the number of training set samples, For the The model output value of samples, For the The actual value of samples;

[0043] Step B3, initializing the position of the longhorn beetle, wherein the initial parameter of the longhorn beetle position takes a random number between [-0.25, 0.75] as the initial solution set of the longhorn beetle whisker algorithm, and saves it in bestX;

[0044] Step B4, calculating the fitness function value at the initial position according to the fitness function and saving it in bestY;

[0045] Step B5, updating the positions of the left and right whiskers of the longhorn beetle, wherein specifically the position coordinates of the left and right whiskers of the longhorn beetle are updated according to step B2;

[0046] Step B6, respectively calculating the fitness function values ​​of the left and right whiskers according to the left and right whisker positions in the beetle whisker algorithm and , and then compare and The strength of the beetle is updated according to formula (8), thereby adjusting the weights and thresholds of the BP neural network, and then calculating the fitness function value at the current position. If the fitness function value at this time is better than bestY, then update bestY and bestX;

[0047] Step B7, determine whether the fitness function value reaches the set accuracy or the iteration reaches the maximum number of times, if the condition is met, proceed to step B8, if not, return to step B5 to continue iteration;

[0048] Step B8, taking the solution in bestX as the optimal solution, thereby obtaining the optimal initial weight and threshold of the BP neural network, and then bringing the optimal solution into the BP neural network for secondary training and learning, thereby forming a three-dimensional coordinate prediction model of key skeleton points, and then obtaining the passenger height detection result.

[0049] The aforementioned method for placing and correcting public transportation advertisements based on a large model and machine vision, step C, uses an SVM algorithm optimized by a hybrid frog leaping algorithm with simulated annealing to classify and detect the gender of passengers, and obtains the passenger gender classification detection result. The specific steps are as follows:

[0050] Step C1, initialize the frog group, set the individual initial position to An arbitrary number between is the total number of individuals in the population, and the dimension of individuals is m, the number of subpopulations is M, the number of subpopulation local search iterations is num, the initial annealing temperature is T, the temperature cooling coefficient is C, and the disturbance amplitude is ;

[0051] Step C2, calculate the fitness value of each individual. If the SVM hyperparameter combination of a certain individual position does not meet the fitness value setting requirements, the fitness value of the individual is defined as , otherwise keep the fitness value unchanged;

[0052] Step C3, sort the fitness values ​​of all current individuals from best to worst and divide the individuals into sub-populations in turn;

[0053] Step C4, using the hybrid frog leaping algorithm to update the worst individual of the subpopulation to obtain a better solution;

[0054] Step C5, using the simulated annealing algorithm to calculate, as shown in formula (10),

[0055]

[0056] in, is a random variable, and the random variable It follows a normal distribution, For a new solution, and a new solution Comparison of post-utilization annealing mechanisms and quality;

[0057] Step C6, sort the sub-population fitness values, then return to step C4 and repeat the update until the number of iterations in the sub-population meets the given requirement num;

[0058] Step C7, after all sub-populations complete the update operation, if the current optimal individual meets the convergence condition, the evolution process ends successfully and returns to the global optimal solution, thereby obtaining the passenger gender classification detection result; otherwise, the annealing temperature of the modified population is the product of the initial annealing temperature T and the temperature cooling coefficient C.

[0059] The aforementioned method for placing and correcting bus advertisements based on a large model and machine vision, step D, generates advertisement text and poster keywords based on the passenger age group detection results, the passenger height detection results and the passenger gender classification detection results combined with the promotional product library, and then makes a promotional poster from the advertisement text and poster keywords and places and displays it on the bus display. The specific steps are as follows:

[0060] Step D1, taking the mode of the passenger age group test results, the mode of the passenger gender classification test results and the mean of the passenger height test results, and generating advertising copy and poster keywords in the order of the mode of the passenger age group test results, the mode of the passenger gender classification test results and the mean of the passenger height test results in combination with the promotional product library;

[0061] Step D2, generating a poster with the advertisement copy and poster keywords and displaying it on a bus-mounted display.

[0062] The aforementioned method for placing and correcting bus advertisements based on a large model and machine vision, step E, detects the passenger's head posture and obtains the passenger's head posture detection result, wherein the passenger's head posture detection is specifically to detect the continuous head deflection angle, and then the head deflection angle is discretized and a multi-output fusion loss method is used to realize the step-by-step detection from coarse to fine. The specific steps are as follows:

[0063] Step E1, classify the 181 continuous angle values ​​of the head posture angle from -90° to 90° according to the angle interval of 3°, then mark each posture angle with a classification label and use the category error rate loss function to perform classification prediction to obtain the head posture angle classification prediction value;

[0064] Step E2, mapping the head posture angle classification prediction value to the original continuous regression angle interval, and then using the smoothed mean absolute error loss function to predict the angle regression;

[0065] Step E3, the class error rate loss function and the smoothed mean absolute value error loss function are weighted and summed with set weights as the final objective function of the network, where the calculation formula of the final objective function of the network is shown in formula (11):

[0066]

[0067] in, represents the total loss function, represents the category error rate loss function, represents the smoothed mean absolute error loss function, is the hyperparameter of multi-loss fusion, and the hyperparameter of multi-loss fusion Used to measure the contribution of the discretized head pose classification task and the continuous head pose regression task to the total loss.

[0068] The aforementioned method for placing and correcting bus advertisements based on a large model and machine vision, step F, sets a threshold based on the passenger head posture detection result and determines the passenger's concentration on the promotional poster, and then corrects the display content of the promotional poster based on the passenger's concentration on the promotional poster, wherein if the head posture is within the set threshold range, it is determined that the passenger is concentrating his attention on the promotional poster.

[0069] A bus advertising placement and correction system based on large models and machine vision, including a passenger age detection module, a passenger height detection module, a passenger gender classification detection module, an advertising placement display module, a head posture detection module and an advertising correction module; the passenger age detection module is used to adopt an RBF based on the optimization of the adaptive sparrow search algorithm The neural network-decision-level fusion algorithm detects the age group of passengers and obtains the detection results of the age group of passengers; the passenger height detection module is used to detect the height of passengers using the BP neural network algorithm optimized by the longicorn beard algorithm to obtain the detection results of the height of passengers; the passenger gender classification detection module is used to classify and detect the gender of passengers using the SVM algorithm optimized by the hybrid frog leaping algorithm with simulated annealing to obtain the detection results of the gender classification of passengers; the advertising display module is used to generate advertising copy and poster keywords based on the passenger age group detection results, the passenger height detection results and the passenger gender classification detection results in combination with the promotional product library, and then make promotional posters from the advertising copy and poster keywords and display them on the bus display; the head posture detection module is used to detect the passenger head posture and obtain the passenger head posture detection results; the advertising correction module is used to set a threshold based on the passenger head posture detection result and judge the passenger's concentration on the promotional poster, and then correct the display content of the promotional poster according to the passenger's concentration on the promotional poster.

[0070] In the aforementioned bus advertisement placement and correction system based on large model and machine vision, the adaptive sparrow search algorithm in the passenger age group detection module is used to optimize the center value, width value, weight and age group accuracy of the RBF neural network.

[0071] In the aforementioned bus advertising placement and correction system based on large models and machine vision, the optimized BP neural network algorithm in the passenger height detection module uses a two-dimensional image to fit the three-dimensional world coordinates and fits the key points detected by the human skeleton points, thereby detecting the three-dimensional visual coordinates of the key skeleton points, and then calculating the length of each body bone segment to estimate the passenger height.

[0072] The beneficial effects of the present invention are as follows: a method and system for placing and correcting bus advertisements based on a large model and machine vision of the present invention, firstly, adopts an RBF neural network-decision-level fusion algorithm optimized based on an adaptive sparrow search algorithm to detect the age group of passengers and obtain the detection results of the age group of passengers, then adopts a BP neural network algorithm optimized based on a longicorn beard algorithm to detect the height of passengers and obtain the detection results of the height of passengers, then adopts an SVM algorithm optimized based on a hybrid frog leaping algorithm with simulated annealing to classify and detect the gender of passengers and obtain the detection results of the gender classification of passengers, then generates advertising copy and poster keywords based on the passenger age group detection results, the passenger height detection results and the passenger gender classification detection results in combination with a promotional product library, then makes a promotional poster from the advertising copy and poster keywords and places it on a bus display for display, then detects the passenger's head posture and obtains the detection results of the passenger's head posture, then sets a threshold based on the passenger's head posture detection results and judges the passenger The present invention can detect the concentration of passengers on the promotional posters, and then modify the displayed content of the promotional posters according to the concentration of passengers on the promotional posters; the present invention can effectively detect the relevant characteristics of passengers and adopt different innovative algorithms for age groups, genders and heights, which not only improves the detection effect, but also improves the practicality of the algorithm. It can also realize the automatic generation of advertising copy and posters based on passenger characteristics and product information using large model technology, without the need for manual design and review, which greatly improves the efficiency of advertising generation and reduces labor costs. It can also realize real-time verification of the effects of advertising copy and posters through the degree of attention, and advertising copy and posters with poor effects will be automatically updated. The whole process does not require manual detection, judgment and updating, which greatly improves the quality of advertising and the efficiency of update feedback, and improves the travel experience of passengers. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 It is an overall flow chart of a method for placing and correcting public transportation advertisements based on a large model and machine vision according to the present invention;

[0074] Figure 2 The present invention adopts the RBF neural network-decision-level fusion algorithm optimized based on the adaptive sparrow search algorithm to detect the flow chart of the age group of passengers;

[0075] Figure 3 The present invention adopts the BP neural network algorithm optimized based on the beetle whisker algorithm to detect the height of passengers.

[0076] Figure 4 This is a flow chart of the present invention using the SVM algorithm optimized by the hybrid frog leaping algorithm with simulated annealing to classify and detect the gender of passengers;

[0077] Figure 5It is a flow chart of the present invention for correcting the displayed content of the promotional poster according to the degree of passenger's attention concentration on the promotional poster;

[0078] Figure 6 This is a diagram of the operating interface of a bus advertisement placement and correction system based on a large model and machine vision according to the present invention. DETAILED DESCRIPTION

[0079] The present invention will be further described below in conjunction with the accompanying drawings.

[0080] like Figure 1 As shown, a method for placing and correcting bus advertisements based on a large model and machine vision of the present invention comprises the following steps:

[0081] like Figure 2 As shown, step A, using the RBF neural network-decision-level fusion algorithm optimized based on the adaptive sparrow search algorithm to detect the age group of passengers, and obtain the passenger age group detection result, wherein the input variable is the feature data after the face data is reduced in dimension using LGBP, and the adaptive crossover operation and mutation operation in the adaptive sparrow search algorithm are used to optimize the center value, width value, weight and age group accuracy of the RBF neural network. The adaptive crossover operation and mutation operation in the adaptive sparrow search algorithm are specifically to sort the fitness of sparrow individuals in the process of sparrow population finding food and following reconnaissance and early warning, and select the sparrow individual with the best fitness to perform crossover operation and mutation operation. The specific steps are as follows,

[0082] Step A1, crossover operation, select sparrow individuals in the population and randomly pair them with each other, then cross the paired sparrow individuals through the crossover operation probability A crossover operation occurs, where the two sparrows in the crossover operation and The crossover formula is shown in formula (1).

[0083]

[0084] in, Represents sparrow Location, Represents sparrow Location, Represents sparrow Position after crossover operation, Represents sparrow Position after crossover operation, Indicates credibility, Represents a random number between the interval (0, 1);

[0085] Step A2: mutation operation, the position of the sparrow individual is mutated with probability Mutation occurs, set the individual optimal value No. The dimension variable is , then Applying random perturbation variables to perform mutation operations is shown in formula (2):

[0086]

[0087] in, is a random disturbance variable and obeys a normal distribution with a mean of 0 and a variance of 1;

[0088] Step A3, adaptive crossover operation and mutation operation, using adaptive crossover operation and mutation operation to make the crossover operation probability and mutation operation probability Can be adjusted automatically, as shown in formula (3) and formula (4),

[0089]

[0090]

[0091] in, and is a constant, is the average fitness of the entire population, is the maximum fitness of the entire population, is the minimum fitness of the entire population, is the global optimal fitness;

[0092] Step A4, compare the fitness of offspring with that of the parent generation and eliminate individuals with poor fitness;

[0093] Step A5, using the RBF neural network to establish an age group determination model, and then using the adaptive crossover operation and mutation operation in the adaptive sparrow search algorithm to optimize the center value, width value, and weight value of the RBF neural network;

[0094] Step A6, use the KPCA algorithm to reduce the feature dimension of the original face image data, and then perform decision-level fusion on the reduced-dimensional data based on the confidence of the result obtained by the SVM algorithm to obtain the passenger age group detection result. The specific steps are as follows:

[0095] Step A61: Use KPCA to extract features from face data Use SVM algorithm for classification and get the usage features Belong to The probability of the class is , and then the probability The probabilities of sorting by size and getting the top three are , and ,in , and They are the top three categories in terms of probability, and the credibility As shown in formula (5),

[0096]

[0097] Step A62: Use LGBP to extract features from face data Use adaptive sparrow search algorithm and RBF neural network for classification and obtain credibility ;

[0098] Step A63, credibility and credibility The weights of the weighted fusion algorithm are calculated and obtained as shown in formula (6):

[0099]

[0100] in, and They represent the weights calculated based on the SVM algorithm and the weights obtained based on the fusion of the adaptive sparrow search algorithm and the RBF neural network respectively;

[0101] Step A64, calculate the fused image belongs to Probability of class As shown in formula (7), the passenger age group detection result is In The value represents the age group.

[0102] .

[0103] like Figure 3 As shown, step B uses a BP neural network algorithm optimized based on the beetle whisker algorithm to detect the passenger's height and obtain the passenger's height detection result, wherein the optimized BP neural network algorithm uses a two-dimensional image to fit the three-dimensional world coordinates and fits the key points detected by the human skeleton points, thereby detecting the three-dimensional visual coordinates of the key skeleton points, and then calculating the length of each body bone segment to estimate the passenger's height. The specific steps are as follows:

[0104] Step B1, set the random vector of the beetle's whiskers and define the spatial dimension , and then set the model structure to , where M is the number of neurons in the input layer, S is the number of neurons in the hidden layer, 3 is the number of neurons in the output layer, and the search space dimension is ;

[0105] Step B2, set the step size factor , where the step size factor The regional search capability used to control the longhorn beetle is shown in formula (8):

[0106]

[0107] Among them, eta is a number close to 1 between [0, 1];

[0108] Step B3, using the root mean square error MSE of the test data as a fitness evaluation function, and the fitness evaluation function is used to promote the search of the spatial area, as shown in formula (9),

[0109]

[0110] in, Represents fitness. When the algorithm stops iterating, the location where the fitness function has the smallest value is the optimal solution. is the number of training set samples, For the The model output value of samples, For the The actual value of samples;

[0111] Step B3, initializing the position of the longhorn beetle, wherein the initial parameter of the longhorn beetle position takes a random number between [-0.25, 0.75] as the initial solution set of the longhorn beetle whisker algorithm, and saves it in bestX;

[0112] Step B4, calculating the fitness function value at the initial position according to the fitness function and saving it in bestY;

[0113] Step B5, updating the positions of the left and right whiskers of the longhorn beetle, wherein specifically the position coordinates of the left and right whiskers of the longhorn beetle are updated according to step B2;

[0114] Step B6, respectively calculating the fitness function values ​​of the left and right whiskers according to the left and right whisker positions in the beetle whisker algorithm and , and then compare and The strength of the beetle is updated according to formula (8), thereby adjusting the weights and thresholds of the BP neural network, and then calculating the fitness function value at the current position. If the fitness function value at this time is better than bestY, then update bestY and bestX;

[0115] Step B7, determine whether the fitness function value reaches the set accuracy or the iteration reaches the maximum number of times, if the condition is met, proceed to step B8, if not, return to step B5 to continue iteration;

[0116] Step B8, taking the solution in bestX as the optimal solution, thereby obtaining the optimal initial weight and threshold of the BP neural network, and then bringing the optimal solution into the BP neural network for secondary training and learning, thereby forming a three-dimensional coordinate prediction model of key skeleton points, and then obtaining the passenger height detection result.

[0117] like Figure 4 As shown, step C uses the SVM algorithm optimized by the hybrid frog leaping algorithm with simulated annealing to classify and detect the gender of passengers, and obtains the passenger gender classification detection result. The specific steps are as follows:

[0118] Step C1, initialize the frog group, set the individual initial position to An arbitrary number between is the total number of individuals in the population, and the dimension of individuals is m, the number of subpopulations is M, the number of subpopulation local search iterations is num, the initial annealing temperature is T, the temperature cooling coefficient is C, and the disturbance amplitude is ;

[0119] Step C2, calculate the fitness value of each individual. If the SVM hyperparameter combination of a certain individual position does not meet the fitness value setting requirements, the fitness value of the individual is defined as , otherwise keep the fitness value unchanged;

[0120] Step C3, sort the fitness values ​​of all current individuals from best to worst and divide the individuals into sub-populations in turn;

[0121] Step C4, using the hybrid frog leaping algorithm to update the worst individual of the subpopulation to obtain a better solution;

[0122] Step C5, using the simulated annealing algorithm to calculate, as shown in formula (10),

[0123]

[0124] in, is a random variable, and the random variable It follows a normal distribution, For a new solution, and a new solution Comparison of post-utilization annealing mechanisms and quality;

[0125] Step C6, sort the sub-population fitness values, then return to step C4 and repeat the update until the number of iterations in the sub-population meets the given requirement num;

[0126] Step C7, after all sub-populations complete the update operation, if the current optimal individual meets the convergence condition, the evolution process ends successfully and returns to the global optimal solution, thereby obtaining the passenger gender classification detection result; otherwise, the annealing temperature of the modified population is the product of the initial annealing temperature T and the temperature cooling coefficient C.

[0127] Step D, based on the passenger age detection results, passenger height detection results and passenger gender classification detection results combined with the promotional product library to generate advertising copy and poster keywords, and then make a promotional poster from the advertising copy and poster keywords and display it on the bus display. The specific steps are as follows:

[0128] Step D1, taking the mode of the passenger age group test results, the mode of the passenger gender classification test results and the mean of the passenger height test results, and generating advertising copy and poster keywords in the order of the mode of the passenger age group test results, the mode of the passenger gender classification test results and the mean of the passenger height test results in combination with the promotional product library;

[0129] Step D2, generating a poster with the advertisement copy and poster keywords and displaying it on a bus-mounted display.

[0130] Step E: Detect the passenger's head posture and obtain the passenger's head posture detection result. Specifically, the passenger's head posture detection is to detect the continuous head deflection angle, and then discretize the head deflection angle and use a multi-output fusion loss method to achieve step-by-step detection from coarse to fine. The specific steps are as follows:

[0131] Step E1, classify the 181 continuous angle values ​​of the head posture angle from -90° to 90° according to the angle interval of 3°, then mark each posture angle with a classification label and use the category error rate loss function to perform classification prediction to obtain the head posture angle classification prediction value;

[0132] Step E2, mapping the head posture angle classification prediction value to the original continuous regression angle interval, and then using the smoothed mean absolute error loss function to predict the angle regression;

[0133] Step E3, the class error rate loss function and the smoothed mean absolute value error loss function are weighted and summed with set weights as the final objective function of the network, where the calculation formula of the final objective function of the network is shown in formula (11):

[0134]

[0135] in, represents the total loss function, represents the category error rate loss function, represents the smoothed mean absolute error loss function, is the hyperparameter of multi-loss fusion, and the hyperparameter of multi-loss fusion Used to measure the contribution of the discretized head pose classification task and the continuous head pose regression task to the total loss.

[0136] like Figure 5 As shown, in step F, a threshold is set based on the passenger head posture detection result and the passenger's concentration level on the promotional poster is determined, and then the display content of the promotional poster is modified according to the passenger's concentration level on the promotional poster. If the head posture is within the set threshold range, it is determined that the passenger is concentrating his attention on the promotional poster.

[0137] like Figure 6 As shown, a bus advertising placement and correction system based on a large model and machine vision includes a passenger age detection module, a passenger height detection module, a passenger gender classification detection module, an advertising placement display module, a head posture detection module and an advertising correction module;

[0138] The passenger age group detection module is used to detect the passenger age group by using the RBF neural network-decision level fusion algorithm optimized based on the adaptive sparrow search algorithm to obtain the passenger age group detection result;

[0139] The passenger height detection module is used to detect the passenger height by using a BP neural network algorithm optimized based on the beetle whisker algorithm to obtain the passenger height detection result;

[0140] The passenger gender classification detection module is used to classify and detect the passenger gender using an SVM algorithm optimized based on a hybrid frog leaping algorithm with simulated annealing, to obtain a passenger gender classification detection result;

[0141] The advertising display module is used to generate advertising copy and poster keywords based on the passenger age group detection results, the passenger height detection results and the passenger gender classification detection results in combination with the promotional product library, and then make a promotional poster from the advertising copy and poster keywords and display it on the bus display;

[0142] The head posture detection module is used to detect the passenger's head posture and obtain the passenger's head posture detection result;

[0143] The advertisement correction module is used to set a threshold based on the passenger head posture detection result and determine the passenger's concentration level on the promotional poster, and then correct the display content of the promotional poster according to the passenger's concentration level on the promotional poster.

[0144] Specifically, the adaptive sparrow search algorithm in the passenger age group detection module is used to optimize the center value, width value, weight value and age group accuracy of the RBF neural network.

[0145] Specifically, the optimized BP neural network algorithm in the passenger height detection module uses the two-dimensional image to fit the three-dimensional world coordinates and fits the key points detected by the human skeleton points, thereby detecting the three-dimensional visual coordinates of the key skeleton points, and then calculating the length of each body bone segment to estimate the passenger height.

[0146] In order to better illustrate the use effect of the present invention, a specific embodiment of the present invention is introduced below.

[0147] In this embodiment, cameras and display hardware devices are installed on the public bus for visual acquisition and advertisement display. After applying the public bus advertisement placement and correction method and system of the present invention, the number of advertisement displays increased by 42%, the average passenger gaze time increased by 37%, and the sales of advertisement products increased by 65%. It can be seen that the present invention effectively improves the accuracy of public bus advertisement placement and the marketing conversion rate, and has good application prospects.

[0148] In summary, the present invention is a method and system for placing and correcting public transportation advertisements based on a large model and machine vision. First, an RBF neural network-decision-level fusion algorithm optimized based on an adaptive sparrow search algorithm is used to detect the age group of passengers and obtain the detection results of the age group of passengers. Then, a BP neural network algorithm optimized based on a longicorn beard algorithm is used to detect the height of passengers and obtain the detection results of the height of passengers. Then, an SVM algorithm optimized based on a hybrid frog leaping algorithm with simulated annealing is used to classify and detect the gender of passengers and obtain the detection results of the gender classification of passengers. Subsequently, based on the detection results of the age group of passengers, the detection results of the height of passengers and the detection results of the gender classification of passengers, an advertisement text and poster keywords are generated in combination with a promotional product library. Then, a promotional poster is made from the advertisement text and poster keywords and placed and displayed on a bus display. Then, the head posture of the passenger is detected and the detection results of the head posture of the passenger are obtained. Then, a threshold is set based on the detection results of the head posture of the passenger and the judgment of the passenger is made. The present invention can detect the concentration of passengers on the promotional posters, and then modify the displayed content of the promotional posters according to the concentration of passengers on the promotional posters; the present invention can effectively detect the relevant characteristics of passengers and adopt different innovative algorithms for age groups, genders and heights, which not only improves the detection effect, but also improves the practicality of the algorithm. It can also realize the automatic generation of advertising copy and posters based on passenger characteristics and product information using large model technology, without the need for manual design and review, which greatly improves the efficiency of advertising generation and reduces labor costs. It can also realize real-time verification of the effects of advertising copy and posters through the degree of attention, and advertising copy and posters with poor effects will be automatically updated. The whole process does not require manual detection, judgment and updating, which greatly improves the quality of advertising and the efficiency of update feedback, and improves the travel experience of passengers.

[0149] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for placing and correcting public transportation advertisements based on a large model and machine vision, characterized in that: The following steps are included: Step A, using the RBF neural network-decision-level fusion algorithm optimized by the adaptive sparrow search algorithm to detect the age of passengers, and obtaining the detection results of the age of passengers, wherein the input variable is the feature data after the face data is reduced in dimension by using LGBP, and the adaptive crossover operation and mutation operation in the adaptive sparrow search algorithm are used to optimize the center value, width value, weight and age accuracy of the RBF neural network, and the adaptive crossover operation and mutation operation in the adaptive sparrow search algorithm are specifically to sort the fitness of individual sparrows in the process of finding food and following reconnaissance and early warning of the sparrow population, and select the sparrow individual with the best fitness to perform crossover operation and mutation operation, and the specific steps are as follows, Step A1, crossover operation, select sparrow individuals in the population and randomly pair them with each other, then cross the paired sparrow individuals through the crossover operation probability P C A crossover operation occurs, where the two sparrows X of the crossover operation i and X j The crossover formula is shown in formula (1). in, Indicates Sparrow X i Location, Indicates Sparrow X j Location, Indicates Sparrow X i Position after crossover operation, Indicates Sparrow X j The position after the crossover operation, γ represents the credibility, and β represents a random number between the interval (0, 1); Step A2: mutation operation, the position of the sparrow individual is mutated with probability P m Mutation occurs, and the individual optimal value P is set i The d-th dimension variable is P i d , and then P i d Applying random disturbance variables to perform mutation operations is shown in formula (2): P i d =P i d ·(1+μ) (2) Among them, μ is a random disturbance variable and obeys a normal distribution with a mean of 0 and a variance of 1; Step A3, adaptive crossover operation and mutation operation, using adaptive crossover operation and mutation operation to make the crossover operation probability P C and mutation operation probability P m Can be adjusted automatically, as shown in formula (3) and formula (4), Among them, P c1 and P m1 is a constant, f1 is the average fitness of the entire population, f2 is the maximum fitness of the entire population, f3 is the minimum fitness of the entire population, and fitness(1) is the global optimal fitness; Step A4, compare the fitness of offspring with that of the parent generation and eliminate individuals with poor fitness; Step A5, using RBF neural network to establish an age group determination model, and then using the adaptive crossover operation and mutation operation in the adaptive sparrow search algorithm to optimize the center value, width value, and weight value of the RBF neural network; Step A6, use the KPCA algorithm to reduce the feature dimension of the original face image data, and then perform decision-level fusion on the reduced-dimensional data based on the confidence of the result obtained by the SVM algorithm to obtain the passenger age group detection result. The specific steps are as follows: Step A61: Use KPCA to extract the features m from the face data. KPCA Use the SVM algorithm for classification and get the feature m KPCA The probability of belonging to the i-th category is P i KPCA , and then the probability P i KPCA The probabilities of sorting by size and getting the top three are and Among them, r1, r2 and r3 are the top three categories in probability, and the credibility γ KPCA As shown in formula (5), Step A62: Use LGBP to extract the feature m from the face data. KPCA Use adaptive sparrow search algorithm and RBF neural network for classification and obtain credibility γ LGBP ; Step A63, the credibility γ KPCA and credibility LGBP The weights of the weighted fusion algorithm are calculated and obtained as shown in formula (6): Among them, ω KPCA and ω LGBP They represent the weights calculated based on the SVM algorithm and the weights obtained based on the fusion of the adaptive sparrow search algorithm and the RBF neural network respectively; Step A64, calculate the probability f that the fused image belongs to the i-th category i As shown in formula (7), the passenger age group detection result is max(f i ) represents the age group, Step B, using the BP neural network algorithm optimized by the beetle whisker algorithm to detect the passenger's height and obtain the passenger's height detection result, wherein the optimized BP neural network algorithm uses the two-dimensional image to fit the three-dimensional world coordinates and fits the key points detected by the human skeleton points, thereby detecting the three-dimensional visual coordinates of the key skeleton points, and then calculating the length of each body bone segment to estimate the passenger's height. The specific steps are as follows: Step B1, set the random vector of the beetle's whisker direction and define the spatial dimension k, then set the model structure to MS-3, where M is the number of neurons in the input layer, S is the number of neurons in the hidden layer, 3 is the number of neurons in the output layer, and the search space dimension Z = M*S+S*3+S+3; Step B2, setting the step length factor δ, wherein the step length factor δ is used to control the regional search capability of the longhorn beetle, as shown in formula (8), δ t+1 =δ t *eta,t=(0,1,2,...,n) (8) Among them, eta takes a number close to 1 between [0, 1]; Step B3, using the root mean square error MSE of the test data as a fitness evaluation function, and the fitness evaluation function is used to promote the search of the spatial area, as shown in formula (9), Among them, fitness represents fitness. The position where the fitness function value is the smallest when the algorithm iteration stops is the optimal solution. N is the number of samples in the training set, t sim(q) is the model output value of the qth sample, y q is the actual value of the qth sample; Step B3, initializing the position of the longhorn beetle, wherein the initial parameter of the longhorn beetle position takes a random number between [-0.25, 0.75] as the initial solution set of the longhorn beetle whisker algorithm, and saves it in bestX; Step B4, calculating the fitness function value at the initial position according to the fitness function and saving it in bestY; Step B5, updating the positions of the left and right whiskers of the longhorn beetle, wherein specifically the position coordinates of the left and right whiskers of the longhorn beetle are updated according to step B2; Step B6, according to the left and right whisker positions in the beetle whisker algorithm, the fitness function values ​​f(x r ) and f(x l ), and then compare f(x r ) and f(x l ) and updates the position of the longicorn according to formula (8), thereby adjusting the weights and thresholds of the BP neural network, and then calculating the fitness function value at the current position. If the fitness function value at this time is better than bestY, then update bestY and bestX; Step B7, determine whether the fitness function value reaches the set accuracy or the iteration reaches the maximum number of times, if the condition is met, proceed to step B8, if not, return to step B5 to continue iteration; Step B8, taking the solution in bestX as the optimal solution, thereby obtaining the optimal initial weight and threshold of the BP neural network, and then bringing the optimal solution into the BP neural network for secondary training and learning, thereby forming a three-dimensional coordinate prediction model of key skeleton points, and then obtaining the passenger height detection result; Step C, using the SVM algorithm optimized by the hybrid frog leaping algorithm with simulated annealing to classify and detect the gender of passengers, and obtaining the passenger gender classification detection results. The specific steps are as follows: Step C1, initialize the frog population, set the individual initial position to be a random number between [0, C′], C′ is the total number of individuals in the population, set the individual dimension m, the number of subpopulations M, the number of subpopulation local search iterations num, the initial annealing temperature T, the temperature cooling coefficient C and the disturbance amplitude η; Step C2: Calculate the fitness value of each individual. If the SVM hyperparameter combination of a certain individual position does not meet the fitness value setting requirements, the fitness value of the individual is defined as 1x10 8 , otherwise keep the fitness value unchanged; Step C3, sort the fitness values ​​of all current individuals from best to worst and divide the individuals into sub-populations in turn; Step C4, using the hybrid frog leaping algorithm to update the worst individual of the subpopulation and obtain a better solution x; Step C5, using simulated annealing algorithm to calculate, as shown in formula (10), x'=x+ηξ (10) Among them, ξ is a random variable, and the random variable ξ is generated by normal distribution, x' is a new solution, and the quality of x and x' is compared by annealing mechanism after the new solution x'; Step C6, sort the sub-population fitness values, then return to step C4 and repeat the update until the number of iterations in the sub-population meets the given requirement num; Step C7, after all sub-populations have completed the update operation, if the current optimal individual meets the convergence condition, the evolution process ends successfully and returns to the global optimal solution, thereby obtaining the passenger gender classification detection result; otherwise, the annealing temperature of the population is modified to be the product of the initial annealing temperature T and the temperature cooling coefficient C; Step D, generating advertising copy and poster keywords based on the passenger age group detection results, the passenger height detection results and the passenger gender classification detection results in combination with the promotional product library, and then making a promotional poster from the advertising copy and poster keywords and displaying it on the bus display; Step E, detecting the passenger's head posture and obtaining the passenger's head posture detection result; Step F, setting a threshold based on the passenger head posture detection result and determining the passenger's concentration level on the promotional poster, and then modifying the display content of the promotional poster according to the passenger's concentration level on the promotional poster.

2. The method for placing and correcting public transportation advertisements based on large models and machine vision according to claim 1 is characterized in that: Step D, based on the passenger age detection results, passenger height detection results and passenger gender classification detection results combined with the promotional product library to generate advertising copy and poster keywords, and then make a promotional poster from the advertising copy and poster keywords and display it on the bus display. The specific steps are as follows: Step D1, taking the mode of the passenger age group test results, the mode of the passenger gender classification test results and the mean of the passenger height test results, and generating advertising copy and poster keywords in the order of the mode of the passenger age group test results, the mode of the passenger gender classification test results and the mean of the passenger height test results in combination with the promotional product library; Step D2, generating a poster with the advertisement copy and poster keywords and displaying it on a bus-mounted display.

3. The method for placing and correcting public transportation advertisements based on large models and machine vision according to claim 2 is characterized in that: Step E: Detect the passenger's head posture and obtain the passenger's head posture detection result. Specifically, the passenger's head posture detection is to detect the continuous head deflection angle, and then discretize the head deflection angle and use a multi-output fusion loss method to achieve step-by-step detection from coarse to fine. The specific steps are as follows: Step E1, classify the 181 continuous angle values ​​of the head posture angle from -90° to 90° according to the angle interval of 3°, then mark each posture angle with a classification label and use the category error rate loss function to perform classification prediction to obtain the head posture angle classification prediction value; Step E2, mapping the head posture angle classification prediction value to the original continuous regression angle interval, and then using the smoothed mean absolute error loss function to predict the angle regression; Step E3, the class error rate loss function and the smoothed mean absolute error loss function are weighted and summed with set weights as the final objective function of the network, where the calculation formula of the final objective function of the network is shown in formula (11): L total =L CER +εL S-MAE (11) Among them, L total Represents the total loss function, L CER represents the category error rate loss function, L S-MAE represents the smoothed mean absolute error loss function, ε is a hyperparameter of multi-loss fusion, and the hyperparameter ε of multi-loss fusion is used to measure the contribution of the discretized head pose classification task and the continuous head pose regression task to the total loss.

4. The method for placing and correcting public transportation advertisements based on a large model and machine vision according to claim 3 is characterized by: Step F, based on the passenger head posture detection result, a threshold is set and the passenger's concentration on the promotional poster is determined, and then the displayed content of the promotional poster is modified according to the passenger's concentration on the promotional poster. If the head posture is within the set threshold range, it is determined that the passenger is concentrating his attention on the promotional poster.

5. A bus advertisement placement and correction system based on a large model and machine vision, wherein the operation process of the bus advertisement placement and correction system is based on the bus advertisement placement and correction method according to any one of claims 1 to 4, and is characterized in that: It includes a passenger age detection module, a passenger height detection module, a passenger gender classification detection module, an advertisement placement display module, a head posture detection module and an advertisement correction module; The passenger age group detection module is used to detect the passenger age group by using the RBF neural network-decision level fusion algorithm optimized based on the adaptive sparrow search algorithm to obtain the passenger age group detection result; The passenger height detection module is used to detect the passenger height by using a BP neural network algorithm optimized based on the beetle whisker algorithm to obtain the passenger height detection result; The passenger gender classification detection module is used to classify and detect the passenger gender using an SVM algorithm optimized based on a hybrid frog leaping algorithm with simulated annealing, to obtain a passenger gender classification detection result; The advertising display module is used to generate advertising copy and poster keywords based on the passenger age group detection results, the passenger height detection results and the passenger gender classification detection results in combination with the promotional product library, and then make a promotional poster from the advertising copy and poster keywords and display it on the bus display; The head posture detection module is used to detect the passenger's head posture and obtain the passenger's head posture detection result; The advertisement correction module is used to set a threshold based on the passenger head posture detection result and determine the passenger's concentration level on the promotional poster, and then correct the display content of the promotional poster according to the passenger's concentration level on the promotional poster.

6. The bus advertisement placement and correction system based on large model and machine vision according to claim 5 is characterized by: The adaptive sparrow search algorithm in the passenger age group detection module is used to optimize the center value, width value, weight value and age group accuracy of the RBF neural network.

7. The bus advertisement placement and correction system based on large model and machine vision according to claim 5 is characterized by: The optimized BP neural network algorithm in the passenger height detection module uses a two-dimensional image to fit the three-dimensional world coordinates and fits the key points detected by the human skeleton points, thereby detecting the three-dimensional visual coordinates of the key skeleton points, and then calculating the length of each body bone segment to estimate the passenger height.

Citation Information

Patent Citations

  • Method and system for intelligent push of individualized advertisement of retail store

    CN107798560A

  • Advertisement putting method and device

    CN108540853A

  • Game advertisement generation method and system and putting method and system thereof

    CN110969484A

  • Advertisement putting optimization method and device based on big data

    CN112396471A

  • Sight line direction estimation method based on deep learning

    CN112800815A