An automatic generation method based on a perception intensity research and evaluation model

Abstract

The present invention is an automatic generation method based on a perception intensity research and evaluation model, which includes the following steps: Step S1: Obtain a research database; Step S2: Regularly access the research database according to the project ID and query the project research data to evaluate whether the data quality meets the conditions for entering the automatic generation stage of the algorithm; Step S3: After the research data quality meets the conditions for entering the automatic generation stage of the algorithm described in Step S2), calculate the perception score of each picture to form picture perception intensity data; Step S4: Train the picture perception intensity data using a deep learning network model to obtain an automatic evaluation model for the research question. The present invention uses research data to automatically generate a perception intensity evaluation algorithm, realizes the quantitative evaluation of the research dimension, enables the research to more accurately reflect the concepts of the subjects, and provides an analysis basis for the application and expansion of the later research results.

Description

Technical Field

[0001] The present invention relates to multiple technical fields such as questionnaire research and algorithm research and development, and particularly relates to an automatic generation method based on a perception intensity research and evaluation model.

Background Art

[0002] Currently, the market scales of domestic and foreign market researches have maintained a high growth trend. According to the report of the European Society for Opinion and Marketing Research (ESOMAR), the turnover of the Chinese market research industry has also maintained a long-term growth trend and reached 27.00 billion yuan in 2019. At the same time, the research and evaluation of public service policies and the procurement volume of data services are roughly equivalent to 6.54% of the total service procurement volume of the above ministries and commissions, and have reached approximately 93 billion yuan in 2019, with great market expansion space in the future.

[0003] In the prior art, for the results after questionnaire surveys, generally, the survey data is manually input, and then the survey results are statistically analyzed using an excel table, and the data analysis of the survey results is carried out in the form of charts. However, the vast majority are qualitative analyses, and the evaluation of the survey results cannot be automatically evaluated, nor can an evaluation model be formed to automatically evaluate new survey objects.

[0004] Therefore, it is necessary to provide a new automatic generation method based on a perception intensity research and evaluation model to solve the above technical problems.

Summary of the Invention

[0005] The main purpose of the present invention is to provide an automatic generation method based on a perception intensity research and evaluation model, which realizes the automatic analysis and automatic evaluation of survey data, forms an automatic evaluation model for set survey questions, and can be further applied to the automatic evaluation and scoring of the survey questions to achieve quantitative evaluation.

[0006] The present invention realizes the above object through the following technical solutions: An automatic generation method based on a perception intensity research and evaluation model, which includes the following steps:

[0007] Step S1: Obtain a research database, where the research data in the research database includes project ID, research questions, a set of research pictures, and the comparison times of each picture in the set of research pictures;

[0008] Step S2: Regularly access the research database according to the project ID and query the project research data to evaluate whether the data quality meets the conditions for entering the automatic generation stage of the algorithm;

[0009] Step S3: After the research data quality meets the conditions for entering the algorithm automatic generation stage described in step S2), calculate the perceived intensity score of each picture, which represents the perceived intensity of the picture on the specific dimension involved in the research question, and form the picture perceived intensity data;

[0010] Step S4: Train the picture perceived intensity data using a deep learning network model to obtain an automatic evaluation model for the research question.

[0011] Furthermore, the method for forming the research database in step S1 includes:

[0012] S11) Obtain the research questions for the research questionnaire to be generated;

[0013] S12) Use the web crawler method to obtain a set of research pictures;

[0014] S13) Generate a research questionnaire using the research questions described in step S11) and the set of research pictures described in step S12), and present two pictures for the user to choose each time in a two-choice manner, and finally complete the research data collection work to form a research database.

[0015] Furthermore, the condition for entering the algorithm automatic generation stage in step S2) is: In the set of research pictures, count the number of times each picture is compared with other different pictures, and require that the average number of comparisons of all pictures is greater than a set number, generally at least more than 10 times, such as 10 times, 15 times, or 20 times, etc.

[0016] Furthermore, the calculation method of the perceived intensity in step S3) includes:

[0017] S31) Calculate the selection probability P of each picture i and the non-selection probability N of the picture i ;

[0018]

[0019]

[0020] For picture i, p i refers to the number of times the picture is selected in the two-choice comparison, n i refers to the number of times the picture is not selected in the comparison, e i refers to the number of times the picture cannot be selected in the comparison;

[0021] S32) Calculate the perceived intensity score Q of the picture according to the selection probability P i and the non-selection probability N of the picture i : i :

[0022]

[0023] Among them,

[0024]

[0025]

[0026] Among them, k1 is the number of times picture i is selected in the comparison, and k2 is the number of times picture i is not selected in the comparison. In addition, by adding a constant 1 and multiplying by a constant The value range of Q-score is made to be 0 to 10.

[0027] Furthermore, step S4) includes the following steps:

[0028] S41) Statistically count the number of pictures in segments for the perceived intensity scores Q-score (0 to 10 points) obtained in step S3);

[0029] S42) For the pictures and their perceived intensity scores in each segment, divide them into a training data set and a test data set according to a set ratio; for example, randomly select 80% of them as the training data set and 20% as the test data set;

[0030] S43) Train the pictures in the training data set in batches using the DenseNet network; specifically, first establish the picture perceived intensity classification network structure, put the pictures and their corresponding perceived intensity scores into the network for training, and calculate the network loss function value through the mean square error formula; feedback forward through the optimizer and adjust the network, and recalculate the loss function value to make the loss function gradually decrease to the minimum and save the model with the minimum loss function;

[0031] S44) Calculate the topk accuracy of the model;

[0032] S45) Repeat steps S43)-S44), adjust the learning rate, loss function, and batch size parameters until the topk accuracy in step S44) meets the set value to obtain the trained model; for example, the topk accuracy meets top1>50% and top2>85% (5-classification), or top1>30% and top3>80% (10-classification);

[0033] S46) Input the pictures in the test data set in step S42) into the trained model in step S45) and output the predicted perceived intensity scores of the pictures.

[0034] Furthermore, step S41) includes:

[0035] S411) Normalize the perceived intensity score data:

[0036] z = (Q - u) / s

[0037] Among them, z is the normalized value, u is the average of the overall data, s is the variance of the overall data, and Q is the perceived intensity score;

[0038] S412) Uniformly divide all the normalized perceived intensity scores into M sections in ascending order, count the number of pictures in each section, and calculate the mean of the perceived intensity scores in each section; for example, using integers 1, 2, 3, 4, 5, 6, 7, 8, 9 as the intermediate breakpoints, divide all the perceived intensity scores into 10 sections; M can take values such as 15, 20, 30, etc., and is flexibly set according to the data sample size;

[0039] S413) If the number of pictures with perceived intensity scores less than 1 and the number of pictures with perceived intensity scores greater than 9 are both greater than 10% of the total number of pictures, then confirm to use M - segment segmentation;

[0040] S414) If the number of pictures with perceived intensity scores less than 1 or the number of pictures with perceived intensity scores greater than 9 is less than 10% of the total number of pictures, then uniformly re - divide the perceived intensity scores into M / 2 sections in ascending order, re - count the number of pictures in each section, and calculate the mean of the perceived intensity scores in each section. For example, re - divide all the perceived intensity scores into 5 sections at integers 2, 4, 6, 8.

[0041] Furthermore, in step S46), take the inner product of each classification probability result calculated in the DenseNet network with the mean value of the corresponding section in step S412) or step S414) to obtain the predicted perceived intensity score for the pictures in the test dataset. This step takes the inner product of each classification probability result calculated by the model with the mean value of the corresponding section, and the obtained average weighted value is used as the predicted perceived intensity score. Considering each perceived intensity section, it effectively avoids extreme situations and makes the predicted score more scientific and reasonable.

[0042] Compared with the prior art, the beneficial effects of an automatic generation method of a perception intensity research and evaluation model of the present invention are as follows: Using the forced - choice method based on picture two - choice to obtain research data for automatically generating a perception intensity evaluation algorithm, realizing the quantitative evaluation of the research dimension; on the one hand, forming a standardized and automated process, simplifying the later operation of research data analysis, and on the other hand, building a quantitative calculation model of perception intensity, enabling the research to more accurately reflect the concepts of the subjects, and providing an analysis basis for the application and expansion of the later research results. Specifically:

[0043] 1) Conduct research based on the forced-choice method of choosing one out of two to obtain a research database for specific research questions. Utilize the research pictures in the research database and the number of comparison times for each research picture during the research process. Combine with the calculation of the perceived intensity of each research picture to obtain perceived intensity data. Finally, use the perceived intensity data as the input for model training, and obtain an automatic evaluation model for the research question based on a deep learning network, realizing the quantitative evaluation of the research dimension;

[0044] 2) After obtaining the set of research pictures in the research data, by limiting the number of picture comparisons as the basis for judging whether a picture should enter subsequent model training, retain the pictures with more comparison times, and delete the pictures with fewer comparison times that have no training value, greatly improving the effectiveness of model training, and thus improving the accuracy of the subsequent evaluation model;

[0045] 3) After obtaining the perceived intensity data, normalize the data and divide it into sections. Then divide the data into 10 groups according to the perceived intensity score and the corresponding number of pictures, eliminating the influence of the mean distribution of perceived intensity on the accuracy of the evaluation model, and providing a quantitative reference basis for scoring subsequent test pictures.

Description of the Drawings

[0046] Figure 1 It is the flowchart of the method of the embodiment of the present invention;

[0047] Figure 2 It is the frequency histogram of the distribution of the number of comparison times of the embodiment of the present invention;

[0048] Figure 3 It is the frequency histogram of the distribution of the perceived intensity scores of the embodiment of the present invention;

[0049] Figure 4 It is the frequency histogram of the distribution of the perceived intensity scores of the 2-section, 3-section, 5-section and 10-section of the road drainage outlet in the embodiment of the present invention;

[0050] Figure 5 It is the frequency histogram of the distribution of the perceived intensity scores of the 2-section, 3-section, 5-section and 10-section of the road railing in the embodiment of the present invention;

[0051] Figure 6 It is the frequency histogram of the distribution of the perceived intensity scores of the 2-section, 3-section, 5-section and 10-section of the road manhole cover in the embodiment of the present invention;

[0052] Figure 7 It is the frequency histogram of the distribution of the perceived intensity scores of the 2-section, 3-section, 5-section and 10-section of the road surface in the embodiment of the present invention.

Detailed Embodiments

[0053] Embodiment 1:

[0054] Please refer to Figure 1 , this embodiment is an automatic generation method based on a perception intensity research and evaluation model, which includes the following steps:

[0055] Step S1: Obtain a research database. In this embodiment, 4 research projects are obtained, and their research questions are respectively "Which road has a good road surface?", "Which road has good manhole covers?", "Which road has good drainage outlets?", "Which road has good guardrails?", and the corresponding research picture sets and the corresponding picture comparison times data are shown in Table 1.

[0056] Table 1

[0057] Investigation Picture Set (Unit: piece) Total Number of Comparisons (Unit: time) Road Pavement 5117 51309 Road Manhole Cover 1022 10328 Road Drainage Outlet 1002 8640 Road Railings 721 7233

[0058] Step S2: Data cleaning. Delete the pictures with the comparison times less than 10 times to obtain the processed picture data, as shown in Table 2.

[0059] Table 2

[0060]

[0061]

[0062] The comparison times corresponding to the above valid pictures are reflected by a histogram, as Figure 2 shown, where the abscissa is the comparison times and the ordinate is the corresponding number of pictures.

[0063] Step S3: Calculate the perception intensity score of each picture, and use the perception intensity score as the abscissa and the number of pictures corresponding to the perception intensity score as the ordinate to generate a histogram as Figure 3 shown.

[0064] Step S4: Normalize all the perception intensity scores, and then evenly divide the pictures into 10 sections according to the highest score and the lowest score, and record the mean value in each section; through the division of 10 sections, the result can be better and more fully mapped into the score interval of (1, 10). The finer the data is divided, the better it can reflect the real situation. In this embodiment, due to the limitation of the data volume, if conditions permit, dividing into 20 sections will be better.

[0065] Then, use the perception intensity score as the abscissa, use a single section as the statistical unit, and use the number of pictures in the corresponding perception intensity score section as the ordinate to generate a 10-section classification histogram. In order to verify that the more classification categories of the sections, the better, in this embodiment, the perception intensity scores are respectively divided into 2 sections, 3 sections and 5 sections as comparative examples, and the corresponding classification histograms are generated at the same time, as Figures 4 - 7 shown.

[0066] Step S5: Randomly divide the pictures in step S4) within each section according to a ratio of 8:2 to obtain an 80% training data set and a 20% test data set. The data set division is shown in Table 3.

[0067] Table 3

[0068] Valid Pictures Training Set Proportion Test Set Proportion Road Pavement 5009 4259 85.03％ 750 14.97％ Road Manhole Cover 1001 851 85.01％ 150 14.99％ Road Drainage Outlet 955 801 83.87％ 150 15.71％ Road Railings 717 567 79.08％ 150 20.92％

[0069] Step S6: Input the training data set in step S5) into the DenseNet network for training, and use the test data set to verify the network, and train an automatic evaluation model based on the set research questions.

[0070] To verify the influence of the number of data set division sections on the accuracy of the algorithm, in this embodiment, the accuracy rates of the models trained by the DenseNet network for the data with the perceived intensity scores divided into 2 sections, 3 sections, 5 sections, and 10 sections are calculated respectively. As shown in Table 4, Table 4 shows the accuracy of the road condition scoring algorithm with different numbers of sections and top-k, as well as the accuracy improvement above the random selection probability.

[0071] Table 4

[0072]

[0073] It can be seen from Table 4 that the fewer the groups, the higher the accuracy rate of the algorithm, and the more the groups, the greater the increase in the accuracy rate of the algorithm above the random selection probability. The selection of top-k also affects the accuracy of the algorithm. Considering the balance between the number of groups / top-k selection and the accuracy, an 8:2 data set division ratio and the use of data with 5 section divisions are determined.

[0074] The accuracy rate results of the final trained automatic evaluation model based on specific research questions are shown in Table 5.

[0075] Table 5

[0076] Accuracy Rate Top 2 Accuracy Rate Road Pavement 68.0％ 81.4％ Road Manhole Cover 64.3％ 94.7％ Road Drainage Outlet 56.0％ 91.3％ Road Railings 50.1％ 80.0％

[0077] It can be seen from Table 5 that:

[0078] 1) Since the number of pictures in the research picture set is large and the number of pairwise comparisons is large, the accuracy rate of the automatic evaluation model obtained by the method in this embodiment for judging the road surface score of the research question "Which road has a good road surface" reaches 68.0%. The reason why the accuracy cannot be further improved is mainly to maintain the generalization of the algorithm, that is, to ensure that a single algorithm can be applied to different types of roads.

[0079] 2) Although road manhole covers and road drainage outlets have approximately the same number of pictures and comparison times, their performances vary greatly. A main reason may be that road manhole covers are easier to compare and do show differences in their conditions. However, more errors were found in the detection of road drainage outlets. Their images are too blurred and similar to be compared, resulting in a relatively low accuracy rate of the algorithm.

[0080] 3) The accuracy rate of the evaluation of road guardrails reached 50.1%, mainly because the number of research pictures and comparison times are relatively small, and the numbers are easier to fit.

[0081] After the automatic evaluation model for the set research questions is obtained in this embodiment, it can be applied. When applying, by inputting a picture, the automatic evaluation model can score the picture. For example, the automatic evaluation model for "which street has a good road" can score the picture regarding street road elements.

[0082] The above are only some embodiments of the present invention. For those of ordinary skill in the art, without departing from the inventive concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention.

Claims

1. An automatic generation method based on a perception intensity research and evaluation model, characterized in that: It includes the following steps: Step S1: Obtain a research database, where the research data in the research database includes project ID, research questions, a set of research pictures, and the number of comparison times for each picture in the set of research pictures; Step S2: Regularly access the research database according to the project ID and query the project research data to evaluate whether the data quality meets the conditions for entering the algorithm automatic generation stage; Step S3: When the research data quality meets the conditions for entering the algorithm automatic generation stage in step S2), calculate the perceived intensity score for each picture to form picture perceived intensity data; the calculation method of the perceived intensity includes: S31) Calculate the selection probability P of each picture i and the non - selection probability N of the picture i ; For picture i, p i refers to the number of times the picture is selected in a two-way comparison, n i refers to the number of times the picture is not selected in the comparison, e i refers to the number of times the picture cannot be selected in the comparison; S32) Calculate the perceived intensity score Q of the picture according to the selection probability P of the picture i and the non-selection probability N of the picture i i :​ where where k1 is the number of times picture i is selected in the comparison, and k2 is the number of times picture i is not selected in the comparison; Step S4: Use a deep learning network model to train the picture perceived intensity data to obtain an automatic evaluation model for the research questions; step S4) includes the following steps: S41) Statistically count the number of pictures in segments for the perceived intensity scores obtained in step S3); S42) For the pictures and their perceived intensity scores in each segment, divide them into a training data set and a test data set according to a set ratio; S43) Use the DenseNet network to train the pictures in the training data set in batches; S44) Calculate the topk accuracy of the model; S45) Repeat steps S43)-S44), adjust the learning rate, loss function, and batch size parameters until the topk accuracy in step S44) meets the set value to obtain a trained model; S46) Input the pictures in the test data set in step S42) into the trained model in step S45) and output the predicted perceived intensity scores of the pictures.

2. The automatic generation method based on the perception intensity research and evaluation model according to claim 1, characterized in that: The formation method of the research database in step S1) includes: S11) Obtain the research questions for the research questionnaire to be generated; S12) Use the web crawler method to obtain a set of research pictures; S13) Use the research questions in step S11) and the set of research pictures in step S12) to generate a research questionnaire, and present two pictures for the user to choose each time using the forced-choice method of choosing one of two, and finally complete the research data collection work to form a research database.

3. The automatic generation method based on the perception intensity research and evaluation model according to claim 1, characterized in that: The condition for entering the algorithm automatic generation stage in step S2) is: In the set of research pictures, count the number of comparisons of each picture with other different pictures, and require the average number of comparisons of all pictures to be greater than the set number.

4. The automatic generation method based on the perception intensity research and evaluation model according to claim 1, characterized in that: Step S43) includes: First, establish a picture perceived intensity classification network structure, put the pictures and their corresponding perceived intensity scores into the network for training, and calculate the network loss function value through the mean square error formula; feedback forward through the optimizer and adjust the network, and recalculate the loss function value to make the loss function gradually decrease to the minimum and save the model with the minimum loss function.

5. The automatic generation method based on the perception intensity research and evaluation model according to claim 1, characterized in that: Step S41) includes: S411) Normalize the perceived intensity score data: z = (Q - u) / s where z is the normalized value, u is the overall data average, s is the overall data variance, and Q is the perceived intensity score; S412) Uniformly divide all the normalized perceived intensity scores into M sections in ascending order, count the number of pictures in each section, and calculate the mean of the perceived intensity scores in each section; S413) If the number of pictures with perceived intensity scores less than 1 and the number of pictures with perceived intensity scores greater than 9 are both greater than 10% of the total number of pictures, then confirm the adoption of M-segment segmentation; S414) If the number of pictures with perceived intensity scores less than 1 or the number of pictures with perceived intensity scores greater than 9 is less than 10% of the total number of pictures, then uniformly re-divide the perceived intensity scores into M / 2 sections in ascending order, re-count the number of pictures in each section, and calculate the mean of the perceived intensity scores in each section.

6. The automatic generation method based on the perception intensity research and evaluation model according to claim 5, wherein: The step S46) includes: taking the inner product of each classification probability result calculated in the DenseNet network with the mean value of the corresponding section in step S412) or step S414) to obtain the predicted perceived intensity score of the pictures in the test dataset.

Images