Intelligent recommendation system and method based on user behavior prediction in complex environment

Abstract

The invention relates to the technical field of static graphic data retrieval, and discloses an intelligent recommendation system and method based on complex environment user behavior prediction. The intelligent recommendation system comprises a user behavior recognition module, a user environment recognition module, a user emotion recognition module, a filter library statistical retrieval module, a data analysis module and an intelligent recommendation module. The data analysis module comprises a user behavior comprehensive evaluation unit and a user environment data evaluation unit. According to the system, static image data uploaded by a user are analyzed, main factors in a complex environment where the user is located are extracted, a user behavior filter is determined according to the motion amplitude of a figure in the static image data and the emotional state of an overall image, an environment filter and a user action behavior filter are combined, and the user behavior filter is obtained. According to the method, the intelligent recommendation filter is generated, the intelligent recommendation filter is applied to the static image data uploaded by the user, and the corresponding filter is intelligently recommended by integrating main factors in a complex environment, facial emotion of the user and behavior actions of the user.

Description

Technical Field

[0001] The present invention relates to the technical field of static graphic data retrieval, and in particular to an intelligent recommendation system and method based on complex environment user behavior prediction. Background Art

[0002] When shooting with a mobile terminal, many people add filters to the photos or videos they take in order to improve the quality of the photos or videos.

[0003] In the related art, the object selects the filter to be added from the filter recommendation list. However, the arrangement order of the filters in the filter recommendation list is customized by manual operation in the background. However, due to the subjective characteristics of manual operation and the personalized characteristics of the object, the recommendation results of the existing filter recommendation method cannot meet the needs of the object, that is, it cannot objectively recommend corresponding filters according to the complex environment of the user, the user's facial emotions and the user's behavioral actions when different scene requirements are met. Summary of the invention

[0004] 1. Technical issues to be solved

[0005] In view of the deficiencies of the prior art, the present invention provides an intelligent recommendation system and method based on user behavior prediction in complex environments, which has the advantages of accurately recommending filters to users and solves the above-mentioned technical problems.

[0006] (II) Technical solution

[0007] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: an intelligent recommendation system based on complex environment user behavior prediction, comprising a user behavior recognition module, a user environment recognition module, a user emotion recognition module, a filter library statistical retrieval module, a data analysis module and an intelligent recommendation module;

[0008] The user behavior recognition module recognizes and evaluates the user's behavior in the static image based on the static image data uploaded by the user, and outputs the user action behavior coefficient DZDF;

[0009] The user environment module identifies and evaluates the user's environment in the static image based on the static image data uploaded by the user, and outputs the user's environment data set;

[0010] The user emotion recognition module recognizes the user's emotion in the static image based on the static image data uploaded by the user, and inputs it into the emotion rating function for calculation, and outputs the user's emotion behavior coefficient QGDF. The specific expression of the emotion rating function is as follows:

[0011]

[0012] Wherein, QXPJ represents the emotion rating function, QXPJ=1 represents strong emotion, QXPJ=0.5 represents fluctuating emotion, QXPJ=0.25 represents neutral emotion, the strong emotion means that the emotion recognized by the user emotion recognition module is anger, disgust, fear, joy and sadness, the neutral emotion means that the emotion recognized by the user emotion recognition module is surprise, and the neutral emotion means that the emotion recognized by the user emotion recognition module is neutral;

[0013] The filter library statistics retrieval module stores a filter library, the filter library statistics retrieval module builds an environment filter library based on the application environment of the filters in the filter library, and the filter library statistics retrieval module builds an action filter library based on the application character actions of the filters in the filter library;

[0014] The data analysis module includes a user behavior comprehensive evaluation unit and a user environment data evaluation unit, wherein the user behavior comprehensive evaluation unit outputs a user behavior filter based on the user action behavior coefficient DZDF and the user emotion behavior coefficient QGDF in combination with the action filter library;

[0015] The user environment data evaluation unit reads the application scenario of each filter in the environment filter library in the filter library statistical retrieval module, and outputs the environment filter based on the user's environment data set;

[0016] The intelligent recommendation module performs calculations based on user behavior filters and environment filters, and outputs intelligent recommendation filters.

[0017] As a preferred technical solution of the present invention, the user behavior recognition module recognizes and evaluates the user's behavior in the static image based on the static image data uploaded by the user, and the specific steps of outputting the user action behavior coefficient DZDF are as follows:

[0018] Step A1: Read the number of characters in the static image data uploaded by the user, and calculate the character interaction coefficient RWHD based on the number of characters. The specific expression is as follows:

[0019]

[0020] Where, e represents the natural constant, a represents the correction coefficient, and RS represents the number of characters in the static image data. Indicates rounding up;

[0021] Step A2: reading the motion state of the character in the static image data, and calculating the character motion coefficient RWYD according to the different motion states of the characters and the number of characters, wherein the motion state evaluation includes the arm motion amplitude SBDZFD and the leg motion amplitude TBDZFD, and the specific steps of determining the arm motion amplitude SBDZFD and the leg motion amplitude TBDZFD are as follows:

[0022] Step A2.a1: Use Openpose to identify the coordinates of the wrist and ankle positions, as well as the position of the central axis of the human body;

[0023] Step A2.a1: Calculate the minimum distance between any wrist position coordinate and any ankle position coordinate and the position of the central axis of the human body, and use them as the arm movement range SBDZFD and the leg movement range TBDZFD respectively;

[0024] Step A3: Calculate the user action coefficient DZDF based on the character interaction coefficient RWHD obtained in step A1 and the character motion coefficient RWYD obtained in step A2. The specific expression is as follows:

[0025] DZDF=RWHD*RWYD

[0026] Among them, RWHD represents the character interaction coefficient, RWYD represents the character motion coefficient, and DZDF represents the user action behavior coefficient.

[0027] As a preferred technical solution of the present invention, when the character motion coefficient RWYD is calculated according to the character motion state in step A2, when the number of characters in the static image data is one, its specific expression is as follows:

[0028] RWYD=w1*SBDZFD+w2*TBDZFD

[0029] Among them, w1 and w2 are two weight coefficients, SBDZFD represents the arm movement range, and TBDZFD represents the leg movement range.

[0030] As a preferred technical solution of the present invention, when calculating the character motion coefficient RWYD according to the character motion state in step A2, when there are multiple characters in the static image data, the specific steps are as follows:

[0031] Step A2.b1: Segment the characters in the static image data into I equal parts according to the number of people;

[0032] Step A2.b2: Calculate the arm motion amplitude SBDZFD and leg motion amplitude TBDZFD of each character, and comprehensively obtain the character motion coefficient RWYD of the entire image. The specific expression is as follows:

[0033]

[0034] Among them, w1 and w2 are two weight coefficients, SBDZFD i represents the arm movement amplitude of the i-th character, It represents the sum of the arm movement amplitudes of a total of I characters, RWYD represents the character movement coefficient, TBDZFD i represents the movement amplitude of the foot of the i-th character, It represents the sum of the leg movement amplitudes of a total of I characters, i∈[1,I].

[0035] As a preferred technical solution of the present invention, the user environment module identifies and evaluates the user's environment in the static image based on the static image data uploaded by the user, and the specific steps of outputting the user environment data set are as follows:

[0036] Step B1: Analyze the environmental data of the static image data and extract all environmental factors in the static image data through the DeepLab network model;

[0037] Step B2: Calculate the proportion of each environmental factor in the static image data, and rank the proportion of all environmental factors in the overall static image data, and select the top three environmental factors as the user's environmental data set. The specific calculation expression is as follows:

[0038]

[0039] Among them, ZB j represents the proportion of the jth environmental factor, XS j represents the total number of pixels of the jth environmental factor, XS max Represents the total number of pixels in the static image data after removing the person.

[0040] As a preferred technical solution of the present invention, when the number of characters in the static image data is one, the user emotion recognition module identifies the user's emotions in the static image based on the static image data uploaded by the user, and the specific process of outputting the user emotion behavior coefficient QGDF is: facial expression recognition is performed through the Pytorch model, and user emotions are output, and the user emotions are placed in the emotion rating function to obtain the user emotion behavior coefficient QGDF.

[0041] As a preferred technical solution of the present invention, when there are multiple characters in the static image data, the user emotion recognition module recognizes the user's emotions in the static image based on the static image data uploaded by the user, and the specific steps of outputting the user's emotional behavior coefficient QGDF are as follows:

[0042] Step C1: Segment the characters in the static image data into I equal parts according to the number of people;

[0043] Step C2: Perform facial expression recognition through the Pytorch model, classify the emotions with the same recognition results of the Pytorch model, and calculate the proportion of each emotion. The specific expression is as follows:

[0044]

[0045] Among them, QX1, ..., QX k ,…,QX K They represent the first emotion, …, the kth emotion, …, the Kth emotion, QXBF1, …, QXBF respectively, which are output and classified by the Pytorch model for expression recognition. k , ..., QXBF K They represent the percentage of the first emotion in all emotions, ..., the percentage of the kth emotion in all emotions, ..., the percentage of the Kth emotion in all emotions, DRQX represents a multi-person emotion dataset, k∈[1, K], K≤I, and the percentage of the kth emotion in all emotions QXBF k The specific expression is as follows:

[0046]

[0047] Among them, QXSL k represents the number of recognized k-th emotion, and K represents the total number of classified emotions;

[0048] Step C3: Determine QXBF1, ..., QXBF in step C2 k , ..., QXBF K Is there a maximum value? If there is a maximum value, select a group of emotions corresponding to the maximum value and the percentage of the emotion in all emotions as the user emotional behavior coefficient QGDF. If QXBF1,…,QXBF k , ..., QXBF K If there are several equal maximum values ​​in , then randomly output an emotion from several equal maximum values ​​and the percentage of the emotion in all emotions;

[0049] Step C4: Input the emotion obtained in step C3 into the emotion rating function for calculation, and output the user emotion behavior coefficient QGDF.

[0050] As a preferred technical solution of the present invention, the specific steps of the user behavior comprehensive evaluation unit outputting the user behavior filter based on the user action behavior coefficient DZDF and the user emotional behavior coefficient QGDF in combination with the action filter library are as follows:

[0051] Step D1: The emotion output by the user emotion behavior coefficient QGDF is combined with the user action behavior coefficient DZDF to obtain the user comprehensive evaluation coefficient, the specific expression of which is as follows:

[0052] ZHPJXS=QGDF*DZDF

[0053] Among them, ZHPJXS represents the user comprehensive evaluation coefficient;

[0054] Step D2: Based on the user comprehensive evaluation coefficient, calculate the closeness coefficient between the comprehensive evaluation coefficient of each filter in the action filter library and the user comprehensive evaluation coefficient, select a filter with the smallest closeness from the action filter library, and store the corresponding contrast, exposure value, sharpness and saturation stored in the filter as the user behavior filter YHLJ. The expression for calculating the closeness coefficient between the comprehensive evaluation coefficient of each filter in the action filter library and the user comprehensive evaluation coefficient is as follows:

[0055] XDJ n =|ZHPJXS-DZK n |

[0056] Among them, DZXS n Indicates the closeness coefficient between the comprehensive evaluation coefficient of the nth filter in the action filter library and the comprehensive evaluation coefficient of the user, DZK n represents the comprehensive evaluation coefficient of the nth filter in the action filter library, and the comprehensive evaluation coefficient DZK of the nth filter in the action filter library n The acquisition method is the same as the acquisition method of the user comprehensive evaluation coefficient ZHPJXS;

[0057] The user environment data evaluation unit reads the application scenario of each filter in the environment filter library in the filter library statistical retrieval module, and outputs the environment filter based on the user's environment data set. The specific steps are: calculating the similarity between the environmental factors of each filter in the environment filter library and the three environmental factors output in step B2, and selecting the contrast, exposure value, sharpness and saturation corresponding to the filter with the greatest similarity to store in the environment filter HJLJ. The specific expression for calculating the similarity between the environmental factors of each filter in the environment filter library and the three environmental factors output in step B2 is as follows:

[0058] HkDJ m =(1-|ZB x -ZB m，x )+(1-|ZB y -ZB m，y |)+(1-|ZB z -ZB m，z |)

[0059] Among them, |*| represents the absolute value, HJXSD m represents the similarity between the mth filter in the environmental filter library and the three environmental factors output in step B2, ZB x , ZBy , ZB z represents the top three environmental factors selected in step B2, ZB m，x Indicates the corresponding ZB in the mth filter usage scenario in the environment filter library x Environmental factors, ZB m，y Indicates the corresponding ZB in the mth filter usage scenario in the environment filter library y Environmental factors, ZB m，z Indicates the corresponding ZB in the mth filter usage scenario in the environment filter library z environmental factors.

[0060] As a preferred technical solution of the present invention, the intelligent recommendation module performs comprehensive analysis based on the user behavior filter and the environment filter, and outputs the specific process of the intelligent recommendation filter as follows: the contrast, exposure value, sharpness and saturation stored in the environment filter HJLJ and the user behavior filter YHLJ are calculated respectively, and the calculation results are used as the contrast, exposure value, sharpness and saturation of the intelligent recommendation filter, and the contrast, exposure value, sharpness and saturation of the intelligent recommendation filter are applied to the static image data uploaded by the user. The specific expression is as follows:

[0061]

[0062] Among them, DBD zz Indicates the contrast of the intelligent recommended filter, DBD HJLJ Represents the contrast stored in the environment filter, DBD YHLJ Indicates the contrast stored in the user behavior filter, BGD zz Indicates the exposure of the smart recommended filter, BGD HJLJ Indicates the exposure stored in the environment filter, BGD YHLJ Represents the exposure stored in the user behavior filter, RD zz Indicates the sharpness of the intelligent recommended filter, RD HJLJ Indicates the sharpness stored in the environment filter, RD YHLJ Indicates the sharpness stored in the user behavior filter, BHD zz Indicates the saturation of the smart recommended filter, BHD HJLJ Indicates the saturation stored in the environment filter, BHD YHLJ Indicates the saturation stored in the user behavior filter.

[0063] The present invention also provides an intelligent recommendation method based on complex environment user behavior prediction, which applies the above-mentioned intelligent recommendation system based on complex environment user behavior prediction, and includes the following steps:

[0064] Step 1: Based on the static image data uploaded by the user, the user's behavior in the static image is identified and evaluated, and the user's action behavior coefficient DZDF is output. Based on the static image data uploaded by the user, the user's emotion in the static image is identified and input into the emotion rating function for calculation, and the user's emotion behavior coefficient QGDF is output;

[0065] Step 2: Based on the static image data uploaded by the user, identify and evaluate the user's environment in the static image, and output the user's environment data set;

[0066] Step 3: Output the user behavior filter based on the user action behavior coefficient DZDF and the user emotion behavior coefficient QGDF and in combination with the action filter library;

[0067] Step 4: Read the application scenario of each filter in the environmental filter library in the filter library statistical retrieval module, and output the environmental filter based on the user's environmental data set;

[0068] Step 5: Calculate based on user behavior filters and environment filters, and output intelligent recommendation filters.

[0069] Compared with the prior art, the present invention provides an intelligent recommendation system and method based on user behavior prediction in complex environments, which has the following beneficial effects:

[0070] 1. The present invention analyzes the static image data uploaded by the user, first extracts the main factors in the complex environment where the user is located, thereby ensuring that in the subsequent environmental retrieval and matching process, the most similar environmental filter in the existing environmental filter library can be quickly compared, and then the user behavior filter is determined according to the motion amplitude of the character in the static image data and the emotional state of the overall image, and the environmental filter and the user action behavior filter are combined to finally generate an intelligent recommendation filter, and the intelligent recommendation filter is applied to the static image data uploaded by the user, realizing the intelligent and objective recommendation of the corresponding filter by integrating the main factors in the complex environment, the user's facial emotions and the user's behavioral actions.

[0071] 2. The present invention classifies the number of people in the image. When there is only one person in the image, the corresponding filter is intelligently recommended based on the process combined with the main factors in the complex environment, the user's facial emotions and the user's behavioral actions. When there are multiple people in the image, after determining the main factors in the complex environment, all the emotions in the image are classified according to the different facial emotions of each person, and one is selected as the main emotion of the image, and the user's comprehensive behavioral actions are calculated. Filters are intelligently recommended based on the main factors in the complex environment, the main emotions of the image and the user's comprehensive behavioral actions, thereby ensuring that filters can be intelligently recommended to users normally when the image faces a single person or multiple people. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 It is a schematic diagram of the system framework of the present invention;

[0073] Figure 2 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0074] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0075] See also Figure 1 - Figure 2 , an intelligent recommendation system based on user behavior prediction in complex environments, including a user behavior recognition module, a user environment recognition module, a user emotion recognition module, a filter library statistical retrieval module, a data analysis module and an intelligent recommendation module;

[0076] The user behavior recognition module recognizes and evaluates the user's behavior in the static image based on the static image data uploaded by the user, and outputs the user action behavior coefficient DZDF. The specific steps of the user behavior recognition module recognizes and evaluates the user's behavior in the static image based on the static image data uploaded by the user, and outputs the user action behavior coefficient DZDF are as follows:

[0077] Step A1: Read the number of characters in the static image data uploaded by the user, and calculate the character interaction coefficient RWHD based on the number of characters. The specific expression is as follows:

[0078]

[0079] Among them, e represents the natural constant, a represents the correction coefficient, the correction coefficient is used to ensure that the value of the character interaction coefficient RWHD is the closest integer. When it is an integer, the correction coefficient a=0, RS represents the number of characters in the static image data, Indicates rounding up;

[0080] Step A2: reading the motion state of the person in the static image data, and calculating the person motion coefficient RWYD according to the motion state of the person and the number of people, wherein the motion state evaluation includes the arm motion amplitude SBDZFD and the leg motion amplitude TBDZFD, and the specific steps of determining the arm motion amplitude SBDZFD and the leg motion amplitude TBDZFD are as follows:

[0081] Step A2.a1: Use Openpose to identify the coordinates of the wrist and ankle positions, as well as the position of the central axis of the human body;

[0082] Step A2.a1: Calculate the minimum distance between any wrist position coordinate and ankle position coordinate and the human body central axis position. When two wrists are detected, select the larger one of the minimum distances between them and the human body central axis position. When one wrist is detected, only calculate the minimum distance between the wrist position coordinate and the human body central axis. The same is true for the ankle. The results of the above coordinate calculations are used as the arm movement range SBDZFD and the leg movement range TBDZFD respectively. The human body central axis at this point can be regarded as the extension line of the line connecting the midpoint of the eye and the tip of the nose, thereby ensuring the reference of the coordinates.

[0083] When calculating the character motion coefficient RWYD according to the character motion state in step A2, when the number of characters in the static image data is one, the specific expression is as follows:

[0084] RWYD=w1*SBDZFD+w2*TBDZFD

[0085] Among them, w1 and w2 are two weight coefficients, SBDZFD represents the arm movement range, and TBDZFD represents the leg movement range;

[0086] When calculating the character motion coefficient RWYD according to the character motion state in step A2, when there are multiple characters in the static image data, the specific steps are as follows:

[0087] Step A2.b1: Segment the characters in the static image data into I equal parts according to the number of people;

[0088] Step A2.b2: Calculate the arm motion amplitude SBDZFD and leg motion amplitude TBDZFD of each character, and comprehensively obtain the character motion coefficient RWYD of the entire image. The specific expression is as follows:

[0089]

[0090] Among them, w1 and w2 are two weight coefficients, SBDZFD i represents the arm movement amplitude of the i-th character, It represents the sum of the arm movement amplitudes of a total of I characters, RWYD represents the character movement coefficient, TBDZFD i represents the movement amplitude of the foot of the i-th character, It represents the sum of the leg motion amplitudes of a total of I characters, i∈[1,I], so that the motion amplitude of the character can be calculated based on the motion of the character in the static image;

[0091] Step A3: Calculate the user action coefficient DZDF based on the character interaction coefficient RWHD obtained in step A1 and the character motion coefficient RWYD obtained in step A2. The specific expression is as follows:

[0092] DZDF=RWHD*RWYD

[0093] Among them, RWHD represents the character interaction coefficient, RWYD represents the character motion coefficient, and DZDF represents the user action behavior coefficient. When there are multiple characters in the static image data, that is, the character interaction in the group photo is stronger than that in the case of a single person shooting, the number of characters in the image and the interactivity are relatively positively correlated. Therefore, when calculating the user action behavior, the number of people and the motion situation are combined to provide feedback on the overall user action behavior, so that a more accurate evaluation result can be obtained;

[0094] The user environment module identifies and evaluates the user's environment in the static image based on the static image data uploaded by the user, and outputs the user's environment data set. The user environment module identifies and evaluates the user's environment in the static image based on the static image data uploaded by the user, and outputs the user's environment data set. The specific steps of outputting the user's environment data set are as follows:

[0095] Step B1: Analyze the environmental data of the static image data and extract all environmental factors in the static image data through the DeepLab network model;

[0096] Step B2: Calculate the proportion of each environmental factor in the static image data, and rank the proportion of all environmental factors in the overall static image data, and select the top three environmental factors as the user's environmental data set. The specific calculation expression is as follows:

[0097]

[0098] Among them, ZB j represents the proportion of the jth environmental factor, XS j represents the total number of pixels of the jth environmental factor, XS max It represents the total pixels after removing people from the static image data. By extracting the environmental factors in the image, it can ensure that the main factors in the current image are selected, so as to ensure that in the subsequent environmental retrieval and matching process, the most similar environmental filter in the existing environmental filter library can be quickly compared;

[0099] The user emotion recognition module recognizes the user's emotions in the static image based on the static image data uploaded by the user, and inputs it into the emotion rating function for calculation, and outputs the user's emotional behavior coefficient QGDF. The specific expression of the emotion rating function is as follows:

[0100]

[0101] Among them, QXPJ represents the emotion rating function, QXPJ=1 represents strong emotion, QXPJ=0.5 represents emotional fluctuation, QXPJ=0.25 represents neutral emotion, strong emotion means that the emotion recognized by the user emotion recognition module is anger, disgust, fear, joy and sadness, neutral emotion means that the emotion recognized by the user emotion recognition module is surprise, and neutral emotion means that the emotion recognized by the user emotion recognition module is neutral. When the number of characters in the static image data is one, the user emotion recognition module recognizes the user's emotion in the static image based on the static image data uploaded by the user, and the specific process of outputting the user emotion behavior coefficient QGDF is: facial expression recognition is performed through the Pytorch model, and user emotions are output, and the user emotions are placed in the emotion rating function to obtain the user emotion behavior coefficient QGDF.

[0102] When there are multiple characters in the static image data, the user emotion recognition module recognizes the user's emotions in the static image based on the static image data uploaded by the user, and the specific steps of outputting the user emotion behavior coefficient QGDF are as follows:

[0103] Step C1: Segment the characters in the static image data into I equal parts according to the number of people;

[0104] Step C2: Perform facial expression recognition through the Pytorch model, classify the emotions with the same recognition results of the Pytorch model, and calculate the proportion of each emotion. The specific expression is as follows:

[0105]

[0106] Among them, QX1, ..., QX k ,…,QX K They represent the first emotion, …, the kth emotion, …, the Kth emotion, QXBF1, …, QXBF respectively, which are output and classified by the Pytorch model for expression recognition. k , ..., QXBF K They represent the percentage of the first emotion in all emotions, ..., the percentage of the kth emotion in all emotions, ..., the percentage of the Kth emotion in all emotions, DRQX represents a multi-person emotion dataset, k∈[1, K], K≤I, and the percentage of the kth emotion in all emotions QXBF k The specific expression is as follows:

[0107]

[0108] Among them, QXSL krepresents the number of recognized k-th emotion, and K represents the total number of classified emotions;

[0109] Step C3: Determine QXBF1, ..., QXBF in step C2 k , ..., QXBF K Is there a maximum value? If there is a maximum value, select a group of emotions corresponding to the maximum value and the percentage of the emotion in all emotions as the user emotional behavior coefficient QGDF. If QXBF1,…,QXBF k , ..., QXBF K If there are several equal maximum values ​​in , then randomly output an emotion from several equal maximum values ​​and the percentage of the emotion in all emotions;

[0110] Step C4: Input the emotion obtained in step C3 into the emotion rating function for calculation, and output the user emotion behavior coefficient QGDF;

[0111] The filter library statistics retrieval module stores the filter library, the filter library statistics retrieval module builds an environment filter library based on the application environment of the filters in the filter library, and the filter library statistics retrieval module builds an action filter library based on the application character actions of the filters in the filter library;

[0112] The data analysis module includes a user behavior comprehensive evaluation unit and a user environment data evaluation unit. The user behavior comprehensive evaluation unit outputs a user behavior filter based on the user action behavior coefficient DZDF and the user emotional behavior coefficient QGDF in combination with the action filter library. The specific steps of the user behavior comprehensive evaluation unit outputting a user behavior filter based on the user action behavior coefficient DZDF and the user emotional behavior coefficient QGDF in combination with the action filter library are as follows:

[0113] Step D1: The emotion output by the user emotion behavior coefficient QGDF is combined with the user action behavior coefficient DZDF to obtain the user comprehensive evaluation coefficient, the specific expression of which is as follows:

[0114] ZHPJXS=QGDF*DZDF

[0115] Among them, ZHPJXS represents the user comprehensive evaluation coefficient;

[0116] Step D2: Based on the user comprehensive evaluation coefficient, calculate the closeness coefficient between the comprehensive evaluation coefficient of each filter in the action filter library and the user comprehensive evaluation coefficient, select a filter with the smallest closeness from the action filter library, and store the corresponding contrast, exposure value, sharpness and saturation stored in the filter as the user behavior filter YHLJ. The expression for calculating the closeness coefficient between the comprehensive evaluation coefficient of each filter in the action filter library and the user comprehensive evaluation coefficient is as follows:

[0117] XDJn =|ZHPJXS-DZK n |

[0118] Among them, DZXS n Indicates the closeness coefficient between the comprehensive evaluation coefficient of the nth filter in the action filter library and the comprehensive evaluation coefficient of the user, DZK n Indicates the comprehensive evaluation coefficient of the nth filter in the action filter library, the comprehensive evaluation coefficient DZK of the nth filter in the action filter library n The acquisition method is the same as the acquisition method of the user comprehensive evaluation coefficient ZHPJXS;

[0119] The user environment data evaluation unit reads the application scenario of each filter in the environment filter library in the filter library statistical retrieval module, and outputs the environment filter based on the user's environment data set. The specific steps of the user environment data evaluation unit reading the application scenario of each filter in the environment filter library in the filter library statistical retrieval module and outputting the environment filter based on the user's environment data set are: calculating the similarity between the environmental factors of each filter in the environment filter library and the three environmental factors output in step B2, and selecting the contrast, exposure value, sharpness and saturation corresponding to the filter with the greatest similarity to be stored in the environment filter HJLJ. The specific expression for calculating the similarity between the environmental factors of each filter in the environment filter library and the three environmental factors output in step B2 is as follows:

[0120] HkDJ m =(1-|ZB x -ZB m，x |)+(1-|ZB y -ZB m，y |)+(1-|ZB z -ZB m，z |)

[0121] Among them, |*| represents the absolute value, HJXSD m represents the similarity between the mth filter in the environmental filter library and the three environmental factors output in step B2, ZB x , ZB y , ZB z represents the top three environmental factors selected in step B2, ZB m，x Indicates the corresponding ZB in the mth filter usage scenario in the environment filter library x Environmental factors, ZB m，y Indicates the corresponding ZB in the mth filter usage scenario in the environment filter library y Environmental factors, ZB m，z Indicates the corresponding ZB in the mth filter usage scenario in the environment filter library z environmental factors;

[0122] The intelligent recommendation module calculates based on the user behavior filter and the environment filter and outputs the intelligent recommendation filter. The intelligent recommendation module performs comprehensive analysis based on the user behavior filter and the environment filter and outputs the intelligent recommendation filter. The specific process is: the contrast, exposure value, sharpness and saturation stored in the environment filter HJLJ and the user behavior filter YHLJ are calculated respectively, and the calculation results are used as the contrast, exposure value, sharpness and saturation of the intelligent recommendation filter, and the contrast, exposure value, sharpness and saturation of the intelligent recommendation filter are applied to the static image data uploaded by the user. The specific expression is as follows:

[0123]

[0124] Among them, DBD zz Indicates the contrast of the intelligent recommended filter, DBD HJLJ Represents the contrast stored in the environment filter, DBD YHLJ Indicates the contrast stored in the user behavior filter, BGD zz Indicates the exposure of the smart recommended filter, BGD HJLJ Indicates the exposure stored in the environment filter, BGD YHLJ Represents the exposure stored in the user behavior filter, RD zz Indicates the sharpness of the intelligent recommended filter, RD HJLJ Indicates the sharpness stored in the environment filter, RD YHLJ Indicates the sharpness stored in the user behavior filter, BHD zz Indicates the saturation of the smart recommended filter, BHD HJLJ Indicates the saturation stored in the environment filter, BHD YHLJ Indicates the saturation stored in the user behavior filter.

[0125] The action filter library and the environment filter library described in this application are essentially directed to the same filter library. The action filter library is obtained by dividing the filters according to the character actions, and the environment filter library is obtained by extracting the use environment factors of the filters. In essence, it is actually generated by adding two different sets of labels to a filter library.

[0126] Embodiment 1:

[0127] In this embodiment, the static image data uploaded by the user is a single person, and the expression recognition result through the Pytorch model is joy. The output of the emotion rating function QXPJ=1 indicates strong emotions, the correction coefficient a=0, w1=0.56, w2=0.44, and all environmental factors extracted from the static image data through the DeepLab network model are: ocean ZB1=22.5%, beach ZB2=17%, tree ZB3=7.5%, sky ZB4=23%, reef ZB5=6%, parasol ZB6=6% and fence ZB7=8%. At this time, the user's environmental data set includes all the above environmental factors and the corresponding percentages. At this time, the top three environmental factors screened out are ocean ZB1=22.5%, beach ZB2=17% and sky ZB4=23%, arm movement amplitude SBDZFD=5, leg movement amplitude TBDZFD=3;

[0128] Calculate the character interaction coefficient at this time The character motion coefficient RWYD=0.56*5+0.44*3=4.12. The user action behavior coefficient DZDF=RWHD*RWYD=4.12*1=4.12 is calculated based on the character interaction coefficient RWHD and the character motion coefficient RWYD. At this time, the user comprehensive evaluation coefficient ZHPJXS=QGDF*DZDF=4.12. The comprehensive evaluation coefficient of each filter in the action filter library recorded in this embodiment is shown in Table 1 below;

[0129] Table 1

[0130] Action Filter Library Comprehensive evaluation coefficient Filter 1 2.15 Filter 2 8.46 Filter 3 5.76 Filter 4 3.08 Filter 5 4.29

[0131] The contrast, exposure value, sharpness and saturation stored in the selection filter 5 are stored in the user behavior filter YHLJ, and the proportion of environmental factors corresponding to the environmental factors uploaded in the static image stored in the environmental filter library is shown in the following Table 2;

[0132] Table 2

[0133] Environment Filter Library Environmental factors Filter 1 Ocean = 30%, Beach = 7%, Sky = 2% Filter 2 Ocean = 0%, Beach = 37%, Sky = 15% Filter 3 Ocean = 0%, Beach = 0%, Sky = 18% Filter 4 Ocean = 23%, Beach = 21%, Sky = 19% Filter 5 Ocean = 5%, Beach = 0%, Sky = 7%

[0134] HJXSD1=2.575、HJXSD2=2.495、HJXSD3=2.555、HJXSD4=2.915、

[0135] HJXSD1=2.495, at this time HJXSD4=2.915 is the maximum, the contrast, exposure value, sharpness and saturation stored in filter 4 are selected and stored in the environment filter HJLJ, and the specific parameters of filter 4 and filter 5 and the obtained intelligent recommended filter are shown in Table 3;

[0136] Table 3

[0137] Filter Name Contrast Exposure Value Sharpness Saturation Filter 4 30 10 25 40 Filter 5 10 6 20 50 Smart recommended filters 20 8 22.5 45

[0138] Embodiment 2:

[0139] In this embodiment, the static image data uploaded by the user is three people, a=-0.389, expression recognition is performed through the Pytorch model, and the emotions with the same recognition results of the Pytorch model are classified to obtain At this time, the maximum value is joy, and joy is selected to be input into the emotion rating function for calculation, and the user emotion behavior coefficient QGDF=1 is output, w1=0.56, w2=0.44, and all environmental factors in the static image data are extracted through the DeepLab network model, specifically: animals ZB1=16%, beaches ZB2=6%, trees ZB3=22%, sky ZB4=18%, benches ZB5=10%, vehicles ZB6=3% and buildings ZB7=25%. At this time, the user's environmental data set includes all the above environmental factors and the corresponding percentages. At this time, the top three environmental factors are selected as buildings ZB7=25%, trees ZB3=22% and sky ZB4=18%;

[0140] Calculate the character interaction coefficient Character motion coefficient The user action behavior coefficient DZDF=RWHD*RWYD=7*2.86=20.02 is calculated based on the character interaction coefficient RWHD and the character motion coefficient RWYD. The comprehensive evaluation coefficient of each filter in the action filter library recorded in this embodiment is shown in Table 4 below;

[0141] Table 4

[0142] Action Filter Library Comprehensive evaluation coefficient Filter 1 8.15 Filter 2 15.46 Filter 3 25.76 Filter 4 13.08 Filter 5 4.29

[0143] The contrast, exposure value, sharpness and saturation stored in the selection filter 3 are stored in the user behavior filter YHLJ, and the proportion of environmental factors corresponding to the environmental factors uploaded in the static image stored in the environmental filter library is shown in Table 5 below;

[0144] Table 5

[0145]

[0146]

[0147] HJXSD1=2.56、HJXSD2=2.67、HJXSD3=2.42、HJXSD4=2.05、

[0148] HJXSD1=1.85, at this time HJXSD2=2.67 is the maximum, the contrast, exposure value, sharpness and saturation stored in filter 2 are selected and stored in the environment filter HJLJ, and the specific parameters of filter 2 and filter 3 and the obtained intelligent recommended filter are shown in Table 6;

[0149] Table 6

[0150] Filter Name Contrast Exposure Value Sharpness Saturation Filter 2 15 -5 8 10 Filter 3 10 -7 1 3 Smart recommended filters 12.5 -6 4.5 6.5

[0151] The present invention also provides an intelligent recommendation method based on complex environment user behavior prediction, which applies the above-mentioned intelligent recommendation system based on complex environment user behavior prediction, and includes the following steps:

[0152] Step 1: Based on the static image data uploaded by the user, the user's behavior in the static image is identified and evaluated, and the user's action behavior coefficient DZDF is output. Based on the static image data uploaded by the user, the user's emotion in the static image is identified and input into the emotion rating function for calculation, and the user's emotion behavior coefficient QGDF is output;

[0153] Step 2: Based on the static image data uploaded by the user, identify and evaluate the user's environment in the static image, and output the user's environment data set;

[0154] Step 3: Output the user behavior filter based on the user action behavior coefficient DZDF and the user emotion behavior coefficient QGDF and combined with the action filter library;

[0155] Step 4: Read the application scenario of each filter in the environmental filter library in the filter library statistical retrieval module, and output the environmental filter based on the user's environmental data set;

[0156] Step 5: Calculate based on user behavior filters and environment filters, and output intelligent recommended filters. The static image data uploaded by the user is consistent with the image data size of the filter library application stored in the filter library statistical retrieval module.

[0157] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An intelligent recommendation system based on user behavior prediction in complex environments, characterized by: It includes user behavior recognition module, user environment recognition module, user emotion recognition module, filter library statistical retrieval module, data analysis module and intelligent recommendation module; The user behavior recognition module recognizes and evaluates the user's behavior in the static image based on the static image data uploaded by the user, and outputs the user action behavior coefficient DZDF; The user environment module identifies and evaluates the user's environment in the static image based on the static image data uploaded by the user, and outputs the user's environment data set; The user emotion recognition module recognizes the user's emotion in the static image based on the static image data uploaded by the user, and inputs it into the emotion rating function for calculation, and outputs the user's emotion behavior coefficient QGDF. The specific expression of the emotion rating function is as follows: Wherein, QXPJ represents the emotion rating function, QXPJ=1 represents strong emotion, QXPJ=0.5 represents fluctuating emotion, QXPJ=0.25 represents neutral emotion, the strong emotion means that the emotion recognized by the user emotion recognition module is anger, disgust, fear, joy and sadness, the neutral emotion means that the emotion recognized by the user emotion recognition module is surprise, and the neutral emotion means that the emotion recognized by the user emotion recognition module is neutral; The filter library statistics retrieval module stores a filter library, the filter library statistics retrieval module builds an environment filter library based on the application environment of the filters in the filter library, and the filter library statistics retrieval module builds an action filter library based on the application character actions of the filters in the filter library; The data analysis module includes a user behavior comprehensive evaluation unit and a user environment data evaluation unit, wherein the user behavior comprehensive evaluation unit outputs a user behavior filter based on the user action behavior coefficient DZDF and the user emotion behavior coefficient QGDF in combination with the action filter library; The user environment data evaluation unit reads the application scenario of each filter in the environment filter library in the filter library statistical retrieval module, and outputs the environment filter based on the user's environment data set; The intelligent recommendation module performs calculations based on user behavior filters and environment filters, and outputs intelligent recommendation filters.

2. The intelligent recommendation system based on complex environment user behavior prediction according to claim 1, characterized in that: The user behavior recognition module recognizes and evaluates the user's behavior in the static image based on the static image data uploaded by the user, and outputs the user action behavior coefficient DZDF in the following specific steps: Step A1: Read the number of characters in the static image data uploaded by the user, and calculate the character interaction coefficient RWHD based on the number of characters. The specific expression is as follows: Where, e represents the natural constant, a represents the correction coefficient, and RS represents the number of characters in the static image data. Indicates rounding up; Step A2: reading the motion state of the character in the static image data, and calculating the character motion coefficient RWYD according to the different motion states of the characters and the number of characters, wherein the motion state evaluation includes the arm motion amplitude SBDZFD and the leg motion amplitude TBDZFD, and the specific steps of determining the arm motion amplitude SBDZFD and the leg motion amplitude TBDZFD are as follows: Step A2.a1: Use Openpose to identify the coordinates of the wrist and ankle positions, as well as the position of the central axis of the human body; Step A2.a1: Calculate the minimum distance between any wrist position coordinate and any ankle position coordinate and the position of the central axis of the human body, and use them as the arm movement range SBDZFD and the leg movement range TBDZFD respectively; Step A3: Calculate the user action coefficient DZDF based on the character interaction coefficient RWHD obtained in step A1 and the character motion coefficient RWYD obtained in step A2. The specific expression is as follows: DZDF=RWHD*RWYD Among them, RWHD represents the character interaction coefficient, RWYD represents the character motion coefficient, and DZDF represents the user action behavior coefficient.

3. The intelligent recommendation system based on complex environment user behavior prediction according to claim 2 is characterized by: When the character motion coefficient RWYD is calculated according to the character motion state in step A2, when the number of characters in the static image data is one, the specific expression is as follows: RWYD=w1*SBDZFD+w2*TBDZFD Among them, w1 and w2 are two weight coefficients, SBDZFD represents the arm movement range, and TBDZFD represents the leg movement range.

4. The intelligent recommendation system based on complex environment user behavior prediction according to claim 2, characterized in that: When calculating the character motion coefficient RWYD according to the character motion state in step A2, when there are multiple characters in the static image data, the specific steps are as follows: Step A2.b1: Segment the characters in the static image data into I equal parts according to the number of people; Step A2.b2: Calculate the arm motion amplitude SBDZFD and leg motion amplitude TBDZFD of each character, and comprehensively obtain the character motion coefficient RWYD of the entire image. The specific expression is as follows: Among them, w1 and w2 are two weight coefficients, SBDZFD i represents the arm movement amplitude of the i-th character, It represents the sum of the arm movement amplitudes of a total of I characters, RWYD represents the character movement coefficient, TBDZFD i represents the movement amplitude of the foot of the i-th character, It represents the sum of the leg movement amplitudes of a total of I characters, i∈[1,I].

5. The intelligent recommendation system based on complex environment user behavior prediction according to claim 1 is characterized by: The user environment module identifies and evaluates the user's environment in the static image based on the static image data uploaded by the user, and the specific steps of outputting the user environment data set are as follows: Step B1: Analyze the environmental data of the static image data and extract all environmental factors in the static image data through the DeepLab network model; Step B2: Calculate the proportion of each environmental factor in the static image data, and rank the proportion of all environmental factors in the overall static image data, and select the top three environmental factors as the user's environmental data set. The specific calculation expression is as follows: Among them, ZB j represents the proportion of the jth environmental factor, XS j represents the total number of pixels of the jth environmental factor, XS max Represents the total number of pixels in the static image data after removing the person.

6. The intelligent recommendation system based on complex environment user behavior prediction according to claim 3 is characterized by: When there is only one person in the static image data, the user emotion recognition module identifies the user's emotions in the static image based on the static image data uploaded by the user, and the specific process of outputting the user emotion behavior coefficient QGDF is: facial expression recognition is performed through the Pytorch model, and the user emotion is output, and the user emotion is placed in the emotion rating function to obtain the user emotion behavior coefficient QGDF.

7. The intelligent recommendation system based on complex environment user behavior prediction according to claim 4 is characterized by: When there are multiple characters in the static image data, the user emotion recognition module recognizes the user's emotions in the static image based on the static image data uploaded by the user, and the specific steps of outputting the user emotion behavior coefficient QGDF are as follows: Step C1: Segment the characters in the static image data into I equal parts according to the number of people; Step C2: Perform facial expression recognition through the Pytorch model, classify the emotions with the same recognition results of the Pytorch model, and calculate the proportion of each emotion. The specific expression is as follows: Among them, QX1,…,QX k ,…,QX K They represent the first emotion, …, the kth emotion, …, the Kth emotion, respectively, which are output and classified by the Pytorch model for expression recognition. QXBF1, …, QXBF k ,…,QXBF K They represent the percentage of the first emotion in all emotions, ..., the percentage of the kth emotion in all emotions, ..., the percentage of the Kth emotion in all emotions, DRQX represents a multi-person emotion dataset, k∈[1,K], K≤I, and the percentage of the kth emotion in all emotions QXBF k The specific expression is as follows: Among them, QXSL k represents the number of recognized k-th emotion, and K represents the total number of classified emotions; Step C3: Determine QXBF1,…,QXBF in step C2 k ,…,QXBF K Is there a maximum value? If there is a maximum value, select a group of emotions corresponding to the maximum value and the percentage of the emotion in all emotions as the user emotional behavior coefficient QGDF. If QXBF1,…,QXBF k ,…,QXBF K If there are several equal maximum values ​​in , then randomly output an emotion from several equal maximum values ​​and the percentage of the emotion in all emotions; Step C4: Input the emotion obtained in step C3 into the emotion rating function for calculation, and output the user emotion behavior coefficient QGDF.

8. The intelligent recommendation system based on complex environment user behavior prediction according to claim 5 is characterized by: The specific steps of the user behavior comprehensive evaluation unit outputting the user behavior filter based on the user action behavior coefficient DZDF and the user emotional behavior coefficient QGDF in combination with the action filter library are as follows: Step D1: The emotion output by the user emotion behavior coefficient QGDF is combined with the user action behavior coefficient DZDF to obtain the user comprehensive evaluation coefficient, the specific expression of which is as follows: ZHPJXS=QGDF*DZDF Among them, ZHPJXS represents the user comprehensive evaluation coefficient; Step D2: Based on the user comprehensive evaluation coefficient, calculate the closeness coefficient between the comprehensive evaluation coefficient of each filter in the action filter library and the user comprehensive evaluation coefficient, select a filter with the smallest closeness from the action filter library, and store the corresponding contrast, exposure value, sharpness and saturation stored in the filter as the user behavior filter YHLJ. The expression for calculating the closeness coefficient between the comprehensive evaluation coefficient of each filter in the action filter library and the user comprehensive evaluation coefficient is as follows: DZXS n =|ZHPJXS-DZK n | Among them, DZXS n Indicates the closeness coefficient between the comprehensive evaluation coefficient of the nth filter in the action filter library and the comprehensive evaluation coefficient of the user, DZK n represents the comprehensive evaluation coefficient of the nth filter in the action filter library, and the comprehensive evaluation coefficient DZK of the nth filter in the action filter library n The acquisition method is the same as the acquisition method of the user comprehensive evaluation coefficient ZHPJXS; The user environment data evaluation unit reads the application scenario of each filter in the environment filter library in the filter library statistical retrieval module, and outputs the environment filter based on the user's environment data set. The specific steps are: calculating the similarity between the environmental factors of each filter in the environment filter library and the three environmental factors output in step B2, and selecting the contrast, exposure value, sharpness and saturation corresponding to the filter with the greatest similarity to store in the environment filter HJLJ. The specific expression for calculating the similarity between the environmental factors of each filter in the environment filter library and the three environmental factors output in step B2 is as follows: HkDJ m =(1-|ZB x -ZB m,x |)+(1-|ZB y -ZB m,y |)+(1-|ZB z -ZB m,z |) where |*| represents the absolute value, HJXSD m represents the similarity between the mth filter in the environmental filter library and the three environmental factors output in step B2, ZB x , ZB y , ZB z represents the top three environmental factors selected in step B2, ZB m,x Indicates the corresponding ZB in the mth filter usage scenario in the environment filter library x Environmental factors, ZB m,y Indicates the corresponding ZB in the mth filter usage scenario in the environment filter library y Environmental factors, ZB m,z Indicates the corresponding ZB in the mth filter usage scenario in the environment filter library z environmental factors.

9. The intelligent recommendation system based on complex environment user behavior prediction according to claim 8, characterized in that: The specific process of the intelligent recommendation module for comprehensive analysis based on the user behavior filter and the environment filter and outputting the intelligent recommendation filter is as follows: the contrast, exposure value, sharpness and saturation stored in the environment filter HJLJ and the user behavior filter YHLJ are calculated respectively, and the calculation results are used as the contrast, exposure value, sharpness and saturation of the intelligent recommendation filter, and the contrast, exposure value, sharpness and saturation of the intelligent recommendation filter are applied to the static image data uploaded by the user. The specific expressions are as follows: Among them, DBD zz Indicates the contrast of the intelligent recommended filter, DBD HJLJ Represents the contrast stored in the environment filter, DBD YHLJ Indicates the contrast stored in the user behavior filter, BGD zz Indicates the exposure of the smart recommended filter, BGD HJLJ Indicates the exposure stored in the environment filter, BGD YHLJ Represents the exposure stored in the user behavior filter, RD zz Indicates the sharpness of the intelligent recommended filter, RD HJLJ Indicates the sharpness stored in the environment filter, RD YHLJ Indicates the sharpness stored in the user behavior filter, BHD zz Indicates the saturation of the smart recommended filter, BHD HJLJ Indicates the saturation stored in the environment filter, BHD YHLJ Indicates the saturation stored in the user behavior filter.

10. An intelligent recommendation method based on user behavior prediction in a complex environment, applying the intelligent recommendation system based on user behavior prediction in a complex environment as claimed in any one of claims 1 to 9, characterized in that: The following steps are involved: Step 1: Based on the static image data uploaded by the user, the user's behavior in the static image is identified and evaluated, and the user's action behavior coefficient DZDF is output. Based on the static image data uploaded by the user, the user's emotion in the static image is identified and input into the emotion rating function for calculation, and the user's emotion behavior coefficient QGDF is output; Step 2: Based on the static image data uploaded by the user, identify and evaluate the user's environment in the static image, and output the user's environment data set; Step 3: Output the user behavior filter based on the user action behavior coefficient DZDF and the user emotion behavior coefficient QGDF and in combination with the action filter library; Step 4: Read the application scenario of each filter in the environmental filter library in the filter library statistical retrieval module, and output the environmental filter based on the user's environmental data set; Step 5: Calculate based on user behavior filters and environment filters, and output intelligent recommendation filters.

Images