Intelligent image display method, device, medium and equipment

Abstract

The present invention discloses an intelligent image display method, device, medium and equipment. The method includes: collecting image data sets of different categories and different user preferences; classifying the image data sets according to a pre-trained image classification model, determining the preference classification label of each image relative to different user preferences, wherein the preference classification label includes: a like category and a dislike category; calculating the score of each image in the like category of different user preferences according to the pre-set different user preference weights; sorting all images in the like category according to the score of each image in the like category of different user preferences, and determining the image sorting result under each user preference; and displaying the images to target users of different user preferences according to the image sorting results of different user preferences.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and more specifically, to an intelligent image display method, device, medium and equipment. Background Art

[0002] In the current lecture system, lecturers will conduct training offline or online. During the training process, some photos or screenshots will be generated. The lecturer will transfer the pictures to the system and display them to other users. The picture types include: lecturer explanation pictures, on-site atmosphere pictures, empty field pictures, off-site pictures, explanation PPT pictures, live screenshots of the explanation process, etc. The lecturer will upload various pictures to the system for users to choose for reference, but because there are many pictures and there is no fixed order, users need to spend a lot of time looking for suitable pictures, which affects the user experience. Summary of the invention

[0003] In view of the deficiencies in the prior art, the present invention provides an intelligent picture display method, device, medium and equipment.

[0004] According to one aspect of the present invention, there is provided an intelligent picture display method, comprising:

[0005] Collect image datasets of different categories and user preferences;

[0006] Classify the image dataset according to the pre-trained image classification model, and determine the preference classification label of each image relative to different users' preferences, where the preference classification label includes: like category and dislike category;

[0007] Calculate the score of each picture in the favorite category of different users according to the pre-set weights of different users' preferences;

[0008] Sort all the pictures in the favorite categories according to the scores of each picture in the favorite categories of different users, and determine the picture sorting results under each user's preference;

[0009] According to the picture sorting results of different users' preferences, the pictures are displayed to target users with different user preferences.

[0010] Optionally, the image dataset is classified according to a pre-trained image classification model to determine a preference classification label of each image relative to different user preferences, including:

[0011] Classify the images in the image dataset according to the image classification model and determine the image label of each image;

[0012] A binary classification model is used to classify the image dataset according to different user preferences and the image labels of the images in the image dataset, and determine the preference classification labels of each image for different user preferences.

[0013] Optionally, the binary classification model is a logistic regression model, and its prediction function is:

[0014] h_theta(x)=g(theta^T*x)

[0015] Among them, g(z)=1 / (1+e^-z) is the Sigmoid function, theta is the model parameter, and x is the feature vector of the image.

[0016] Optionally, the loss function for the binary classification model is:

[0017] J(theta)=-1 / m*∑[y log(h_theta(x))+(1-y)log(1-h_theta(x))]

[0018] Among them, y is the preference classification label of the picture, 1 is if the user likes it, and 0 is if the user dislikes it, and m is the total number of pictures.

[0019] According to another aspect of the present invention, there is provided an intelligent picture display device, comprising:

[0020] The collection module is used to collect image datasets of different categories and user preferences;

[0021] A classification module is used to classify the image data set according to a pre-trained image classification model, and determine the preference classification label of each image relative to different user preferences, wherein the preference classification label includes: a like category and a dislike category;

[0022] A calculation module, used to calculate the score of each picture in the favorite category of different users according to the preset preference weights of different users;

[0023] A sorting module is used to sort all the pictures in the favorite category according to the scores of each picture in the favorite category of different users, and determine the picture sorting result under each user preference;

[0024] The display module is used to display pictures to target users with different user preferences according to the picture sorting results of different user preferences.

[0025] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute the method described in any one of the above aspects of the present invention.

[0026] According to another aspect of the present invention, an electronic device is provided, comprising: a processor; a memory for storing instructions executable by the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the method described in any one of the above aspects of the present invention.

[0027] Therefore, this application automatically identifies the image content of images in the favorite categories of users with different preferences by setting weights of different user preferences, sorts the images according to user preferences and displays them to users, thereby reducing the time users spend searching for images and improving user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] A more complete understanding of exemplary embodiments of the present invention may be obtained by referring to the following drawings:

[0029] Figure 1 is a flow chart of an intelligent picture display method provided by an exemplary embodiment of the present invention;

[0030] Figure 2 is a schematic structural diagram of an intelligent picture display device provided by an exemplary embodiment of the present invention;

[0031] Figure 3 This is a structure of an electronic device provided by an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0032] Below, the exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments of the present invention, and it should be understood that the present invention is not limited to the exemplary embodiments described here.

[0033] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless specifically stated otherwise.

[0034] Those skilled in the art can understand that the terms "first" and "second" in the embodiments of the present invention are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor indicate the necessary logical order between them.

[0035] It should also be understood that, in the embodiments of the present invention, “plurality” may refer to two or more than two, and “at least one” may refer to one, two or more than two.

[0036] It should also be understood that any component, data or structure mentioned in the embodiments of the present invention can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0037] In addition, the term "and / or" in the present invention is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present invention generally indicates that the associated objects before and after are in an "or" relationship.

[0038] It should also be understood that the description of the various embodiments of the present invention focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.

[0039] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0040] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

[0041] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0042] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0043] Embodiments of the present invention can be applied to electronic devices such as terminal devices, computer systems, servers, etc., which can operate with many other general or special computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, servers, etc. include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, small computer systems, large computer systems, and distributed cloud computing technology environments including any of the above systems, etc.

[0044] Electronic devices such as terminal devices, computer systems, servers, etc. can be described in the general context of computer system executable instructions (such as program modules) executed by computer systems. Generally, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media including storage devices.

[0045] Exemplary Methods

[0046] Figure 1 FIG. 1 is a flow chart of an intelligent picture display method provided by an exemplary embodiment of the present invention. This embodiment can be applied to electronic devices, such as Figure 1 As shown, the intelligent picture display method 100 includes the following steps:

[0047] Step 101, collecting image data sets of different categories and different user preferences;

[0048] Step 102, classifying the image data set according to the pre-trained image classification model, determining the preference classification label of each image relative to different user preferences, wherein the preference classification label includes: a like category and a dislike category;

[0049] Step 103, calculating the score of each picture in the favorite category of different users according to the preset preference weights of different users;

[0050] Step 104, sorting all the pictures in the favorite categories according to the scores of each picture in the favorite categories of different users, and determining the picture sorting result under each user preference;

[0051] Step 105, according to the picture sorting results of different users' preferences, the pictures are displayed to target users of different user preferences respectively.

[0052] Specifically, the present application provides a specific process of intelligent image display as follows:

[0053] 1. Collect and organize image datasets of different categories and user preferences, and build user models based on user preference data. The image datasets include lecturer explanation pictures, on-site atmosphere pictures, empty venue pictures, off-site pictures, explanation PPT pictures, and live screenshots of the explanation process.

[0054] 2. Classify the uploaded pictures through the picture intelligent classification model;

[0055] 3. Match different types of pictures to users based on their preferences, assign a score to each picture, and then sort them according to the score. You can use a rule-based sorting algorithm, such as the weighted average method or a rule-based recommendation algorithm, or you can use a machine learning-based sorting algorithm, such as a ranking neural network (RankNet), ranking learning (Ranking SVM), etc.

[0056] In the recommendation system, scoring images often involves collaborative filtering based on user preferences or content-based recommendations. To simplify, only considering the user's exact "like" or "dislike", we can use machine learning classification problems to understand and solve this problem. Assume that we have obtained a list of "liked" and "disliked" images based on the user's behavior, and have extracted the features of the images. We can transform the problem into a binary classification problem, that is, for each image, whether the user likes (label 1) or dislikes (label 0).

[0057] 4. Sort the pictures that the user likes, generate a sorted picture list, and display the sorting results to the user. The user can perform corresponding operations based on the sorting results.

[0058] Therefore, this application automatically identifies the image content of images in the favorite categories of users with different preferences by setting weights of different user preferences, sorts the images according to user preferences and displays them to users, thereby reducing the time users spend searching for images and improving user experience.

[0059] Optionally, the image dataset is classified according to a pre-trained image classification model to determine a preference classification label of each image relative to different user preferences, including:

[0060] Classify the images in the image dataset according to the image classification model and determine the image label of each image;

[0061] A binary classification model is used to classify the image dataset according to different user preferences and the image labels of the images in the image dataset, and determine the preference classification labels of each image for different user preferences.

[0062] Optionally, the binary classification model is a logistic regression model, and its prediction function is:

[0063] h_theta(x)=g(theta^T*x)

[0064] Among them, g(z)=1 / (1+e^-z) is the Sigmoid function, theta is the model parameter, and x is the feature vector of the image.

[0065] Optionally, the loss function for the binary classification model is the cross entropy loss function:

[0066] J(theta)=-1 / m*∑[y log(h_theta(x))+(1-y)log(1-h_theta(x))]

[0067] Among them, y is the preference classification label of the picture, 1 is if the user likes it, and 0 is if the user dislikes it, and m is the total number of pictures.

[0068] In one embodiment, the algorithm implementation process of the present application is as follows:

[0069] 1. Import the trained intelligent image classification model;

[0070] 2. Classification of picture content preferred by users;

[0071] user_preferences=['cat','dog','flower','car','...'].

[0072] 3. Set the weight of user preferences;

[0073] preference_weights={'cat':3,'dog':2,'flower':1,'car':0}.

[0074] 4. Define a function to predict the content of the image and sort it according to user preferences;

[0075] defsort_images_by_preference(images).

[0076] 5. Initialize a dictionary to store images and their corresponding scores;

[0077] scores = defaultdict(int).

[0078] 6. Traverse each picture;

[0079] for img_path in images.

[0080] 7. Predict the content of the picture;

[0081] prediction=learn.predict(Image(img_path)).

[0082] 8. Get the index of the image content (and the image content classification in 2);

[0083] content_index=user_preferences.index(prediction[0]).

[0084] 9. Calculate the score based on the weight of user preferences;

[0085] scores[img_path]=preference_weights[prediction[0]]*(content_index+1).

[0086] 10. Sort the pictures according to the scores;

[0087] sorted_images=sorted(scores,key=scores.get,reverse=True)

[0088] return sorted_images.

[0089] Therefore, this application automatically identifies the image content of images in the favorite categories of users with different preferences by setting weights of different user preferences, sorts the images according to user preferences and displays them to users, thereby reducing the time users spend searching for images and improving user experience.

[0090] Exemplary Devices

[0091] Figure 2 FIG. 1 is a schematic diagram of the structure of an intelligent picture display device provided by an exemplary embodiment of the present invention. Figure 2 As shown, the device 200 includes:

[0092] A collection module 210 is used to collect picture data sets of different categories and different user preferences;

[0093] The classification module 220 is used to classify the image data set according to the pre-trained image classification model, and determine the preference classification label of each image relative to different user preferences, wherein the preference classification label includes: a like category and a dislike category;

[0094] The calculation module 230 is used to calculate the score of each picture in the favorite category of different users according to the preset preference weights of different users;

[0095] A sorting module 240 is used to sort all the pictures in the favorite categories according to the scores of each picture in the favorite categories of different users, and determine the picture sorting results under each user preference;

[0096] The display module 250 is used to display pictures to target users with different user preferences according to the picture sorting results of different user preferences.

[0097] Optionally, the classification module 220 includes:

[0098] The first classification submodule is used to classify the images in the image dataset according to the image classification model and determine the image label of each image;

[0099] The second classification submodule is used to classify the image data set according to different user preferences and image labels of the images in the image data set using a binary classification model, and determine the preference classification label of each image for different user preferences.

[0100] Optionally, the binary classification model is a logistic regression model, and its prediction function is:

[0101] h_theta(x)=g(theta^T*x)

[0102] Among them, g(z)=1 / (1+e^-z) is the Sigmoid function, theta is the model parameter, and x is the feature vector of the image.

[0103] Optionally, the loss function for the binary classification model is:

[0104] J(theta)=-1 / m*∑[y log(h_theta(x))+(1-y)log(1-h_theta(x))]

[0105] Among them, y is the preference classification label of the image, 1 if the user likes it and 0 if the user dislikes it, and m is the total number of images.

[0106] Exemplary Electronic Devices

[0107] Figure 3 This is a structure of an electronic device provided by an exemplary embodiment of the present invention. Figure 3 As shown, the electronic device 30 includes one or more processors 31 and a memory 32 .

[0108] The processor 31 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0109] The memory 32 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 31 may run the program instructions to implement the methods of the software programs of the various embodiments of the present invention described above and / or other desired functions. In one example, the electronic device may also include: an input device 33 and an output device 34, which are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0110] In addition, the input device 33 may also include, for example, a keyboard, a mouse, and the like.

[0111] The output device 34 can output various information to the outside. The output device 34 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto.

[0112] Of course, to simplify, Figure 3 Only some of the components related to the present invention in the electronic device are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application conditions, the electronic device may also include any other appropriate components.

[0113] Exemplary computer program products and computer-readable storage media

[0114] In addition to the above-mentioned methods and devices, an embodiment of the present invention may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method according to various embodiments of the present invention described in the above-mentioned "Exemplary Method" section of this specification.

[0115] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the embodiments of the present invention, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0116] In addition, an embodiment of the present invention may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enables the processor to execute the steps of the method for information mining of historical change records according to various embodiments of the present invention described in the above “Exemplary Method” section of this specification.

[0117] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, system or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0118] The basic principle of the present invention is described above in conjunction with specific embodiments. However, it should be pointed out that the advantages, strengths, effects, etc. mentioned in the present invention are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. must be possessed by each embodiment of the present invention. In addition, the specific details disclosed above are only for the purpose of illustration and facilitation of understanding, rather than limitation, and the above details do not limit the present invention to being implemented by adopting the above specific details.

[0119] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0120] The block diagrams of the devices, systems, equipment, and systems involved in the present invention are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, systems, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The word "such as" used here refers to the phrase "such as but not limited to", and can be used interchangeably with it.

[0121] The method and system of the present invention may be implemented in many ways. For example, the method and system of the present invention may be implemented by software, hardware, firmware or any combination of software, hardware, firmware. The above order of steps for the method is only for illustration, and the steps of the method of the present invention are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present invention may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present invention. Thus, the present invention also covers a recording medium storing a program for executing the method according to the present invention.

[0122] It should also be noted that in the system, device and method of the present invention, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent schemes of the present invention. The above description of the disclosed aspects is provided to enable any technician in the field to make or use the present invention. Various modifications to these aspects are very obvious to those skilled in the art, and the general principles defined here can be applied to other aspects without departing from the scope of the present invention. Therefore, the present invention is not intended to be limited to the aspects shown here, but in accordance with the widest range consistent with the principles and novel features disclosed here.

[0123] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present invention to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. An intelligent picture display method, characterized in that: include: Collect image datasets of different categories and user preferences; Classifying the image data set according to a pre-trained image classification model, and determining a preference classification label of each image relative to preferences of different users, wherein the preference classification label includes: a like category and a dislike category; Calculate the score of each picture in the favorite category of different users according to the preset preference weights of different users; Sort all the pictures in the favorite category according to the scores of each picture in the favorite category of different users, and determine the picture sorting result under each user preference; According to the picture sorting results of different users' preferences, the pictures are displayed to target users with different user preferences respectively; The image dataset is classified according to a pre-trained image classification model to determine a preference classification label of each image relative to different user preferences, including: Classifying the images in the image dataset according to the image classification model to determine an image label for each image; Using a binary classification model to classify the image dataset according to different user preferences and image labels of the images in the image dataset, and determining a preference classification label for each image according to different user preferences; The binary classification model is a logistic regression model, and its prediction function is: h_theta(x)=g(theta^T*x) Where g(z)=1 / (1+e^-z) is the Sigmoid function, theta is the model parameter, x is the feature vector of the image, and T is the transpose; The loss function of the binary classification model is: J(theta)=-1 / m*∑[y log(h_theta(x))+(1-y)log(1-h_theta(x))] Among them, y is the preference classification label of the picture, 1 is if the user likes it, and 0 is if the user dislikes it, and m is the total number of pictures.

2. An intelligent picture display device, characterized in that: include: The collection module is used to collect image datasets of different categories and user preferences; A classification module, used to classify the image data set according to a pre-trained image classification model, and determine a preference classification label of each image relative to different user preferences, wherein the preference classification label includes: a like category and a dislike category; A calculation module, used to calculate the score of each picture in the favorite category of different users according to preset preference weights of different users; A sorting module is used to sort all the pictures in the favorite category according to the scores of each picture in the favorite category of different users, and determine the picture sorting result under each user preference; A display module, used to display pictures to target users with different user preferences according to the picture sorting results of different user preferences; Classification modules, including: A first classification submodule, used to classify the images in the image dataset according to the image classification model, and determine an image label for each image; The second classification submodule is used to classify the image data set according to different user preferences and image labels of the images in the image data set using a binary classification model, and determine a preference classification label for each image according to different user preferences; The binary classification model is a logistic regression model, and its prediction function is: h_theta(x)=g(theta^T*x) Where g(z)=1 / (1+e^-z) is the Sigmoid function, theta is the model parameter, x is the feature vector of the image, and T is the transpose; The loss function of the binary classification model is: J(theta)=-1 / m*∑[y log(h_theta(x))+(1-y)log(1-h_theta(x))] Among them, y is the preference classification label of the picture, 1 is if the user likes it, and 0 is if the user dislikes it, and m is the total number of pictures.

3. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the method according to claim 1.

4. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing instructions executable by the processor; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the method of claim 1.

Images