Image processing method and device, computer equipment and computer readable storage medium

Abstract

The invention discloses an image processing method and device, computer equipment and a computer readable storage medium, and a target image is determined based on target scene information and a to-be-processed image.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, apparatus, computer equipment, and computer-readable storage medium. Background Art

[0002] With the rapid development of mobile phone camera technology, people are demanding higher and higher levels of camera intelligence and ease of use. In many shooting scenarios (for example, sports shooting), blurry photos are common. To obtain high-quality images, people typically capture multiple images by continuously pressing the shutter button and selecting the best one from among them. However, related camera devices require users to manually select the best image after capturing multiple images. Summary of the Invention

[0003] Embodiments of the present application provide an image processing method, apparatus, computer device, and computer-readable storage medium.

[0004] In a first aspect, the present application provides an image processing method, comprising:

[0005] Get the image to be processed;

[0006] Perform scene recognition on the image to be processed to obtain target scene information;

[0007] A target image is determined based on the target scene information and the image to be processed.

[0008] In a second aspect, the present application provides an image processing device, comprising:

[0009] An image acquisition module, used for acquiring an image to be processed;

[0010] A scene recognition module is used to perform scene recognition on the image to be processed and obtain target scene information;

[0011] The image determination module is used to determine the target image based on the target scene information and the image to be processed.

[0012] In a third aspect, the present application further provides a computer device, comprising:

[0013] one or more processors;

[0014] Memory; and

[0015] One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement any one of the image processing methods of the first aspect.

[0016] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which is loaded by a processor to execute the steps of any one of the image processing methods in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 is a schematic diagram of a scenario of an image processing system provided by an embodiment of the present invention;

[0019] Figure 2 is a flow chart of an embodiment of an image processing method provided by an embodiment of the present invention;

[0020] Figure 3 This is a flowchart of a specific embodiment of scene recognition provided by an embodiment of the present invention;

[0021] Figure 4 is a flowchart of a specific embodiment of determining a target image provided by an embodiment of the present invention;

[0022] Figure 5 is a flowchart of a specific embodiment of the scene evaluation process provided by an embodiment of the present invention;

[0023] Figure 6 1 is a schematic diagram of key points of a human body in a running scene provided by an embodiment of the present invention;

[0024] Figure 7 This is a scene diagram of key points of a human body in a basketball scene provided by an embodiment of the present invention;

[0025] Figure 8 This is a schematic diagram of key points of a human body in a football scene provided by an embodiment of the present invention;

[0026] Figure 9 This is a schematic diagram of a scene of key points of a human body in a general scenario provided by an embodiment of the present invention;

[0027] Figure 10 is a principle block diagram of an image processing device provided by an embodiment of the present invention;

[0028] Figure 11 It is a schematic structural diagram of an embodiment of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0030] In the description of the present application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first", "second", "third" and the like are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, features defined as "first", "second", "third", etc. may explicitly or implicitly include one or more of the said features. In the description of the present application, "multiple" means two or more, unless otherwise clearly and specifically defined.

[0031] In this application, the word "exemplary" is used to mean "serving as an example, illustration, or illustration." Any embodiment described in this application as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. The following description is given to enable any person skilled in the art to implement and use the present application. In the following description, details are listed for the purpose of explanation. It should be understood that one of ordinary skill in the art can recognize that the present application can be implemented without using these specific details. In other instances, well-known structures and processes are not elaborated in detail to avoid obscuring the description of the present application with unnecessary details. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in this application.

[0032] It should be noted that since the method of the embodiment of the present application is executed in a computer device, the processing objects of each computer device exist in the form of data or information. For example, time is actually time information. It can be understood that if size, quantity, position, etc. are mentioned in subsequent embodiments, the corresponding data exist for the computer device to process. The details will not be repeated here.

[0033] The embodiments of the present application provide an image processing method, apparatus, computer device, and storage medium, which are described in detail below.

[0034] See also Figure 1 , Figure 1 This is a schematic diagram of a scene of an image processing system provided in an embodiment of the present application. The image processing system may include a computer device 100, in which an image processing device is integrated, such as Figure 1 Computer equipment in.

[0035] In the embodiment of the present application, the computer device 100 is mainly used to obtain the image to be processed; perform scene recognition on the image to be processed to obtain target scene information; determine the target image based on the target scene information and the image to be processed, and automatically filter out the best image from the image to be processed based on the target scene information.

[0036] In the embodiments of the present application, the computer device 100 may be an independent server, or a server network or server cluster composed of servers. For example, the computer device 100 described in the embodiments of the present application includes, but is not limited to, a computer, a network host, a single network server, a set of multiple network servers, or a cloud server composed of multiple servers. A cloud server is composed of a large number of computers or network servers based on cloud computing.

[0037] It is understood that the computer device 100 used in the embodiments of the present application can be a device that includes both receiving and transmitting hardware, that is, a device that has receiving and transmitting hardware capable of performing two-way communication over a two-way communication link. Such a device may include: a cellular or other communication device that has a single-line display, a multi-line display, or a cellular or other communication device without a multi-line display. The specific computer device 100 can be a desktop terminal or a mobile terminal. The computer device 100 can also be a mobile phone, a tablet computer, a laptop computer, etc.

[0038] Those skilled in the art will understand that Figure 1 The application environment shown in the figure is only one application scenario of the present application solution and does not constitute a limitation on the application scenario of the present application solution. Other application environments may also include Figure 1 More or fewer computer devices as shown in Figure 1 Only one computer device is shown in the figure. It can be understood that the image processing system can also include one or more other services, which are not limited here.

[0039] In addition, if Figure 1As shown, the image processing system may further include a memory 200 for storing data, such as image data, such as the image to be processed, the first cropped image, the second cropped image, etc., and evaluation information, such as scene evaluation result information, target evaluation result information, etc.

[0040] It should be noted that Figure 1 The scene diagram of the image processing system shown is only an example. The image processing system and scene described in the embodiment of the present application are intended to more clearly illustrate the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided by the embodiment of the present application. Ordinary technicians in this field can know that with the evolution of the image processing system and the emergence of new business scenarios, the technical solution provided by the embodiment of the present application is also applicable to similar technical problems.

[0041] First, an image processing method is provided in an embodiment of the present application. The executor of the image processing method is an image processing device, and the image processing device is applied to a computer device. The image processing method includes: obtaining an image to be processed; performing scene recognition on the image to be processed to obtain target scene information; and determining the target image based on the target scene information and the image to be processed.

[0042] like Figure 2 FIG. 2 is a flow chart of an embodiment of an image processing method in an embodiment of the present application. The image processing method may include the following steps S201 to S203, which are specifically as follows:

[0043] S201: Obtain an image to be processed.

[0044] Taking into account that photos often become blurred when taking pictures, for example, when taking pictures of children running on the road, the children's hands and / or feet often become blurred, in order to obtain images with better image quality, people usually press the shutter button continuously to capture multiple images. The images to be processed can be multiple images captured by the user by continuously pressing the shutter button. Of course, the images to be processed can also be multiple images captured by the user at different times. But in general, in the application scenarios of the present application, the images to be processed are multiple images, not a single image.

[0045] Specifically, the image to be processed may be an image acquired by an imaging module configured by the computer device itself, or an image acquired by an imaging module of another computer device via a network, Bluetooth, infrared, etc. For example, when the image processing method of the present application is applied to a smartphone, the smartphone may directly acquire the image to be processed via its own imaging module; when the image processing method of the present application is applied to a server, the server may acquire the image to be processed via the smartphone's imaging module and obtain the image to be processed from the smartphone via a network, Bluetooth, infrared, etc.

[0046] S202: Perform scene recognition on the image to be processed to obtain target scene information.

[0047] The scene refers to the shooting scene of the image to be processed, more specifically, the sports scene of the target person in the image to be processed. Sports scenes include but are not limited to jumping scenes, basketball scenes, football scenes, running scenes, smiling scenes, and general scenes. The target person is the target object being photographed. For example, in a photo of a child running on a road, the target person is the child, and the sports scene is running. The target scene information is the sports scene corresponding to the identified image to be processed. For example, the target scene information can be one or more of jumping scenes, basketball scenes, football scenes, running scenes, smiling scenes, and general scenes.

[0048] The image to be processed contains multiple images. When performing scene recognition on the image to be processed, scene recognition can be performed on the image to be processed based on a multi-frame method, or based on a single image. Considering that existing multi-frame-based scene recognition methods such as Two stream CNN, Temporal Segments Networks (TSN), 3-Dimensional Convolution (C3D) and Coupled Recurrent Network (CRN) have strict requirements on the length of video clips, which is not conducive to image scoring in continuous shooting mode, this embodiment performs scene recognition on the image to be processed based on a single image.

[0049] In one embodiment, Figure 3 As shown, the above step S202 performs scene recognition on the image to be processed to obtain target scene information corresponding to the image to be processed, which may include the following steps S301 to S304, specifically as follows:

[0050] S301: Perform target detection on the image to be processed to obtain first scene information.

[0051] Target detection refers to the detection of preset targets in the image to be processed. The preset targets can be set as needed. In one specific embodiment, the preset targets include basketballs and / or soccer balls. That is, target detection is used to detect whether the image to be processed contains basketballs and / or soccer balls. The first scene information is the scene information of the image to be processed determined based on the target detection results. For example, when target detection is used to detect whether the image to be processed contains basketballs or soccer balls, the first scene information includes scenes in which the image to be processed contains basketballs (i.e., basketball scenes), scenes in which the image to be processed contains soccer balls (i.e., soccer scenes), and scenes in which the image to be processed contains neither basketballs nor soccer balls (i.e., general scenes).

[0052] In a specific embodiment, the step of performing target detection on the image to be processed to obtain the first scene information includes: inputting the image to be processed into a target detection model to obtain a target detection result; and determining the first scene information based on the target detection result.

[0053] The target detection model can be obtained by training a preset first network model using a preset first dataset. The first dataset can be data collected by the user and integrated with open source datasets (such as coco2017, openimages dataset, etc.). The data collected by the user can be pre-labeled using the Grounding-Dino large model. The target detection model (i.e., the first network model) can adopt the Nanodet framework and quantize the encoder and decoder. After the target detection model used in this embodiment is converted to tflite format, the model size is 1.8M. In the case of an input of 320*320*3, the inference time is 25ms (MTK6833 Device).

[0054] In a specific embodiment, the first scene information includes a third scene, a fourth scene, and a fifth scene, wherein the third scene is a scene in which the image to be processed contains the first target, the fourth scene is a scene in which the image to be processed contains the second target, and the fifth scene is a scene in which the image to be processed contains neither the first target nor the second target. Determining the first scene information based on the target detection result includes: determining a first frame number, a second frame number, and a third frame number based on the target detection result, wherein the first frame number is the number of image frames in the image to be processed that contain the first target, the second frame number is the number of image frames in the image to be processed that contain the second target, and the third frame number is the number of image frames in the image to be processed that do not contain the first target or the second target; if the first frame number is greater than a preset first number threshold, determining the first scene information is the third scene; if the second frame number is greater than a preset second number threshold, determining the first scene information is the fourth scene; and if the third frame number is greater than a preset third number threshold, determining the first scene information is the fifth scene. The first number threshold, the second number threshold, and the third number threshold can be set as needed, and the first number threshold, the second number threshold, and the third number threshold can be set the same, or different.

[0055] For example, the first target is a basketball and the second target is a football. If it is detected that 8 out of 10 images to be processed contain a basketball, the first scene information is determined to be a basketball scene; if it is detected that 8 out of 10 images to be processed contain a football, the first scene information is determined to be a football scene; if it is detected that 8 out of 10 images to be processed contain neither a basketball nor a football, the first scene information is determined to be a general scene.

[0056] S302: Perform action classification on the image to be processed to obtain second scene information.

[0057] Action classification refers to classifying the actions of a target person in the image to be processed. The target person is the subject being photographed, and the target person's actions include, but are not limited to, jumping and running. The second scene information is scene information determined based on the action classification results of the image to be processed. In one specific embodiment, the target person's actions include jumping, running, and other. The second scene information includes scenes where the target person's action is jumping (i.e., jumping scenes), scenes where the target person's action is running (i.e., running scenes), and scenes where the target person's action is other than jumping (i.e., general scenes).

[0058] In a specific embodiment, the step of performing action classification on the image to be processed to obtain the second scene information specifically includes:

[0059] (1) cropping the image to be processed based on the object frame information corresponding to the image to be processed to obtain a second cropped image;

[0060] (2) inputting the second cropped image into the action classification model to obtain action classification information;

[0061] (3) The action classification information is filtered and processed based on a first offset and a second offset corresponding to the image to be processed to obtain second scene information; the first offset is the cumulative horizontal offset of the object frame corresponding to the image to be processed, and the second offset is the cumulative vertical offset of the object frame corresponding to the image to be processed.

[0062] The object frame information is the coordinate information of the human body detection frame of the target person in the image to be processed. The target person is the target object being photographed. The object frame information can be obtained by inputting the image to be processed into the target detection model in the aforementioned step 301. For details, please refer to the discussion in the aforementioned step 301. To avoid repetition, this application will not go into details here.

[0063] The second cropped image is a human body image corresponding to each image to be processed, which is obtained by cropping each image to be processed based on the object frame information corresponding to each image to be processed. By inputting the human body image corresponding to the image to be processed (i.e., the second cropped image) into the action classification model, the action classification information corresponding to the image to be processed can be obtained.

[0064] The action classification model can be obtained by training a preset second network model using a preset second data set. The second data set can be a data set obtained by filtering data consisting of "jumping", "running", and "walking" data crawled from various networks, collected LAION-5B related data (mainly trained with a text-to-image model), and video data collected by the user himself. The action classification model (i.e., the second network model) can use an improved Mobilenetv3-small model. After the action classification model used in this embodiment is converted to tflite format, the model size is 170KB. In the case of an input of 96*96*3, the inference time is 3ms (MTK6833 Device).

[0065] Furthermore, the second network model can be trained using the following loss function: Loss(x, class) = weight[class](-x[class]+log(Σ j exp(x[j]))), where number[class] represents the number of samples corresponding to each category, log1p(x)=ln(x+1), x[class] represents the confidence of the model output corresponding to the true category, and x[j] represents the confidence of the model output corresponding to the jth category.

[0066] The first offset is the cumulative horizontal offset of the object frame corresponding to the image to be processed, which is the accumulated value of the horizontal offsets of the object frames corresponding to adjacent images to be processed in multiple images to be processed. The second offset is the cumulative vertical offset of the object frame corresponding to the image to be processed, which is the accumulated value of the vertical offsets of the object frames corresponding to adjacent images to be processed in multiple images to be processed. For example, if the object frame is a human body detection frame, the first offset is the cumulative horizontal offset of the human body detection frame corresponding to the image to be processed, and the second offset is the cumulative vertical offset of the human body detection frame corresponding to the image to be processed.

[0067] In order to solve the problem that the existing multi-frame-based scene recognition method has strict requirements on the length of video clips, which is not conducive to image scoring in continuous shooting mode, this embodiment performs scene recognition on the image to be processed based on a single image. However, due to the failure to utilize inter-frame information and the limitations of training data and model size, the action classification information based on a single image may have false detection or missed detection. In order to improve the accuracy of the action classification results, after obtaining the action classification information through the action classification model, this embodiment further obtains the first offset and the second offset corresponding to the image to be processed, and determines the second scene information in combination with the first offset, the second offset and the action classification information, which can improve the accuracy of scene recognition.

[0068] In a specific embodiment, the second scene information includes a first scene and / or a second scene, wherein the first scene is a scene in which the target person in the image to be processed has an action type of the first type, and the second scene is a scene in which the target person in the image to be processed has an action type of the second type, wherein the first type and the second type are different action types, for example, the first type is jumping, the second type is running, the first scene is a jumping scene, and the second scene is a running scene. The above-mentioned step of filtering and processing the action classification information based on the offset information corresponding to the image to be processed to obtain the second scene information includes: determining a first candidate scene based on the action classification information; if the first candidate scene is the first scene and the second offset is greater than a preset first offset threshold, determining the second scene information to be the first scene; if the first candidate scene is the first scene and the second offset is not greater than the preset first offset threshold or the first offset is greater than the preset second offset threshold, determining the second scene information to be the second scene. The first offset threshold and the second offset threshold can be set as needed, for example, the first offset threshold can be set to 0.3 times the height of a human body, and the second offset threshold can be set to 0.3 times the width of a human body.

[0069] Furthermore, the step of determining the first candidate scene based on the action classification information includes: determining a fourth frame number, a fifth frame number, and a sixth frame number based on the action classification information, wherein the fourth frame number is the number of image frames in which the image to be processed is classified as jumping, the fifth frame number is the number of image frames in which the image to be processed is classified as running, and the sixth frame number is the number of image frames in which the image to be processed is classified as other; if the fourth frame number is greater than a preset fourth number threshold, determining the first candidate scene as the first scene; if the fifth frame number is greater than a preset fifth number threshold, determining the first candidate scene as the second scene; and if the sixth frame number is greater than a preset sixth number threshold, determining the first candidate scene as the fifth scene. The fourth number threshold, the fifth number threshold, and the sixth number threshold can be set as needed, and the fourth number threshold, the fifth number threshold, and the sixth number threshold can be set the same, or the fourth number threshold, the fifth number threshold, and the sixth number threshold can be set different.

[0070] For example, if it is detected that 8 out of 10 images to be processed are classified as jumping, the first candidate scene is determined to be a jumping scene; if it is detected that 8 out of 10 images to be processed are classified as running, the first candidate scene is determined to be a running scene; if it is detected that 8 out of 10 images to be processed are classified as others, the first candidate scene is determined to be a general scene.

[0071] S303: Perform expression classification on the image to be processed to obtain third scene information.

[0072] Expression classification refers to classifying the facial expression of a target person in the image to be processed. The target person is the subject being photographed. The classification results for the target person's facial expression include, but are not limited to, smiling and not smiling. Third scene information is scene information determined based on the expression classification results of the image to be processed. In one specific embodiment, the target person's expression classification results include smiling and not smiling. The third scene information includes scenes where the target person's facial expression is smiling (i.e., smiling scenes) and scenes where the target person's facial expression is not smiling (i.e., common scenes).

[0073] In a specific embodiment, the step of performing expression classification on the image to be processed and obtaining the third scene information includes: inputting the image to be processed into a face frame detection model to obtain face frame information corresponding to the image to be processed; cropping the image to be processed based on the face frame information to obtain a face image corresponding to the image to be processed; inputting the face image into an expression classification model to obtain an expression classification result corresponding to the image to be processed; and determining the third scene information based on the expression classification result.

[0074] Among them, after the face frame detection model used in this embodiment is converted to tflite format, the model size is 1.6M. Under the condition of input size of 256*256*3, the model inference time is 20ms (MTK6833 device).

[0075] In a specific embodiment, the third scene information includes a smiling scene and a general scene. The step of determining the third scene information based on the expression classification result includes: determining a seventh frame number and an eighth frame number based on the expression classification result, wherein the seventh frame number is the number of image frames in the image to be processed that are classified as smiling, and the eighth frame number is the number of image frames in the image to be processed that are classified as not smiling; if the seventh frame number exceeds a preset seventh number threshold, the third scene information is determined to be a smiling scene; if the eighth frame number exceeds a preset eighth number threshold, the third scene information is determined to be a general scene. The seventh number threshold and the eighth number threshold can be set as needed, and the seventh number threshold and the eighth number threshold can be set to the same, or the seventh number threshold and the eighth number threshold can be set to different.

[0076] For example, if 8 out of 10 images to be processed are classified as smiling, the third scene information is determined to be a smiling scene; if 8 out of 10 images to be processed are classified as not smiling, the third scene information is determined to be a general scene.

[0077] It should be noted that this embodiment does not limit the execution order of the above-mentioned steps S301, S302, and S303. That is to say, the above-mentioned steps S301, S302, and S303 can be executed in any order. For example, step S301 can be executed first, then step S302, and finally step S303; for another example, step S302 can be executed first, then step S301, and finally step S303; for another example, step S303 can be executed first, then step S302, and finally step S301, etc.

[0078] S304: Determine target scene information based on the first scene information, the second scene information, and the third scene information.

[0079] The first scene information, the second scene information and the third scene information can be the same scene information or different scene information. For example, the first scene information, the second scene information and the third scene information are all universal scenes. For example, the first scene information is a basketball scene, the second scene information is a jumping scene, and the third scene information is a smiling scene.

[0080] When the first scene information, the second scene information, and the third scene information are the same scene information, the same scene information can be directly determined as the target scene information. When the first scene information, the second scene information, and the third scene information are not completely the same scene information, the step of determining the target scene information based on the first scene information, the second scene information, and the third scene information may include: obtaining pre-set scene priority information; and determining the target scene information based on the scene priority information, the first scene information, the second scene information, and the third scene information. For example, the scene priority information is jumping>basketball=football>running>smiling>general, the first scene information is a basketball scene, the second scene information is a jumping scene, and the third scene information is a smiling scene, then the target scene information is a jumping scene. That is to say, when the first scene information, the second scene information, and the third scene information are not completely the same scene information, the target scene information is the scene information with the highest priority among the first scene information, the second scene information, and the third scene information.

[0081] S203: Determine a target image based on the target scene information and the image to be processed.

[0082] The target image is an optimal image selected from the image to be processed based on the target scene information. When the target image is determined based on the target scene information and the image to be processed, the target image can be directly selected from the image to be processed based on the target scene information, or the target image can be selected from the image to be processed in combination with the target scene information and other information.

[0083] Considering that in sports photography scenes, the local area or the whole body moves too fast (such as "hand shooting", "foot shooting", "running", etc.), which will cause motion blur, in order to be able to filter out the best image from the images to be processed, this embodiment combines the target scene information, blur information and expression score information to determine the target image from the images to be processed. Figure 4 As shown, the above step S203 determines the target image based on the target scene information and the image to be processed, which may include the following steps S401 to S403, specifically as follows:

[0084] S401 : Perform scene evaluation processing on the image to be processed based on target scene information to obtain scene evaluation result information.

[0085] In order to select the best image from the images to be processed, this embodiment designs different scene evaluation mechanisms for different sports scenes. After obtaining the target scene information corresponding to the image to be processed, the image to be processed is subjected to scene evaluation processing based on the scene evaluation mechanism corresponding to the identified target scene information, thereby obtaining scene evaluation result information corresponding to the image to be processed. For example, jumping scenes are mainly scored based on the vertical height of the human body. After determining that the target scene information is a jumping scene, the human height information corresponding to the image to be processed is determined, and the scene evaluation processing of the image to be processed is performed based on the human height information. For another example, running scenes are mainly scored based on the angle between the two legs. After determining that the target scene information is a running scene, the angle between the two legs of the target person in the image to be processed is determined, and the scene evaluation processing of the image to be processed is performed based on the angle between the two legs.

[0086] When this embodiment performs scene evaluation processing on the image to be processed based on the target scene information, scene evaluation processing can be performed on all images in the image to be processed based on the target scene information to obtain scene evaluation result information corresponding to each image to be processed, or scene evaluation processing can be performed on part of the images in the image to be processed based on the target scene information to obtain scene evaluation result information corresponding to part of the images in the image to be processed.

[0087] S402. Determine target evaluation result information corresponding to each candidate image based on blur information, expression score information, and scene evaluation result information corresponding to a plurality of candidate images; the plurality of candidate images are at least two images among the images to be processed.

[0088] The blur information is used to characterize the blur degree corresponding to each candidate image. Specifically, the blur information is obtained based on the following steps:

[0089] (1) cropping each candidate image based on object key point information corresponding to each of the plurality of candidate images to obtain a plurality of first cropped images corresponding to each candidate image;

[0090] (2) inputting the plurality of first cropped images into a blur detection model to obtain blur score information corresponding to each first cropped image;

[0091] (3) Based on the fuzzy score information, determine the fuzziness information corresponding to each candidate image.

[0092] Among them, the object key point information corresponding to multiple candidate images is obtained based on the following steps: each candidate image is cropped based on the object detection frame information corresponding to each candidate image to obtain the object image corresponding to each candidate image; the object image is input into the object key point detection model to obtain the object key point information corresponding to each candidate image.

[0093] In a specific embodiment, the object key point information is human key point information. When cropping each candidate image based on the object key point information, images of key areas such as the head, hands, and feet of the human body can be cropped to obtain multiple first cropped images, and then the multiple first cropped images are input into a blur detection model for motion blur detection. The blur detection model can be obtained by training a preset third network model using a preset third data set. The blur detection model (i.e., the third network model) can use a twin network comprising two branches. The third data set includes multiple first images and a second image corresponding to each first image. The resolution of each first image is higher than the resolution of its corresponding second image. The second image can be obtained by randomly adding different degrees of Gaussian blur and motion blur to its corresponding first image.

[0094] Accordingly, before inputting multiple first cropped images into the blur detection model and obtaining the blur score information corresponding to each first cropped image, the method includes: obtaining a third data set, the third data set including multiple first images and a second image corresponding to each first image, and the resolution of each first image is higher than the resolution of the second image corresponding to each first image; inputting the first image and the second image into two branches of a preset third network model respectively, and outputting the first blur information corresponding to the first image and the second blur information corresponding to the second image respectively through the two branches; and training the third network model based on the first blur information, the second blur information and the loss function of the third network model to obtain the blur detection model.

[0095] Among them, the loss function of the third network model can be expressed as: loss = max(0, δ-Φ(I clear ,θ)+Φ(I blur ,θ)),I clear and I blurrepresent the first image and the second image respectively, Φ and θ represent the third network model and its corresponding model parameters respectively, δ is a fixed constant margin, δ can be set as needed, for example, δ can be set to 1.

[0096] After obtaining the blur detection model, multiple first cropped images can be input into any branch of the blur detection model to obtain blur score information corresponding to each first cropped image.

[0097] When the target scene information is the third scene (basketball scene) or the fourth scene (football scene), the step of determining the blur information corresponding to each candidate image based on the blur score information includes: obtaining a third cropped image corresponding to each candidate image, where the third cropped image is a cropped image of the area where the basketball or football is located in the candidate image; inputting the third cropped image into the blur detection model to obtain a sphere blur score corresponding to each candidate image; comparing the blur score information and the sphere blur score with a preset score threshold respectively; when the blur score information and the sphere blur score are both greater than the score threshold, determining the preset score information as the blur information corresponding to each candidate image; when the blur score information and the sphere blur score are not both greater than the score threshold, determining the average of the blur score information and the sphere blur score as the blur information corresponding to each candidate image.

[0098] When the target scene information is not the third scene and the fourth scene, that is, the target scene information is the first scene, the second scene or the fifth scene, the step of determining the blur information corresponding to each candidate image based on the blur score information includes: comparing the blur score information with a preset score threshold; when the blur score information is greater than the score threshold, determining the preset score information as the blur information corresponding to each candidate image; when the blur score information is not greater than the score threshold, determining the average of the blur score information as the blur information corresponding to each candidate image.

[0099] The process of determining the ambiguity information can be expressed as: score[blur] represents blur information, blur_scores represents blur score information and / or sphere blur score, blur_th represents score threshold, R represents preset score information, and R can be set as needed, for example, R=1.0.

[0100] The expression score information can be determined based on the expression classification results corresponding to each candidate image output by the expression classification model, or the confidence of each candidate image output by the expression classification model under the preset expression category can be directly determined as the expression score information corresponding to each candidate image.

[0101] In one specific embodiment, the confidence level of each candidate image output by the expression classification model under a preset expression category is directly determined as the expression score information corresponding to each candidate image. Accordingly, the step of obtaining the expression score information corresponding to each of the multiple candidate images includes: inputting the facial image corresponding to each candidate image into the expression classification model, outputting the confidence level of each candidate image under the preset expression category through the expression classification model; and determining the confidence level under the preset expression category as the expression score information corresponding to each candidate image. The preset expression category can be set according to user needs. For example, if the preset expression category is smile, the smile confidence level corresponding to each candidate image output by the expression classification model is determined as the expression score information corresponding to each candidate image.

[0102] Specifically, based on the scene evaluation result information, blur information, and expression score information, the step of determining the target evaluation result information corresponding to each candidate image includes: obtaining first weight information corresponding to the scene evaluation result information, second weight information corresponding to the blur information, and third weight information corresponding to the expression score information; fusing the scene evaluation result information, blur information, and expression score information based on the first weight information, the second weight information, and the third weight information to obtain the target evaluation result information corresponding to each candidate image. The calculation process of the target evaluation result information can be expressed as: score = c*score[i]+d*score[blur]+e*score[smile], where score represents the target evaluation result information, score[i] represents the scene evaluation result information when the target scene information is the i-th scene, score[blur] represents the blur information, score[smile] represents the expression score information, c represents the first weight information, d represents the second weight information, and e represents the third weight information. For example, when the target scene information is the first scene, the calculation process of the target evaluation result information can be expressed as: score = c*score[1]+d*score[blur]+e*score[smilt]; when the target scene information is the second scene, the calculation process of the target evaluation result information can be expressed as: score = c*score[2]+d*score[blur]+e*score[smile].

[0103] S403: Determine a target image from multiple candidate images based on the target evaluation result information.

[0104] The target evaluation result information includes a target evaluation score. The target image is the image with the highest target evaluation score among multiple candidate images. When determining the target image from multiple candidate images based on the target evaluation result information, the multiple candidate images can be sorted in descending order based on the target evaluation scores, and the candidate image with the highest ranking can be selected as the target image. Alternatively, the multiple candidate images can be sorted in descending order based on the target evaluation scores, and the candidate image with the lowest ranking can be selected as the target image. The image processing method provided in this embodiment automatically filters out the best image from multiple images to be processed through a computer device, without the user having to manually select a best image from multiple images, which can improve the user experience.

[0105] In a specific embodiment, the scene evaluation result information includes at least one target scene evaluation result information. In order to quickly filter the best image from the image to be processed, scene evaluation processing is performed on part of the image to be processed based on the target scene information. Accordingly, Figure 5 As shown, the above step S401 performs scene evaluation processing on the image to be processed based on the target scene information to obtain scene evaluation result information, which may include the following steps S501 to S502, which are specifically as follows:

[0106] S501: Filter the image to be processed based on target scene information to obtain candidate image information.

[0107] Specifically, the candidate image information includes at least one candidate image, and the target scene information includes a first scene, and / or a second scene, and / or a third scene, and / or a fourth scene, and / or a fifth scene, wherein the first scene is a scene in which the action type of the target person in the image to be processed is the first type, the second scene is a scene in which the action type of the target person in the image to be processed is the second type, the third scene is a scene in which the image to be processed contains the first target, the fourth scene is a scene in which the image to be processed contains the second target, and the fifth scene is other scenes except the first scene, the second scene, the third scene and the fourth scene. The first type and the second type are different action types, and the first target and the second target are different target objects. For example, the first type is jumping, the second type is running, the first target is basketball, and the second target is football. Then, the first scene is a jumping scene, the second scene is a running scene, the third scene is a basketball scene, the fourth scene is a football scene, and the fifth scene is other scenes except the jumping scene, the running scene, the basketball scene, and the football scene. The above-mentioned step of filtering the image to be processed based on the target scene information to obtain the candidate image information includes:

[0108] (1) If the target scene information is the first scene, perform frame filtering on the image to be processed, filter out the frames that are adjacent to the edge, then obtain object frame information corresponding to each image to be processed, determine object height information corresponding to each image to be processed based on the object frame information, and determine candidate image information from multiple images to be processed based on the object height information, wherein the candidate image information includes at least one candidate image, and the at least one candidate image is an image with a higher human body height ranking among the multiple images to be processed; for example, the at least one candidate image is six images with a higher human body height ranking among the multiple images to be processed;

[0109] (2) If the target scene information is the second scene, perform frame filtering on the image to be processed, filter out the frames that are adjacent to the edge, and then obtain object frame information corresponding to each image to be processed, and determine candidate image information from the multiple images to be processed based on the object frame information, wherein the candidate image information includes at least one candidate image, and the at least one candidate image is an image with a human body detection frame in the middle of the multiple images to be processed; for example, the at least one candidate image is the first six images with a human body detection frame in the middle of the multiple images to be processed;

[0110] (3) If the target scene information is not the first scene and the second scene (i.e., the target scene information is the third scene, the fourth scene, or the fifth scene), frame filtering is performed on the image to be processed to filter out the frames that are adjacent to the edge, and then candidate image information is sampled from the multiple images to be processed, wherein the candidate image information includes at least one candidate image, and the at least one candidate image can be an image uniformly sampled from the image to be processed, or can be an image randomly sampled from the image to be processed. For example, the at least one candidate image is 6 images uniformly sampled from the multiple images to be processed.

[0111] S502: Perform scene evaluation processing on any candidate image in the candidate image information based on the target scene information to obtain target scene evaluation result information corresponding to the candidate image.

[0112] Specifically, the target scene information includes a first scene, and / or a second scene, and / or a third scene, and / or a fourth scene, and / or a fifth scene, wherein the first scene is a scene in which the action type of the target person in the image to be processed is the first type, the second scene is a scene in which the action type of the target person in the image to be processed is the second type, the third scene is a scene in which the image to be processed contains the first target, the fourth scene is a scene in which the image to be processed contains the second target, and the fifth scene is other scenes except the first scene, the second scene, the third scene and the fourth scene. The first type and the second type are different action types, and the first target and the second target are different target objects. For example, the first type is jumping, the second type is running, the first target is basketball, and the second target is football. Then, the first scene is a jumping scene, the second scene is a running scene, the third scene is a basketball scene, the fourth scene is a football scene, and the fifth scene is other scenes except the jumping scene, the running scene, the basketball scene, and the football scene. The above-mentioned step of performing scene evaluation processing on any candidate image in the candidate image information based on the target scene information to obtain target scene evaluation result information corresponding to the candidate image includes:

[0113] (1) For any candidate image in the candidate image information, if the target scene information is the first scene, the object height information corresponding to the candidate image is determined based on the object frame information corresponding to the candidate image, and the candidate image is subjected to scene evaluation processing based on the object height information to obtain the target scene evaluation result information corresponding to the candidate image.

[0114] Among them, when the target scene information is the first scene, the target scene evaluation result information can be expressed as Indicates the target scene evaluation result information when the target scene information is the first scene, index hight Indicates the object height ranking corresponding to the candidate images. The higher the object height ranking corresponding to the candidate images, the higher the score corresponding to the target scene evaluation result information. For example, the object height information is human height information. Based on the human height information, the human height rankings corresponding to the 6 candidate images are determined to be 1, 2, 3, 4, 5, and 6. The corresponding target scene evaluation result information is respectively 0.

[0115] (2) If the target scene information is the second scene, the first angle information and the second angle information are determined based on the object key point information corresponding to the candidate image, and the candidate image is subjected to scene evaluation processing based on the first angle information and the second angle information to obtain the target scene evaluation result information corresponding to the candidate image.

[0116] The first angle information is the angle information between the thigh and calf of one leg of the target person, and the second angle information is the angle information between the thigh and calf of the other leg of the target person. Figure 6 As shown, the first angle information can be the angle between the key points "8-10-12" (i.e., the angle information between the thigh and calf of the target person's left leg), and the second angle information can be the angle between the key points "13-11-9" (i.e., the angle information between the thigh and calf of the target person's right leg). When the target scene information is the second scene, the target scene evaluation result information can be expressed as: score[2]=||angle10-angle11||, score[2] represents the target scene evaluation result information when the target scene information is the second scene, angle10 represents the first angle information, and angle11 represents the second angle information.

[0117] (3) If the target scene information is the third scene, determine the first distance information corresponding to the candidate image based on the object key point information corresponding to the candidate image, perform scene evaluation processing on the candidate image based on the first distance information, and obtain the target scene evaluation result information corresponding to the candidate image.

[0118] The third scene is a basketball scene, and the first distance information is the distance between the basketball and the hand in each candidate image. For example, Figure 7 As shown, the first distance information is the distance from the key point 6 or 7 of the hand to the basketball. When the target scene information is the third scene, the target scene evaluation result information can be expressed as: score[3] = ||position-ball-keypoint6 / 7||, score[3] represents the target scene evaluation result information when the target scene information is the third scene, and keypoint6 / 7 represents the key point of the hand.

[0119] (4) If the target scene information is the fourth scene, the first angle information, the second angle information and the second distance information are determined based on the object key point information corresponding to the candidate image, and the candidate image is subjected to scene evaluation processing based on the first angle information, the second angle information and the second distance information to obtain the target scene evaluation result information corresponding to the candidate image.

[0120] The fourth scene is a football scene. The first angle information is the angle information between the thigh and calf of one leg of the target person, the second angle information is the angle information between the thigh and calf of the other leg of the target person, and the second distance information is the distance between the football and the foot in each candidate image. For example, referring to Figure 8As shown, the second distance information is the distance from the key point 12 or 13 of the leg to the soccer ball. In one specific embodiment, scene evaluation processing is performed on the candidate image based on the first angle information, the second angle information, and the second distance information to obtain target scene evaluation result information corresponding to the candidate image. The process includes: determining first score information based on the first angle information and the second angle information; determining second score information based on the second distance information; and determining target scene evaluation result information corresponding to the candidate image based on the first score information and the second score information.

[0121] Among them, when the target scene information is the fourth scene, the target scene evaluation result information can be expressed as: score[4]=a*score

[41] +b*score

[42] , score

[41] =||angle10-angle11||, score

[42] =||position-ball-keypoint12 / 13||, score[4] represents the target scene evaluation result information when the target scene information is the fourth scene, score

[41] represents the first score information, score

[42] represents the second score information, angle10 represents the first angle information, angle11 represents the second angle information, keypoint12 / 13 represents the key points of the legs, a and b represent the corresponding weight factors, and a and b can be set as needed.

[0122] (5) If the target scene information is the fifth scene, the third angle information, the fourth angle information and the fifth angle information are determined based on the object key point information corresponding to the candidate image, and the candidate image is subjected to scene evaluation processing based on the third angle information, the fourth angle information and the fifth angle information to obtain the target scene evaluation result information corresponding to the candidate image.

[0123] The third angle information is the angle between the two legs of the target person in the candidate image. The fourth angle information and the fifth angle information are the angles between the two arms and the body of the target person in the candidate image. For example, Figure 9 As shown, the third angle information is angle θ1, the fourth angle information is angle θ2, and the fifth angle information is angle θ3. When the target scene information is the fifth scene, the target scene evaluation result information can be expressed as: score[5]=θ1+θ2+θ3, and score[5] represents the target scene evaluation result information when the target scene information is the fifth scene. Generally speaking, the larger the third angle information, the fourth angle information, and the fifth angle information, the higher the score corresponding to the target scene evaluation result information.

[0124] In summary, the image processing method provided by this embodiment obtains an image to be processed, performs scene recognition on the image to be processed, obtains target scene information corresponding to the image to be processed, performs scene evaluation processing on multiple candidate images in the image to be processed based on the target scene information, obtains scene evaluation result information corresponding to each candidate image, determines target evaluation result information corresponding to each candidate image based on the scene evaluation result information, blur information and expression score information, and determines the target image from multiple candidate images based on the target evaluation result information. In this solution, the target image is determined based on the target scene information and the image to be processed, and the best image can be automatically filtered out from the image to be processed, avoiding the user from manually selecting the best image from multiple images, thereby improving the user experience. Furthermore, based on the target scene information, scene evaluation processing is performed on the image to be processed to obtain scene evaluation result information, and target evaluation result information is determined based on the scene evaluation result information, blur information and expression score information. Then, the target image is determined from multiple candidate images based on the target evaluation result information. Image filtering can be performed in combination with the motion scene, blur information and expression score information of the image to be processed to improve the accuracy of the filtered target image.

[0125] In order to better implement the image processing method in the embodiment of the present application, based on the image processing method, the embodiment of the present application also provides an image processing device, such as Figure 10 As shown, the image processing device 600 includes:

[0126] An image acquisition module 610 is used to acquire an image to be processed;

[0127] A scene recognition module 620 is used to perform scene recognition on the image to be processed to obtain target scene information;

[0128] The image determination module 630 is configured to determine a target image based on target scene information and the image to be processed.

[0129] In an embodiment of the present application, scene recognition is performed on the image to be processed, and a target image is determined based on the recognized target scene information and the image to be processed. The best image can be automatically filtered out from multiple images to be processed based on the target scene information.

[0130] In some embodiments of the present application, the scene recognition module 620 performs scene recognition on the image to be processed to obtain target scene information, including:

[0131] Performing target detection on the image to be processed to obtain first scene information;

[0132] Perform action classification on the image to be processed to obtain second scene information;

[0133] Perform expression classification on the image to be processed to obtain third scene information;

[0134] Target scene information is determined based on the first scene information, the second scene information, and the third scene information.

[0135] In some embodiments of the present application, the scene recognition module 620 performs action classification on the image to be processed to obtain second scene information, including:

[0136] cropping the image to be processed based on the object frame information corresponding to the image to be processed, obtaining a second cropped image;

[0137] Inputting the second cropped image into the action classification model to obtain action classification information;

[0138] The action classification information is filtered and processed based on a first offset and a second offset corresponding to the image to be processed to obtain second scene information; the first offset is a cumulative horizontal offset of the object frame corresponding to the image to be processed, and the second offset is a cumulative vertical offset of the object frame corresponding to the image to be processed.

[0139] In some embodiments of the present application, the image determination module 630 determines the target image based on the target scene information and the image to be processed, including:

[0140] Performing scene evaluation processing on the image to be processed based on the target scene information to obtain scene evaluation result information;

[0141] Determining target evaluation result information corresponding to each candidate image based on blur information, expression score information, and scene evaluation result information corresponding to the plurality of candidate images; the plurality of candidate images being at least two images among the images to be processed;

[0142] A target image is determined from multiple candidate images based on the target evaluation result information.

[0143] In some embodiments of the present application, the blur information is obtained by the image determination module 630 by performing the following steps:

[0144] cropping each candidate image based on object key point information corresponding to each of the plurality of candidate images to obtain a plurality of first cropped images corresponding to each candidate image;

[0145] Inputting the plurality of first cropped images into a blur detection model to obtain blur score information corresponding to each first cropped image;

[0146] Based on the fuzzy score information, the fuzziness information corresponding to each candidate image is determined.

[0147] In some embodiments of the present application, the expression score information is obtained by the image determination module 630 by performing the following steps:

[0148] Inputting the facial image corresponding to each candidate image into an expression classification model, and outputting the confidence score of each candidate image under a preset expression category through the expression classification model;

[0149] The confidence level under the preset expression category is determined as the expression score information corresponding to each candidate image.

[0150] In some embodiments of the present application, the scene evaluation result information includes at least one target scene evaluation result information. The image determination module 630 performs scene evaluation processing on the image to be processed based on the target scene information to obtain scene evaluation result information including:

[0151] Filtering the image to be processed based on the target scene information to obtain candidate image information; the candidate image information includes at least one candidate image;

[0152] Based on the target scene information, scene evaluation processing is performed on any candidate image in the candidate image information to obtain target scene evaluation result information corresponding to the candidate image.

[0153] In some embodiments of the present application, the target scene information includes a first scene, and / or a second scene, and / or a third scene, and / or a fourth scene, and / or a fifth scene. The first scene is a scene in which the action type of the target person in the image to be processed is the first type, the second scene is a scene in which the action type of the target person in the image to be processed is the second type, the third scene is a scene in which the image to be processed contains the first target, the fourth scene is a scene in which the image to be processed contains the second target, and the fifth scene is a scene other than the first, second, third, and fourth scenes. The first type and the second type are different action types, and the first target and the second target are different target objects. The image determination module 630 performs scene evaluation processing on any candidate image in the candidate image information based on the target scene information to obtain target scene evaluation result information corresponding to the candidate image, including:

[0154] For any candidate image in the candidate image information, if the target scene information is the first scene, determining the object height information corresponding to the candidate image based on the object frame information corresponding to the candidate image;

[0155] Perform scene evaluation processing on the candidate image based on the object height information to obtain target scene evaluation result information corresponding to the candidate image; or

[0156] If the target scene information is the second scene, determining the first angle information and the second angle information based on the object key point information corresponding to the candidate image;

[0157] Performing scene evaluation processing on the candidate image based on the first angle information and the second angle information to obtain target scene evaluation result information corresponding to the candidate image; or

[0158] If the target scene information is the third scene, determining first distance information corresponding to the candidate image based on the object key point information corresponding to the candidate image;

[0159] Performing scene evaluation processing on the candidate image based on the first distance information to obtain target scene evaluation result information corresponding to the candidate image; or,

[0160] If the target scene information is the fourth scene, determining the first angle information, the second angle information, and the second distance information based on the object key point information corresponding to the candidate image;

[0161] Performing scene evaluation processing on the candidate image based on the first angle information, the second angle information, and the second distance information to obtain target scene evaluation result information corresponding to the candidate image; or

[0162] If the target scene information is the fifth scene, determining the third angle information, the fourth angle information, and the fifth angle information based on the object key point information corresponding to the candidate image;

[0163] A scene evaluation process is performed on the candidate image based on the third angle information, the fourth angle information, and the fifth angle information to obtain target scene evaluation result information corresponding to the candidate image.

[0164] The present application also provides a computer device that integrates any one of the image processing devices provided in the present application. The computer device includes:

[0165] one or more processors;

[0166] Memory; and

[0167] One or more applications, wherein the one or more applications are stored in the memory and are configured to cause the processor to execute the steps of the image processing method in any of the above-mentioned image processing method embodiments.

[0168] The embodiment of the present application also provides a computer device that integrates any image processing device provided in the embodiment of the present application. Figure 11 , which shows a schematic diagram of the structure of the computer device involved in the embodiment of the present application, specifically:

[0169] The computer device may include one or more processing core processors 801, one or more computer readable storage media memories 802, a power supply 803, an input unit 804 and other components. Those skilled in the art will understand that Figure 11The computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange components differently.

[0170] Processor 801 is the control center of the computer device. It connects the various components of the entire computer device using various interfaces and lines. By running or executing software programs and / or modules stored in memory 802 and accessing data stored in memory 802, it performs various functions of the computer device and processes data, thereby providing overall monitoring of the computer device. Optionally, processor 801 may include one or more processing cores. Preferably, processor 801 may integrate an application processor and a modem processor, wherein the application processor primarily processes the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into processor 801.

[0171] The memory 802 can be used to store software programs and modules. The processor 801 executes various functional applications and data processing by running the software programs and modules stored in the memory 802. The memory 802 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 802 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory 802 may also include a memory controller to provide the processor 801 with access to the memory 802.

[0172] The computer device also includes a power supply 803 for supplying power to various components. Preferably, the power supply 803 can be logically connected to the processor 801 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The power supply 803 can also include one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and other arbitrary components.

[0173] The computer device may further include an input unit 804, which may be configured to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0174] Although not shown, the computer device may further include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 801 in the computer device will load the executable files corresponding to one or more application processes into the memory 802 according to the following instructions, and the processor 801 will run the application stored in the memory 802 to implement various functions as follows:

[0175] Get the image to be processed;

[0176] Perform scene recognition on the image to be processed to obtain target scene information;

[0177] A target image is determined based on the target scene information and the image to be processed.

[0178] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be accomplished by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0179] To this end, an embodiment of the present application provides a computer-readable storage medium, which may include a read-only memory (ROM), a random access memory (RAM), a disk, or an optical disk. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps of any of the image processing methods provided in the embodiments of the present application. For example, the computer program loaded by the processor may execute the following steps:

[0180] Get the image to be processed;

[0181] Perform scene recognition on the image to be processed to obtain target scene information;

[0182] A target image is determined based on the target scene information and the image to be processed.

[0183] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the detailed description of other embodiments above and will not be repeated here.

[0184] In specific implementation, the above units or structures can be implemented as independent entities, or can be arbitrarily combined to implement as the same or several entities. The specific implementation of the above units or structures can refer to the previous method embodiments and will not be repeated here.

[0185] The specific implementation of the above operations can be found in the previous embodiments and will not be repeated here.

[0186] The above is a detailed introduction to an image processing method, device, computer equipment and computer-readable storage medium provided in the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An image processing method, characterized in that: include: Get the image to be processed; Performing scene recognition on the image to be processed to obtain target scene information; A target image is determined based on the target scene information and the image to be processed.

2. The method according to claim 1, characterized in that The determining of a target image based on the target scene information and the image to be processed includes: Performing scene evaluation processing on the image to be processed based on the target scene information to obtain scene evaluation result information; Determining target evaluation result information corresponding to each candidate image based on blur information, expression score information, and scene evaluation result information corresponding to a plurality of candidate images, wherein the plurality of candidate images are at least two images among the images to be processed; A target image is determined from the plurality of candidate images based on the target evaluation result information.

3. The method according to claim 2, characterized in that The ambiguity information is obtained based on the following steps, including: cropping each candidate image based on object key point information corresponding to each of the plurality of candidate images to obtain a plurality of first cropped images corresponding to each of the candidate images; Inputting the plurality of first cropped images into a blur detection model to obtain blur score information corresponding to each of the first cropped images; Based on the blur score information, blur degree information corresponding to each candidate image is determined.

4. The method according to claim 2, characterized in that The expression score information is obtained based on the following steps, including: Inputting the facial image corresponding to each candidate image into an expression classification model, and outputting the confidence score of each candidate image under a preset expression category through the expression classification model; The confidence level under the preset expression category is determined as the expression score information corresponding to each candidate image.

5. The method according to claim 2, characterized in that The scenario evaluation result information includes at least one target scenario evaluation result information; The performing scene evaluation processing on the image to be processed based on the target scene information to obtain scene evaluation result information includes: Filtering the image to be processed based on the target scene information to obtain candidate image information; the candidate image information includes at least one candidate image; Based on the target scene information, scene evaluation processing is performed on any candidate image in the candidate image information to obtain target scene evaluation result information corresponding to the candidate image.

6. The method according to claim 5, characterized in that The target scene information includes a first scene, and / or a second scene, and / or a third scene, and / or a fourth scene, and / or a fifth scene, the first scene being a scene in which an action type of a target person in the image to be processed is a first type, the second scene being a scene in which an action type of a target person in the image to be processed is a second type, the third scene being a scene in which the image to be processed contains a first target, the fourth scene being a scene in which the image to be processed contains a second target, and the fifth scene being a scene other than the first scene, the second scene, the third scene, and the fourth scene, the first type and the second type are different action types, and the first target and the second target are different target objects; The performing scene evaluation processing on any candidate image in the candidate image information based on the target scene information to obtain target scene evaluation result information corresponding to the candidate image includes: For any candidate image in the candidate image information, if the target scene information is the first scene, determine the object height information corresponding to the candidate image based on the object frame information corresponding to the candidate image; perform scene evaluation processing on the candidate image based on the object height information to obtain target scene evaluation result information corresponding to the candidate image; or If the target scene information is the second scene, determining first angle information and second angle information based on the object key point information corresponding to the candidate image; performing scene evaluation processing on the candidate image based on the first angle information and the second angle information to obtain target scene evaluation result information corresponding to the candidate image; or If the target scene information is the third scene, determining first distance information corresponding to the candidate image based on object key point information corresponding to the candidate image; performing scene evaluation processing on the candidate image based on the first distance information to obtain target scene evaluation result information corresponding to the candidate image; or If the target scene information is the fourth scene, determining first angle information, second angle information, and second distance information based on the object key point information corresponding to the candidate image; performing scene evaluation processing on the candidate image based on the first angle information, the second angle information, and the second distance information to obtain target scene evaluation result information corresponding to the candidate image; or If the target scene information is the fifth scene, the third angle information, the fourth angle information and the fifth angle information are determined based on the object key point information corresponding to the candidate image; the candidate image is subjected to scene evaluation processing based on the third angle information, the fourth angle information and the fifth angle information to obtain the target scene evaluation result information corresponding to the candidate image.

7. The method according to claim 1, characterized in that The performing scene recognition on the image to be processed to obtain target scene information includes: Performing target detection on the image to be processed to obtain first scene information; performing action classification on the image to be processed to obtain second scene information; Performing expression classification on the image to be processed to obtain third scene information; Target scene information is determined based on the first scene information, the second scene information, and the third scene information.

8. The method according to claim 7, characterized in that The performing action classification on the image to be processed to obtain second scene information includes: cropping the image to be processed based on the object frame information corresponding to the image to be processed to obtain a second cropped image; inputting the second cropped image into an action classification model to obtain action classification information; The action classification information is filtered and processed based on a first offset and a second offset corresponding to the image to be processed to obtain second scene information; the first offset is a cumulative lateral offset of the object frame corresponding to the image to be processed, and the second offset is a cumulative longitudinal offset of the object frame corresponding to the image to be processed.

9. An image processing device, characterized in that: include: An image acquisition module, used for acquiring an image to be processed; A scene recognition module, configured to perform scene recognition on the image to be processed to obtain target scene information; An image determination module, configured to determine a target image based on the target scene information and the image to be processed; Preferably, the scene recognition module performs scene recognition on the image to be processed to obtain target scene information, including: Performing target detection on the image to be processed to obtain first scene information; performing action classification on the image to be processed to obtain second scene information; Performing expression classification on the image to be processed to obtain third scene information; Determining target scene information based on the first scene information, the second scene information, and the third scene information; Preferably, the scene recognition module performs action classification on the image to be processed to obtain second scene information, including: cropping the image to be processed based on the object frame information corresponding to the image to be processed to obtain a second cropped image; inputting the second cropped image into an action classification model to obtain action classification information; The action classification information is filtered based on a first offset and a second offset corresponding to the image to be processed to obtain second scene information; the first offset is a cumulative horizontal offset of the object frame corresponding to the image to be processed, and the second offset is a cumulative vertical offset of the object frame corresponding to the image to be processed; Preferably, the image determination module determines the target image based on the target scene information and the image to be processed, including: Performing scene evaluation processing on the image to be processed based on the target scene information to obtain scene evaluation result information; Determining target evaluation result information corresponding to each candidate image based on blur information, expression score information, and scene evaluation result information corresponding to a plurality of candidate images, wherein the plurality of candidate images are at least two images among the images to be processed; determining a target image from the plurality of candidate images based on the target evaluation result information; Preferably, the blur information is obtained by the image determination module by executing the following steps: cropping each candidate image based on object key point information corresponding to each of the plurality of candidate images to obtain a plurality of first cropped images corresponding to each of the candidate images; Inputting the plurality of first cropped images into a blur detection model to obtain blur score information corresponding to each of the first cropped images; Determining blurriness information corresponding to each candidate image based on the blur score information; Preferably, the expression score information is obtained by the image determination module by executing the following steps: Inputting the facial image corresponding to each candidate image into an expression classification model, and outputting the confidence score of each candidate image under a preset expression category through the expression classification model; Determining the confidence level under the preset expression category as expression score information corresponding to each candidate image; Preferably, the scene evaluation result information includes at least one target scene evaluation result information, and the image determination module performs scene evaluation processing on the image to be processed based on the target scene information to obtain scene evaluation result information, including: Filtering the image to be processed based on the target scene information to obtain candidate image information; the candidate image information includes at least one candidate image; Performing scene evaluation processing on any candidate image in the candidate image information based on the target scene information to obtain target scene evaluation result information corresponding to the candidate image; Preferably, the target scene information includes a first scene, and / or a second scene, and / or a third scene, and / or a fourth scene, and / or a fifth scene, the first scene being a scene in which the action type of the target person in the image to be processed is the first type, the second scene being a scene in which the action type of the target person in the image to be processed is the second type, the third scene being a scene in which the image to be processed contains the first target, the fourth scene being a scene in which the image to be processed contains the second target, and the fifth scene being a scene other than the first scene, the second scene, the third scene, and the fourth scene, the first type and the second type are different action types, the first target and the second target are different target objects, and the image determination module performs scene evaluation processing on any of the candidate images in the candidate image information based on the target scene information to obtain target scene evaluation result information corresponding to the candidate image, including: For any candidate image in the candidate image information, if the target scene information is the first scene, determining the object height information corresponding to the candidate image based on the object frame information corresponding to the candidate image; Performing scene evaluation processing on the candidate image based on the object height information to obtain target scene evaluation result information corresponding to the candidate image; or If the target scene information is the second scene, determining first angle information and second angle information based on object key point information corresponding to the candidate image; Performing scene evaluation processing on the candidate image based on the first angle information and the second angle information to obtain target scene evaluation result information corresponding to the candidate image; or If the target scene information is the third scene, determining first distance information corresponding to the candidate image based on object key point information corresponding to the candidate image; Performing scene evaluation processing on the candidate image based on the first distance information to obtain target scene evaluation result information corresponding to the candidate image; or If the target scene information is the fourth scene, determining the first angle information, the second angle information, and the second distance information based on the object key point information corresponding to the candidate image; Performing scene evaluation processing on the candidate image based on the first angle information, the second angle information, and the second distance information to obtain target scene evaluation result information corresponding to the candidate image; or If the target scene information is the fifth scene, determining the third angle information, the fourth angle information, and the fifth angle information based on the object key point information corresponding to the candidate image; Based on the third angle information, the fourth angle information and the fifth angle information, scene evaluation processing is performed on the candidate image to obtain target scene evaluation result information corresponding to the candidate image.

10. A computer device, characterized in that: The computer device comprises: one or more processors; Memory; and One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the image processing method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in the image processing method according to any one of claims 1 to 8.

Images