Image processing method, computer device and readable storage medium

Abstract

Embodiments of the present invention disclose an image processing method, computer device, and readable storage medium. In one specific embodiment, the method includes: obtaining an image captured by an image acquisition device as an original image; performing human figure detection on the original image to obtain a person detection frame; and performing super-resolution processing on the person detection frame to generate a processed image with the same resolution as the original image. This embodiment maximizes the effective information of the image by amplifying information about high-value human areas in images such as video conferencing images and behavior monitoring images, thereby improving the user experience of the information recipient.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and more particularly to an image processing method, a computer device, and a readable storage medium. Background Art

[0002] In scenarios such as remote video conferencing, the image of the entire venue captured by the other party's camera is usually displayed on one party's screen, but generally speaking, this approach is not optimal. For example, if the image covers a large area, but there are only a few areas in the venue where participants exist, then the image covering the entire venue will contain a large amount of redundant information, and the information receiving party will have a poor experience. Currently, the common way to solve this problem is to limit the image to only include the area containing the participants by adjusting the camera's field of view. However, this approach has some problems, such as inconvenient operation and the inability to automatically adjust according to changes such as the movement of participants or the temporary addition of participants during the meeting. Summary of the Invention

[0003] An object of the present invention is to provide an image processing method, a computer device, and a readable storage medium to solve at least one of the problems existing in the prior art.

[0004] In order to achieve the above object, the present invention adopts the following technical solutions:

[0005] A first aspect of the present invention provides an image processing method, comprising:

[0006] Acquire an image captured by an image acquisition device as an original image;

[0007] Performing human figure detection on the original image to obtain a person detection frame; and

[0008] Perform super-resolution processing on the person detection frame to generate a processed image with the same resolution as the original image.

[0009] Optionally, performing super-resolution processing on the person detection frame to generate a processed image with the same resolution as the original image includes:

[0010] Determine the minimum covering rectangular area covering all person detection frames in the original image;

[0011] The minimum value of a first ratio of the original image lateral resolution to the lateral resolution of the minimum covered rectangular area and a second ratio of the original image longitudinal resolution to the longitudinal resolution of the minimum covered rectangular area is used as the magnification factor;

[0012] Performing super-resolution processing on the minimum coverage rectangular area so that the horizontal resolution and the vertical resolution of the minimum coverage rectangular area are respectively amplified according to the magnification factor to obtain an amplified minimum coverage rectangular area; and

[0013] Determine whether the resolution of the enlarged minimum coverage rectangle area is the same as the original image resolution:

[0014] If so, the image containing the enlarged minimum covered rectangular area is used as the processed image;

[0015] If not, the minimum coverage rectangular area is edge-extended to obtain a processed image having the same resolution as the original image and including the enlarged minimum coverage rectangular area and the edge-extended area.

[0016] Optionally, the step of performing edge extension on the minimum coverage rectangular area to obtain a processed image having the same resolution as the original image and including the enlarged minimum coverage rectangular area and the edge extension area further includes:

[0017] Perform background blur processing on the enlarged minimum coverage rectangular area.

[0018] Optionally, when performing edge extension on the minimum coverage rectangular area, the method further includes: filling original image related information in the edge extension area.

[0019] Optionally, when the edge of the minimum coverage rectangular area is extended, the enlarged minimum coverage rectangular area is set at the center of the processed image.

[0020] Optionally, after performing human figure detection on the original image and before performing super-resolution processing on the human detection frame, the method further includes:

[0021] Speaker detection is performed on the original image, and person detection frames of non-speakers in the person detection frames obtained by the human shape detection are filtered out.

[0022] Optionally, performing super-resolution processing on the person detection frame to generate a processed image with the same resolution as the original image includes:

[0023] Calculate the mean horizontal resolution of the person detection frame and the mean vertical and horizontal resolution N is the number of person detection frames, W n is the horizontal resolution of the n-th person detection frame, H n is the vertical resolution of the n-th person detection frame, N is a positive integer, and n is a positive integer less than or equal to N;

[0024] Calculate the horizontal resolution W of the original image and the mean horizontal resolution W of the person detection frame Mean The ratio ω=W / W Mean , and calculate the vertical resolution H of the original image and the vertical resolution mean H of the person detection frameMean The ratio η=H / H Mean ;

[0025] For each natural number T in the interval [N, 2N-1], factorize T = A*B and get all the factorization results of the natural number T (A k ,B k );

[0026] For each natural number T in the interval [N, 2N-1], calculate the adjustment value θ corresponding to each factor decomposition result of the natural number T k =min(ω / A k ,η / B k ), and the adjustment value θ corresponding to each factor decomposition result of the natural number T k The maximum value among them is taken as the adjustment value θ corresponding to the natural number T T ,θ T =max(θ k );

[0027] The maximum value among the adjustment values ​​corresponding to the natural numbers T in the interval [N, 2N-1] is taken as the super-resolution adjustment value θ # ,θ # =max(θ T ), and determine the super-resolution adjustment value θ # The corresponding natural number T and factor decomposition result (A # ,B # );

[0028] Generate a blank image with the same resolution as the original image and divide the blank image into B # *A # arranged sub-image areas; and

[0029] Perform super-resolution processing on each person detection frame so that the horizontal resolution of each person detection frame is θ # *W Mean And the vertical resolution is θ # *H Mean , and fill each person detection frame after super-resolution processing into a sub-image area respectively, and obtain a processed image with the same resolution as the original image.

[0030] Optionally, filling each person detection frame after super-resolution processing into a sub-image area includes:

[0031] The center coordinates of each person detection frame in the original image are calculated, and the sub-image area corresponding to each person detection frame after super-resolution processing is determined according to the center coordinates of each person detection frame in the original image.

[0032] Optionally, the step of filling each person detection frame after super-resolution processing into a sub-image area to obtain a processed image with the same resolution as the original image further includes:

[0033] Background blur processing is performed on the sub-image area filled with the person detection frame after super-resolution processing.

[0034] Optionally, after filling each person detection frame after super-resolution processing into a sub-image area respectively, if there is a remaining sub-image area that does not correspond to the person detection frame, the method also includes: filling the original image related information in the remaining sub-image area.

[0035] A second aspect of the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the image processing method provided in the first aspect of the present invention is implemented.

[0036] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image processing method provided in the first aspect of the present invention.

[0037] The beneficial effects of the present invention are as follows:

[0038] The technical solution of the present invention can maximize the effective information of the image and improve the user experience of the information receiver by amplifying the information of high-value human areas in images such as video conferencing images and behavior monitoring images. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0040] Figure 1 A flowchart of an image processing method provided by an embodiment of the present invention is shown.

[0041] Figure 2 Schematic diagram showing the super-resolution processing steps using the first mode.

[0042] Figure 3 A schematic diagram showing the steps of super-resolution processing using the second mode.

[0043] Figure 4 A schematic diagram showing the structure of a computer system implementing a server provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0044] In order to more clearly illustrate the present invention, the present invention will be further described below in conjunction with the embodiments and drawings. Similar components in the drawings are represented by the same reference numerals. It should be understood by those skilled in the art that the following specific description is illustrative rather than restrictive and should not be used to limit the scope of protection of the present invention.

[0045] An embodiment of the present invention provides an image processing method, comprising the following steps:

[0046] Acquire an image captured by an image acquisition device as an original image;

[0047] Performing human figure detection on the original image to obtain a person detection frame; and

[0048] Perform super-resolution processing on the person detection frame to generate a processed image with the same resolution as the original image.

[0049] The image processing method provided by the present invention maximizes the effective information in images by amplifying high-value human areas in images such as video conferencing images and behavior monitoring images, thereby enhancing the user experience for the recipient. The method for determining human areas through human figure detection enables automatic detection and automatic adjustment based on changes in human position, the addition or removal of people in the image, and other factors.

[0050] The image processing method provided in this embodiment can be implemented by a computer device with data processing capabilities. Specifically, the computer device can be a computer with data processing capabilities, including a personal computer (PC), a minicomputer or a mainframe, or a server or server cluster with data processing capabilities, etc. This embodiment does not limit this.

[0051] Next, from the perspective of a processing device with data processing capabilities, an image processing method provided by this embodiment is described by taking application in scenarios such as remote video conferencing as an example.

[0052] An embodiment of the present invention provides an image processing method, such as Figure 1 As shown, the following steps are included:

[0053] S110 , obtaining an image captured by an image acquisition device as an original image.

[0054] In scenarios such as remote video conferencing, the image acquisition device is, for example, a camera installed above the conference room for capturing images of the conference venue. The computer device that executes the image processing method is connected to the camera, and can obtain the video conference image captured by the camera (usually the entire conference venue image) in real time. The original image is the video conference image.

[0055] S120: Perform human figure detection on the original image to obtain a person detection frame.

[0056] Compared to face detection and head detection, the human figure detection used in step S120 has a wider detection range. It can detect any person in the image, regardless of their posture, whether they are looking down or facing away from the image capture device, such as the camera above the conference room. For example, human figure detection can be implemented using deep learning models such as YOLO (You Only Look Once) V5, or machine learning models such as HoG and SVM.

[0057] In a specific example, the human figure detection in step S120 is implemented using the YOLOv5 model. For example, the trained YOLOv5 model is used to perform human figure detection on the original image (video conference image) to obtain a person detection frame (participant detection frame) {P i |i=1,..,N}, where N is the number of people (participants) in the original image.

[0058] The YOLO (You Only Look Once) model is an object recognition and localization model based on deep neural networks. The YOLOV5 model offers excellent recognition performance and fast inference speed, making it suitable for online deployment. Furthermore, the YOLOV5 model takes into account the multi-scale nature of objects in its network structure and anchor box design, effectively addressing the recognition of smaller objects (people). During training, the YOLOV5 model passes and augments each batch of training data through a data loader (the data loader performs three types of data augmentation: scaling, color space adjustment, and mosaic enhancement). This process significantly expands the training data and significantly improves the model's generalization ability. For example, the YOLOV5 model comprises a backbone network, a fusion network, and a bounding box prediction layer. The backbone network can be a convolutional neural network that upsamples the image at multiple levels to produce images of varying granularity (resolution). The upsampled feature maps corresponding to each level have a higher granularity than the previous level, and image features are extracted from each upsampled feature map. The fusion network consists of a series of network layers that mix and combine image features. The fusion network can be a Feature Pyramid Network (FPN) or a Path Aggregation Network (PANet). It is used to fuse image features and pass the fused image features to the bounding box prediction layer. The bounding box prediction layer is used to generate the bounding box corresponding to the target to be identified and segment the target from the image to be identified based on the bounding box, resulting in the target to be identified, that is, the person detection box.

[0059] S130 , performing super-resolution processing on the person detection frame to generate a processed image with the same resolution as the original image.

[0060] In scenarios such as remote video conferencing, after the image is processed locally, it can be transmitted to the other party of the video conference for display. This can maximize the effective information in the video conference image and improve the user experience of the other party of the video conference, that is, the information recipient.

[0061] In a possible implementation, after step S120 and before step S130, the image processing method provided in this embodiment further includes:

[0062] Speaker detection is performed on the original image, and person detection frames of non-speakers in the person detection frames obtained by the human shape detection are filtered out.

[0063] In scenarios such as remote video conferencing, this implementation, based on human figure detection, determines the active speaker among the participants and uses only the speaker's person detection frame for subsequent super-resolution processing, eliminating the need to use the person detection frames of non-speakers. For example, after speaker detection, if one participant in a two-party video conference has only one speaker, super-resolution processing is performed only on the person detection frame of that speaker, and only the super-resolution-processed image containing that speaker is displayed to the other party. If the speaker subsequently changes, the super-resolution-processed image containing the new speaker is displayed to the other party based on the real-time detection results. In this implementation, convolutional neural networks such as ResNet and DenseNet can be used for speaker detection, for example, by extracting lip feature information for recognition and detection.

[0064] Next, we will continue to introduce step S130, which is a super-resolution processing step. For step S130, this embodiment provides two modes in the following possible implementation methods. For example, the user can select a mode through an input device such as a mouse, keyboard, touch screen, etc. of a computer device that executes the image processing method.

[0065] The two modes of the super-resolution processing step S130 are described below respectively.

[0066] First Mode

[0067] In the first mode, step S130 further includes sub-steps S1311-S1314:

[0068] S1311. Determine the minimum covering rectangular area that covers all person detection frames in the original image.

[0069] In a specific example, after obtaining the person detection frame (participant detection frame) {P i |i=1,..,N}, N is the number of people (participants) in the original image, and it is determined that all person detection frames {P i The smallest rectangle among the rectangles of |i=1,..,N} is taken as the minimum covering rectangular area R. For example, the method of determining the minimum covering rectangular area R is as follows: determine all the person detection boxes {P i The maximum and minimum values ​​of the horizontal coordinate of |i=1,..,N} are respectively denoted as x max and x min , and determine the maximum and minimum values ​​of the vertical coordinates of all person detection frames {Pi|i=1,..,N}, respectively, and record them as y max and y min , determine the coordinates of the four vertices of the minimum covering rectangular area R are (x min ,y max )、(x max ,y max )、(x min ,y min ) and (x max ,y min ), or in other words, the minimum covering rectangular area R is (x min ,y min ) and (x max ,y max ) are the diagonal lines of the rectangular area.

[0070] S1312: Use the minimum value of a first ratio of the original image horizontal resolution to the horizontal resolution of the minimum covered rectangular area and a second ratio of the original image vertical resolution to the vertical resolution of the minimum covered rectangular area as the magnification factor.

[0071] In a specific example, let the horizontal resolution of the original image (the width of the original image) be W, the vertical resolution of the original image (the height of the original image) be H, and the horizontal resolution of the minimum covered rectangular area R (the width of the minimum covered rectangular area R) be W. R , the minimum coverage rectangular area R vertical resolution (minimum coverage rectangular area R height) is H R , then the magnification is S = min (W / W R ,H / H R ).

[0072] S1313 . Perform super-resolution processing on the minimum coverage rectangular area, so that the horizontal resolution and the vertical resolution of the minimum coverage rectangular area are respectively amplified according to the magnification factor, to obtain an amplified minimum coverage rectangular area.

[0073] In a specific example, the minimum coverage rectangular area R is magnified S times by super-resolution processing to obtain the magnified minimum coverage rectangular area R1, wherein the i-th person detection frame in the original image is magnified to P i `.

[0074] In a specific example, the super-resolution process may adopt an interpolation-based, reconstruction-based, and learning-based method, for example, a bilinear interpolation method.

[0075] S1314: Determine whether the resolution of the enlarged minimum coverage rectangular area is the same as the original image resolution:

[0076] If yes, the image containing the enlarged minimum coverage rectangular area is taken as the processed image. It can be understood that the horizontal resolution W of the original image and the horizontal resolution W of the minimum coverage rectangular area R are different. R The first ratio W / W R The vertical resolution H of the original image and the vertical resolution H of the minimum coverage rectangular area R R The second ratio H / H R Under the same conditions, the resolution of the minimum covered rectangular area R after enlargement is the same as the original image resolution. R and the second ratio H / H R In different situations, the horizontal resolution of the enlarged minimum coverage rectangular area R will be smaller than the horizontal resolution W of the original image, or the vertical resolution of the enlarged minimum coverage rectangular area R will be smaller than the vertical resolution H of the original image;

[0077] If not, the minimum coverage rectangular area is edge-extended to obtain a processed image having the same resolution as the original image and including the enlarged minimum coverage rectangular area and the edge-extended area.

[0078] In a possible implementation, in sub-step S1314, the step of performing edge extension on the minimum coverage rectangular area to obtain a processed image having the same resolution as the original image and including the enlarged minimum coverage rectangular area and the edge extension area further includes:

[0079] Perform background blur processing on the enlarged minimum coverage rectangular area.

[0080] In this way, combined with background blur processing, the information of low-value areas can be further compressed, thereby further enhancing the saliency of high-information-value areas.

[0081] In a specific example, background blur can be achieved by performing Gaussian filtering on the background area except the enlarged person detection frame in the enlarged minimum coverage rectangular area to blur the background.

[0082] In one possible implementation, when the edge of the minimum coverage rectangular area is extended, sub-step S1314 further includes: filling the edge extension area with information related to the original image. For example, in a remote video conferencing scenario, the information related to the original image may include the meeting name, time, conference room, and precautions.

[0083] In a possible implementation, in sub-step S1314, when the edge of the minimum coverage rectangular area is extended, the enlarged minimum coverage rectangular area is set at the center of the processed image.

[0084] Directly super-resolving the minimum coverage rectangle to the original image resolution would likely distort the face due to the change in aspect ratio, impacting the user experience. However, the first mode described above maximizes the focus on the person and displays more detail without distorting the person, while ensuring that each face area is magnified at a consistent ratio. Furthermore, the first mode is simple and fast to implement, and is suitable for situations where the number of person detection frames N ≥ 1.

[0085] In a specific example, in step S130 of the first mode, the original image, the minimum covering rectangular area and the processed image are as follows: Figure 2 As shown, Figure 2 The top image is the original image, the middle is the minimum coverage rectangle, and the bottom is the processed image. As can be seen, the first mode can effectively amplify information about high-value areas of people, such as in video conferencing images, thereby maximizing the effective information in the image.

[0086] Second Mode

[0087] The second mode may be referred to as an adaptive mode. In the second mode, step S130 further includes sub-steps S1321-S1327:

[0088] S1321, calculate the mean horizontal resolution of the person detection frame and the mean vertical and horizontal resolution N is the number of person detection frames in the original image, W n H is the horizontal resolution (width) of the n-th person detection frame, n is the vertical resolution (height) of the n-th person detection frame, N is a positive integer, and n is a positive integer less than or equal to N.

[0089] S1322, calculate the horizontal resolution W of the original image and the average horizontal resolution W of the person detection frame Mean The ratio ω=W / W Mean , and calculate the vertical resolution H of the original image and the vertical resolution mean H of the person detection frame Mean The ratio η=H / HMean .

[0090] S1323. For each natural number T in the interval [N, 2N-1], factorize T = A*B to obtain all factorization results of the natural number T (A k ,B k ).

[0091] The number of factorization results may vary for different natural numbers T. For example, if the number of person detection frames N = 5, the natural number T can be T = 5, 6, 7, 8, or 9. For T = 5, the set of factorization results ф is {(1, 5), (5, 1)}, meaning that for T = 5, k = 2. For T = 6, the set of factorization results ф is {(1, 6), (2, 3), (3, 2), (6, 1)}, meaning that for T = 6, k = 4.

[0092] S1324. For each natural number T in the interval [N, 2N-1], calculate the adjustment value θ corresponding to each factor decomposition result of the natural number T. k =min(ω / A k ,η / B k ), for example, in a specific example, if ω=8 and η=6, and the number of person detection frames N=5, then for the two factor decomposition results (1,5) and (5,1) of T=5, the adjustment value θ1=min(8 / 1,6 / 5)=1.2 for the first factor decomposition result (1,5), and the adjustment value θ2=min(8 / 5,6 / 1)=1.6 for the second factor decomposition result (5,1). Similarly, for the four factor decomposition results (1,6), (2,3), (3,2) and (6, 1) The corresponding adjustment values ​​are 1, 2, 2.67 and 1.33 respectively. For the two factor decomposition results of T = 7, the corresponding adjustment values ​​are 0.86 and 1.14 respectively. For the four factor decomposition results of T = 8, the corresponding adjustment values ​​are (1, 8), (2, 4), (4, 2) and (8, 1) respectively. The corresponding adjustment values ​​are 0.75, 1.5, 2 and 1 respectively. For the three factor decomposition results of T = 9, the corresponding adjustment values ​​are (1, 9), (3, 3) and (9, 1) respectively. The adjustment value θ corresponding to each factor decomposition result of the natural number T is k The maximum value among them is taken as the adjustment value θ corresponding to the natural number T T ,θ T =max(θ k ), for example, continuing the above example, the adjustment value θ corresponding to T=5 T =max(1.2,1.6)=1.6. Similarly, the adjustment value θ corresponding to T=6 T =2.67, T=7 corresponding adjustment value θT =1.14, T=8 corresponding adjustment value θ T =2, T=9 corresponding adjustment value θ T =2.

[0093] S1325: The maximum value among the adjustment values ​​corresponding to the natural numbers T in the interval [N, 2N-1] is used as the super-resolution adjustment value θ. # ,θ # =max(θ T ), and determine the super-resolution adjustment value θ # The corresponding natural number T and factor decomposition result (A # ,B # ). Continuing with the above example, the super-resolution adjustment value θ # =2.67, super-resolution adjustment value θ # =2.67 corresponds to the natural number T=6 and the corresponding factor decomposition result is (3,2).

[0094] S1326: Generate a blank image with the same resolution as the original image, and divide the blank image into B # *A # Arranged sub-image areas. Continuing with the above example, the blank image is divided into 2*3 arranged sub-image areas, ie, sub-image areas arranged in two rows and three columns.

[0095] S1327, perform super-resolution processing on each person detection frame so that the horizontal resolution of each person detection frame is θ # *W Mean And the vertical resolution is θ # *H Mean Continuing with the above example, super-resolution processing is performed on each person detection frame so that the horizontal resolution of each person detection frame is 2.67*W Mean And the vertical resolution is 2.67*H Mean Each person detection frame after super-resolution processing is filled into a sub-image area respectively, and a processed image with the same resolution as the original image is obtained.

[0096] In a specific example, the super-resolution process may adopt an interpolation-based, reconstruction-based, and learning-based method, for example, a bilinear interpolation method.

[0097] Regarding step S1327, it is understood that when there are multiple person detection frames in the original image, in rare cases, one or some person detection frames may be significantly larger than the others. Therefore, step S1327 generally enlarges all person detection frames. Even in the extreme case where one person detection frame in the original image is significantly larger than the others, step S1327 still enlarges the person detection frames overall.

[0098] In a possible implementation, in sub-step S1327, filling each person detection frame after super-resolution processing into a sub-image area includes:

[0099] The center coordinates of each person detection frame in the original image are calculated, and the sub-image area corresponding to each person detection frame after super-resolution processing is determined according to the center coordinates of each person detection frame in the original image.

[0100] In a specific example, the vertical coordinates can be sorted from top to bottom first, and then from left to right by the horizontal coordinates, so that the relative position relationship of each super-resolution processed person detection frame in the processed image is as close as possible to the relative position relationship of each person detection frame in the original image. Specifically, continuing with the above example, the center coordinates of each person detection frame in the original image are first determined and calculated, and the vertical coordinates of the center of the person detection frame are sorted from top to bottom (when the vertical coordinates of two or more person detection frames are the same, the order can be randomly arranged internally), and the 1st to A # The corresponding sub-image area is in the first row, # +1 to 2A # The first one corresponds to the sub-image area in the second row, and so on, the last row has the sub-image area (B # -1)*A # +1 to the Nth (may not be enough for A # Then, within the same row of sub-image areas, sort the horizontal coordinates of the center of the person detection frames from left to right (when two or more person detection frames have the same horizontal coordinates, the order can be randomly arranged). Then, fill in the corresponding sub-image areas in this order.

[0101] In a possible implementation, in sub-step S1327, filling each person detection frame after super-resolution processing into a sub-image area to obtain a processed image with the same resolution as the original image further includes:

[0102] Background blur processing is performed on the sub-image area filled with the person detection frame after super-resolution processing.

[0103] In a specific example, the size of the sub-image area may be larger than the person detection frame after super-resolution processing. The background blur can be achieved by filling the background area of ​​the sub-image area except the person detection frame after super-resolution processing with the pixel values ​​of the edge area of ​​the face detection frame and then blurring it by Gaussian filtering.

[0104] In a possible implementation, after filling each person detection frame after super-resolution processing into a sub-image area respectively, if there is a remaining sub-image area that does not correspond to the person detection frame, sub-step S1327 also includes: filling the original image related information in the remaining sub-image area.

[0105] For example, in a remote video conference scenario, the original image-related information includes conference name information, time information, conference room information, precaution information, etc.

[0106] Continuing the above example, in step S130 of the second mode, the original image and the processed image are as follows: Figure 3 As shown, Figure 3 The upper part shows the original image, and the lower part shows the processed image. It can be seen that the second mode can effectively amplify information about high-value areas of people, such as in video conferencing images, thereby maximizing the effective information of the image.

[0107] It is understandable that the second mode is applicable to the case where the number of person detection frames N is greater than or equal to 1, and is more suitable for the case where N is greater than 1.

[0108] Another embodiment of the present invention provides a server, comprising:

[0109] An acquisition module, used to acquire an image acquired by an image acquisition device as an original image;

[0110] A detection module, configured to perform human figure detection on the original image to obtain a human detection frame; and

[0111] The super-resolution module is used to perform super-resolution processing on the person detection frame and generate a processed image with the same resolution as the original image.

[0112] It should be noted that the principle and workflow of the server provided in this embodiment are similar to those of the above-mentioned image processing method. For relevant details, please refer to the above description and will not be repeated here.

[0113] like Figure 4As shown, a computer system suitable for implementing the server provided in the above embodiment includes a central processing module (CPU), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage portion into a random access memory (RAM). In the RAM, various programs and data required for the operation of the computer system are also stored. The CPU, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0114] The following components are connected to the I / O interface: an input section including a keyboard, mouse, etc.; an output section including a liquid crystal display (LCD) and a speaker; a storage section including a hard disk; and a communication section including a network interface card such as a LAN card and a modem. The communication section performs communication processing via a network such as the Internet. A drive is also connected to the I / O interface as needed. Removable media such as magnetic disks, optical disks, magneto-optical disks, semiconductor memories, etc. are installed in the drive as needed so that computer programs read from the media can be installed in the storage section as needed.

[0115] In particular, according to this embodiment, the process described in the flowchart above can be implemented as a computer software program. For example, this embodiment includes a computer program product comprising a computer program tangibly embodied on a computer-readable medium, the computer program containing program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication component and / or installed from a removable medium.

[0116] The flowcharts and schematic diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the system, method and computer program product of the present embodiment. In this regard, each box in the flowchart or schematic diagram can represent a module, program segment or part of code, and the part of the above-mentioned module, program segment or code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the schematic diagram and / or flowchart, and the combination of boxes in the schematic diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0117] The modules described in this embodiment may be implemented in software or hardware. The modules described may also be provided in a processor. For example, they may be described as a processor comprising an acquisition module, a detection module, and a super-resolution module. The names of these modules do not, in some cases, limit the modules themselves. For example, the super-resolution module may also be described as a "super-resolution module."

[0118] As another aspect, this embodiment further provides a non-volatile computer storage medium. This non-volatile computer storage medium may be the non-volatile computer storage medium included in the apparatus described in the above embodiment, or may be a standalone non-volatile computer storage medium not incorporated into a terminal. The non-volatile computer storage medium stores one or more programs. When executed by a device, the one or more programs cause the device to: acquire an image captured by an image acquisition device as an original image; perform human figure detection on the original image to obtain a person detection frame; and perform super-resolution processing on the person detection frame to generate a processed image with the same resolution as the original image.

[0119] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "upper" and "lower" is based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. Unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be internal communication between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.

[0120] It should also be noted that, in the description of the present invention, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0121] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not limitations on the implementation methods of the present invention. For ordinary technicians in this field, other different forms of changes or modifications can be made based on the above description. It is impossible to list all the implementation methods here. All obvious changes or modifications derived from the technical solution of the present invention are still within the scope of protection of the present invention.

Claims

1. An image processing method, characterized in that: include: Acquire an image captured by an image acquisition device as an original image; Performing human figure detection on the original image to obtain a person detection frame; as well as Perform super-resolution processing on the person detection frame to generate a processed image with the same resolution as the original image; The super-resolution processing of the person detection frame to generate a processed image with the same resolution as the original image includes: Calculate the mean horizontal resolution W of the person detection frame Mean =( W n ) / N and the mean vertical and horizontal resolution H Mean =( H n ) / N, N is the number of person detection frames, W n is the horizontal resolution of the n-th person detection frame, H n is the vertical resolution of the n-th person detection frame, N is a positive integer, and n is a positive integer less than or equal to N; Calculate the horizontal resolution W of the original image and the mean horizontal resolution W of the person detection frame Mean The ratio ω=W / W Mean , and calculate the vertical resolution H of the original image and the vertical resolution mean H of the person detection frame Mean The ratio η = H / H Mean ; For each natural number T in the interval [N, 2N-1], factorize T=A*B and get all the factorization results of the natural number T (A k ,B k ); For each natural number T in the interval [N, 2N-1], calculate the adjustment value θ corresponding to each factor decomposition result of the natural number T k =min(ω / A k ,η / B k ), and the adjustment value θ corresponding to each factor decomposition result of the natural number T k The maximum value among them is taken as the adjustment value θ corresponding to the natural number T T ,θ T =max(θ k ); The maximum value among the adjustment values ​​corresponding to the natural numbers T in the interval [N, 2N-1] is taken as the super-resolution adjustment value θ # ,θ # =max(θ T ), and determine the super-resolution adjustment value θ # The corresponding natural number T and factor decomposition result (A # ,B # ); Generate a blank image with the same resolution as the original image and divide the blank image into B # *A # arranged sub-image areas; and Perform super-resolution processing on each person detection frame so that the horizontal resolution of each person detection frame is θ # *W Mean And the vertical resolution is θ # *H Mean , and fill each person detection frame after super-resolution processing into a sub-image area respectively, and obtain a processed image with the same resolution as the original image.

2. The method according to claim 1, characterized in that After performing human figure detection on the original image and before performing super-resolution processing on the human detection frame, the method further includes: Speaker detection is performed on the original image, and person detection frames of non-speakers in the person detection frames obtained by the human shape detection are filtered out.

3. The method according to claim 1, characterized in that Filling each person detection frame after super-resolution processing into a sub-image area includes: The center coordinates of each person detection frame in the original image are calculated, and the sub-image area corresponding to each person detection frame after super-resolution processing is determined according to the center coordinates of each person detection frame in the original image.

4. The method according to claim 1, wherein The step of filling each person detection frame after super-resolution processing into a sub-image area to obtain a processed image with the same resolution as the original image further includes: Background blur processing is performed on the sub-image area filled with the person detection frame after super-resolution processing.

5. The method according to claim 1, wherein After filling each person detection frame after super-resolution processing into a sub-image area respectively, if there is a remaining sub-image area that does not correspond to the person detection frame, the method further includes: filling the remaining sub-image area with original image related information.

6. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 5 is implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images