Method and device for improving image quality of local area of image, storage medium and equipment

Abstract

The invention provides a method and device for improving the image quality of a local region of an image, a storage medium and electronic equipment, and the method comprises the steps: obtaining an initial monitoring image, and determining a target region containing a target in the initial monitoring image; cutting the initial monitoring image to obtain a target area image containing the target area; calculating a shooting distance of the target area; if the shooting distance of the target area is greater than a set threshold value, carrying out super-resolution processing on the target area image to obtain a super-resolution target area image; the set threshold value is a parameter for judging whether a target in the initial monitoring image is clear or not; and adopting the super-resolution target area image to cover the area where the target area image is located in the initial monitoring image to obtain a final monitoring image for improving the image quality of the local area. In the scheme, the processing of improving the resolution is only carried out on the target area image of which the shooting distance is greater than the set threshold value without carrying out super-resolution processing on the whole shot image, that is, self-adaptive selection is carried out according to the shooting distance of each target area image in the initial monitoring image; the local target area image with the shooting distance exceeding the set threshold value is subjected to super-division processing, so that the data processing amount of super-division processing is reduced, and the processing efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, storage medium, and electronic device for improving the image quality of a local area of ​​an image. Background Art

[0002] Target recognition is a common requirement in image processing. Targets can be faces, gestures, and more. For example, gesture control devices are used in many scenarios. A camera captures hand movements within the monitoring range and tracks the hands to determine the gesture recognition results. The gesture recognition results are then compared with various control gestures in a pre-stored gesture library to determine the control gesture corresponding to the gesture recognition result. The device can then be controlled based on the control gesture. For example, in a vehicle, occupants can use gestures to control some of the vehicle's devices. For example, a downward swipe gesture indicates lowering the air conditioning temperature, while a right swipe indicates increasing the volume of the car's speakers.

[0003] When existing solutions are implemented, the distance between the target and the camera affects the clarity of the image. For example, people sitting in the front row of a vehicle are closer to the camera than those sitting in the back row. The hand movements of the front row people captured by the camera are clearer, while the hand movements of the back row people are less clear. To ensure the clarity of all hand movements, the captured image is processed to increase the resolution (super-resolution) to obtain a super-resolution image. This means that the resolution of each pixel in the image is increased. Due to the large number of pixels in the image as a whole, the computational complexity of the overall super-resolution processing is very high, affecting processing efficiency. Summary of the Invention

[0004] The technical problem to be solved by this application is that the existing technology has a large amount of computation and low efficiency in the super-resolution processing of images taken by cameras. To this end, this application proposes a method, device, storage medium and electronic device for improving the image quality of local areas.

[0005] In a first aspect, the technical solution of the present application provides a method for improving the image quality of a local area of ​​an image, comprising:

[0006] Acquire an initial monitoring image, and determine a target area containing a target in the initial monitoring image;

[0007] Cropping the initial monitoring image to obtain a target area image containing the target area;

[0008] Calculating the shooting distance of the target area;

[0009] If the shooting distance of the target area is greater than a set threshold, the target area image is super-resolved to obtain a super-resolution target area image; the set threshold is a parameter for determining whether the target in the initial monitoring image is clear;

[0010] The super-resolution target area image is used to cover the area where the target area image is located in the initial monitoring image, so as to obtain a final monitoring image with improved image quality of the local area.

[0011] Some methods for improving the image quality of local areas described in the solutions:

[0012] The calculating the shooting distance of the target area includes: calculating the average depth of all pixels in the target area image, and using the average depth as the shooting distance of the target area;

[0013] If the shooting distance of the target area is greater than a set threshold, it includes: if the average depth is greater than a set depth threshold; wherein the set depth threshold is: the actual average depth of the target area image when the shooting distance is equal to the set threshold.

[0014] In some embodiments of the method for improving the image quality of a local area of ​​an image, the step of calculating the average depth of all pixels in the target area image includes:

[0015] Obtaining a depth matrix of the target area image, wherein the depth matrix records the depth value of each pixel in the target area image;

[0016] An average value of all depth values ​​in the depth matrix is ​​calculated, and the average value is used as the average depth.

[0017] In some embodiments of the method for improving the image quality of a local area of ​​an image, the super-resolution processing of the target area image to obtain a super-resolution target area image includes:

[0018] Retrieving a super-resolution model, wherein the super-resolution model is used to improve the resolution of the input image, and the super-resolution model is trained by image sample pairs obtained by shooting the target to be identified at different shooting distances;

[0019] The target area image is input into the super-resolution model for super-resolution processing to obtain the super-resolution target area image.

[0020] In some embodiments of the method for improving the image quality of a local area of ​​an image, the training process of the super-resolution model includes:

[0021] Acquire captured images of the target to be identified at different test positions; wherein, when the target to be identified is at different test positions, the shooting distance from the target to be identified to the camera is different;

[0022] cropping the target image to be identified that contains the target to be identified in each of the captured images, and dividing the target images to be identified collected at the same test position into a group of image samples;

[0023] Selecting the target images to be identified from any two groups of image samples to form a pair of image sample pairs, wherein the two target images to be identified in each image sample pair have different resolutions;

[0024] The super-resolution model is obtained by training an initial super-resolution algorithm using multiple pairs of image samples.

[0025] In some embodiments of the method for improving the image quality of a local area of ​​an image, the initial super-resolution algorithm includes a learning algorithm based on a CNN structure and a training loss function, and the learning algorithm based on the CNN structure includes a feature extraction network, a feature reconstruction network, and an upsampling network;

[0026] The feature extraction network includes multiple convolutional layers, which extract features from the input image samples. The features extracted by each convolutional layer are simultaneously used as input to the last convolutional layer. The features extracted by each convolutional layer are combined to obtain different scale features of the image samples.

[0027] The feature reconstruction network adopts a deconvolution network, and the deconvolution network performs feature reconstruction on the different-scale features extracted by the feature extraction network to improve the resolution of the different-scale features;

[0028] The upsampling network performs interpolation processing on the image samples, and splices the interpolated image samples with the different scale features with improved resolution output by the feature reconstruction network to obtain image samples with improved resolution;

[0029] The image samples with increased resolution are used to train the training loss function, and the training results of the training loss function are used to adjust the network weight values ​​in the learning algorithm based on the CNN structure.

[0030] Some methods for improving the image quality of local areas described in the solutions:

[0031] The step of cropping the initial monitoring image to obtain a target area image containing the target area includes:

[0032] Obtaining a circumscribed rectangle of the target area in the initial monitoring image;

[0033] cropping the circumscribed rectangle in the initial monitoring image to obtain the target area image;

[0034] The step of cropping the target image to be identified that contains the target to be identified in each of the captured images comprises:

[0035] Obtaining the circumscribed rectangle of the target to be identified in each of the captured images;

[0036] The target image to be identified is obtained by cropping the circumscribed rectangle in each of the captured images.

[0037] In some embodiments, the method for improving the image quality of a local area of ​​an image, wherein the super-resolution target area image is used to cover the area where the target area image is located in the initial surveillance image to obtain a final surveillance image with improved image quality of the local area, includes:

[0038] normalizing the position data of the edge points of the target area image in the initial monitoring image using the position data of the edge points of the target area image in the initial monitoring image as a reference to determine the target position data of the edge points of the super-resolution target area image in the initial monitoring image;

[0039] The super-resolution target area image is overlaid in the area defined by the target position data in the initial monitoring image to obtain the final monitoring image.

[0040] In a second aspect, the technical solution of the present application provides a device for improving the image quality of a local area of ​​an image, comprising:

[0041] An image acquisition module is configured to acquire an initial monitoring image and determine a target area containing a target in the initial monitoring image;

[0042] an image cropping module, configured to crop the initial monitoring image to obtain a target area image containing the target area;

[0043] a distance calculation module, configured to calculate the shooting distance of the target area image;

[0044] a super-resolution processing module configured to perform super-resolution processing on the target area image to obtain a super-resolution target area image if the shooting distance is greater than a set threshold; the set threshold is a parameter for determining whether the target in the initial monitoring image is clear;

[0045] The image overlay module is configured to overlay the area of ​​the initial surveillance image where the target area image is located with the super-resolution target area image, thereby obtaining a final surveillance image with improved image quality in the local area. In a third aspect, the technical solution of the present application provides a computer-readable storage medium having program information stored therein, and a computer, after reading the program information, executes the method for improving the image quality of a local area of ​​an image as described in any one of the solutions of the first aspect.

[0046] In a fourth aspect, the technical solution of the present application provides an electronic device, comprising at least one processor and at least one memory, wherein at least one of the memories stores program information, and at least one of the processors reads the program information and executes the method for improving the image quality of a local area of ​​an image as described in any one of the solutions of the first aspect.

[0047] In a fifth aspect, the technical solution of the present application provides a computer program product, including a computer program / instruction, characterized in that when the computer program / instruction is executed by a processor, the steps of the method for improving the image quality of a local area of ​​an image as described in any one of the solutions of the first aspect are implemented.

[0048] The above technical solution has the following beneficial effects:

[0049] The method, device, storage medium, and electronic device provided herein for improving the image quality of a local region of an image acquire an initial surveillance image, determine the target region containing the target in the initial surveillance image, and then crop the target region image. The method then calculates the shooting distance of the target region image and determines whether the shooting distance is greater than a set threshold. If the shooting distance of the target region image is greater than the set distance threshold, indicating that the shooting distance corresponding to the target region image is too far, resulting in poor target clarity, the resolution of the target region image is then improved to obtain a super-resolution target region image, thereby improving the clarity of the target region image. The super-resolution target region image is then used to overlay the area of ​​the initial surveillance image where the target region image resides, resulting in a final surveillance image with improved image quality in the local region. This improves the clarity of the target in the final surveillance image and achieves the goal of improving the image quality of the local region of the surveillance image. In this solution, the resolution is only enhanced for target region images whose shooting distance is greater than the set threshold, without requiring super-resolution processing for the entire captured image. Specifically, adaptive selection is made based on the shooting distance of each target region image in the initial surveillance image, and local target region images whose shooting distance exceeds the set threshold are super-resolution processed, thereby reducing the data processing workload of the super-resolution processing and improving processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 This is a flow chart of a method for improving the image quality of a local area of ​​an image in one embodiment of the present application;

[0051] Figure 2 Schematic diagram of depth values ​​of pixels in an image of a target area in an embodiment of the present application;

[0052] Figure 3 This is a schematic diagram of the working state when a camera is used to capture images of a target to be identified at different distances in one embodiment of the present application;

[0053] Figure 4This is a schematic diagram showing the clarity of a target hand movement at different distances captured by a camera in one embodiment of the present application;

[0054] Figure 5 A schematic diagram comparing the gesture recognition results obtained by using the existing technology without performing local image quality enhancement and the gesture recognition results obtained by using the local image quality enhancement method in the present application solution;

[0055] Figure 6 This is a structural block diagram of an apparatus for improving the image quality of a local area of ​​an image in one embodiment of the present application;

[0056] Figure 7 A schematic diagram of the hardware connection structure of an electronic device for executing a method for improving the image quality of a local area of ​​an image according to an embodiment of the present application. DETAILED DESCRIPTION

[0057] The specific implementation of this application is further described below with reference to the accompanying drawings.

[0058] It is easy to understand that according to the technical solution of this application, a variety of structural methods and implementation methods can be replaced with each other by those skilled in the art without changing the essential spirit of this application. Therefore, the following specific embodiments and drawings are only exemplary descriptions of the technical solution of this application and should not be regarded as the entire application or as a limitation or restriction of the technical solution of the application.

[0059] The directional terms such as up, down, left, right, front, back, front, back, top, and bottom mentioned or possibly mentioned in this specification are defined relative to the structure shown in the drawings. They are relative concepts and may vary depending on the location and usage of the device. Therefore, these or other directional terms should not be interpreted as restrictive.

[0060] The embodiment of the present application provides a method for improving the image quality of a local area of ​​an image, which can be applied to a processor, which can be set on the vehicle side or the server side. Figure 1 As shown, the method includes:

[0061] S10: Acquire an initial monitoring image, and determine a target area containing a target in the initial monitoring image.

[0062] The initial monitoring image is captured by a camera, which may be a camera installed in an office area or a vehicle. The camera is preferably an infrared camera, which may be a monocular camera or a binocular camera.

[0063] The target may be a face or a hand, etc., and may be determined according to the actual application scenario and requirements.

[0064] In this step, only the target area needs to be identified. At this time, there is no need to make accurate identification of the target outline, and some existing recognition methods can be used to achieve it.

[0065] S20: Cropping the initial monitoring image to obtain a target area image including the target area.

[0066] The cropping method can be: by determining the target area in the initial monitoring image, the number of pixel rows or columns occupied by the target area can be determined, and the (number of rows, number of columns) of the pixel points at the edge of the target area can be used as its position coordinates. When the position coordinates of each pixel point at the edge of the target area are determined, they can be cropped to obtain the target area image.

[0067] S30: Calculating the shooting distance of the target area image.

[0068] Specifically, the image captured by the camera can be used to determine the shooting distance corresponding to each pixel based on the pixel value of each pixel in the image. The shooting distance is the distance between the object and the camera. Generally, the smaller the shooting distance, the clearer the image captured.

[0069] S40: If the shooting distance is greater than a set threshold, the resolution of the target area image is increased to obtain a super-resolution target area image. The set threshold is a parameter for determining whether the target in the initial monitoring image is clear.

[0070] Calibration tests can be used to calibrate the relationship between camera image clarity and shooting distance. Clarity can be determined empirically or by observation, with the threshold distance at which the image transitions from clear to unclear being established. When the shooting distance exceeds the threshold, the target area is considered unclear. In this case, the resolution of the target area is increased, which naturally improves the clarity of the objects contained within.

[0071] S50: Using the super-resolution target area image to cover the area where the target area image is located in the initial monitoring image, to obtain a final monitoring image with improved image quality of the local area.

[0072] That is, the super-resolution target area image is used to replace the original target area image. At this time, the clarity of the target has been improved, and the image quality of the local area containing the target in the final monitoring image has been improved.

[0073] In the above-described solution provided by this embodiment, after acquiring an initial surveillance image, the target region containing the target is determined within the initial surveillance image, and then the target region image is cropped. The shooting distance of the target region image is calculated, and a determination is made as to whether the shooting distance is greater than a set threshold. If the shooting distance is greater than the set threshold, indicating that the shooting distance corresponding to the target region image is too far, resulting in poor target clarity, the resolution of the target region image is then increased to obtain a super-resolution target region image, thereby improving the clarity of the target region image. The super-resolution target region image is then overlaid on the region containing the target region image in the initial surveillance image to obtain a final surveillance image with enhanced image quality in the local region. This improves target clarity in the final surveillance image, achieving the goal of improving the image quality of the local region in the surveillance image. In this solution, resolution enhancement is performed only on target region images with a shooting distance greater than the set threshold, without requiring super-resolution processing for the entire captured image. Specifically, adaptive selection is made based on the shooting distance of each target region image in the initial surveillance image, and only the local target region images with a shooting distance exceeding the set threshold are super-resolution processed. This reduces the amount of data to be processed in the super-resolution process and improves processing efficiency.

[0074] In some embodiments, calculating the shooting distance of the target area image in step S30 includes:

[0075] S31: Calculating an average depth of all pixels in the target area image, where the average depth is used to represent the shooting distance.

[0076] When using an infrared camera, the depth value of each pixel in the captured image is expressed as a parameter. The depth value is used to indicate the shooting distance. The average depth of all pixels in the target area image represents the average shooting distance of all pixels in the target area image.

[0077] In step S40, if the shooting distance of the target area is greater than the set threshold, the average depth may be greater than the set depth threshold. The set depth threshold is the actual average depth of the target area image when the shooting distance is equal to the set depth threshold. That is, the average depth of the target area image corresponds to the shooting distance. When the shooting distance of the target area image is equal to the set depth threshold, the actual average depth of the image obtained is equal to the set depth threshold.

[0078] Usually, the target area image includes the target and the background, and the shooting distance of the background is basically unchanged, so the parameter that affects the average distance is the shooting distance of the target, which can be used to represent the distance between the target and the camera. Figure 2Taking the target area image shown as an example, the target is a human face. The target area image includes a human face image and a background area image. The depth value of the background area is D1, and the depth value of the human face image is D2. If the image is in a vehicle, and the background area is the vehicle seat, body, etc., the depth value of the background area will basically not change. Because the depth value D1 of the background area is basically fixed, the depth value D2 of the human face image has a greater impact on the average depth. If the average depth is large, it means that the depth value of the human face image is too large, that is, the distance between the human face and the camera is large, and it is very likely to be unclear. Through this solution, the camera shooting results are directly used to obtain the judgment result of whether the shooting distance is greater than the set threshold, which simplifies the calculation process.

[0079] Furthermore, in step S31, the average depth of all pixels in the target area image is calculated, and the average depth is used to represent the shooting distance, including:

[0080] S311: Acquire a depth matrix of the target area image, where the depth matrix records the depth value of each pixel in the target area image.

[0081] like Figure 2 The target area image in is a rectangular frame containing the face image. The number of rows and columns of pixels it contains is fixed, and the depth value of each pixel can also be determined. Therefore, a matrix is ​​established according to the rows and columns of pixels. Each value in the matrix corresponds to the depth value of the pixel at the same position.

[0082] S312: Calculate the average value of all depth values ​​in the depth matrix as the average depth.

[0083] The average depth can also be obtained by using methods such as weighted averaging. In this solution, the method of directly obtaining the average value is simpler to calculate.

[0084] Furthermore, the method of performing super-resolution processing on the target area image to obtain a super-resolution target area image includes:

[0085] S401: Retrieve a super-resolution model, where the super-resolution model is used to improve the resolution of an input image. The super-resolution model is trained using image sample pairs obtained by capturing a target to be identified at different shooting distances.

[0086] The super-resolution model can select an existing machine learning algorithm, a deep learning algorithm, etc., and use image samples obtained by shooting the target to be identified at different shooting distances to input into the above learning algorithm for training to obtain a super-resolution model. After obtaining the super-resolution model, it can be built into the control system and directly applied.

[0087] S402: Inputting the target region image into the super-resolution model for super-resolution processing to obtain the super-resolution target region image.

[0088] Because the super-resolution model is trained using pairs of image samples of the target to be identified, taken at different distances, any target area image at any distance can be input into the super-resolution model for resolution improvement. Feature extraction, feature reconstruction, interpolation, and the superposition of the interpolated image with the reconstructed image are common processing steps in learning algorithms. A super-resolution model trained using a learning algorithm can also perform these steps on input target area images to achieve super-resolution of the target area image.

[0089] This solution achieves resolution improvement directly through model training, which is more efficient. Furthermore, this solution only processes the target area image, not the entire initial surveillance image, reducing the amount of data processing and further shortening the time required to improve the target resolution.

[0090] Preferably, in the above solution, the training process of the super-resolution model includes:

[0091] S4001: Acquire captured images of a target to be identified at different test positions; wherein, when the target to be identified is at different test positions, the shooting distance between the target to be identified and the camera is different.

[0092] like Figure 3 As shown, camera ca can be placed on a test bench to simulate actual application scenarios. For example, if the camera is used in a vehicle, it can be placed on the vehicle test bench. The target to be recognized can be a hand. The hand is captured at various test positions, such as 60mm, 90mm, and 120mm, while performing different manipulation gestures. Camera ca captures images ta1, ta2, and ta3 at these various test positions.

[0093] S4002: Crop the target image to be identified that contains the target to be identified in each of the captured images, and divide the target images to be identified collected at the same test position into a group of image samples.

[0094] like Figure 4 As shown in the figure, assuming that the distance is divided into close distance S1, medium distance S2, long distance S3, and very long distance S4, the cropped target images of various gestures to be recognized at each distance are shown in the figure. Obviously, the clarity of the hand image decreases as the distance increases. In specific applications, multiple sets of these images can be captured and cropped to meet the training algorithm's requirements.

[0095] S4003: Select the target images to be identified from any two groups of image samples to form an image sample pair, wherein the two groups of target images to be identified in each image sample pair have different resolutions.

[0096] Different groups of image samples correspond to different shooting distances, and the target images to be identified can be images of the same hand movement, that is, each pair of image samples contains a low-resolution image and a high-resolution image. During the training process, the low-resolution image and the high-resolution image are combined into a pair of samples for training, which can realize a model that processes low-resolution images into the same group of high-resolution images. Therefore, any low-resolution image (such as ultra-long distance S4) can be paired with any high-resolution image (such as close distance S1, medium distance S2, long distance S3). In actual application, the resolution of the high-resolution image is greater than the resolution corresponding to the clarity requirement. The super-resolution model obtained in this way can process any input low-resolution image into a high-resolution image that meets the clarity requirement.

[0097] S4004: The super-resolution model is obtained by training the initial super-resolution algorithm using multiple pairs of image samples.

[0098] After selecting the learning algorithm, you can use samples to train the algorithm. When the amount of sample data is large enough, the training results will be more accurate, and the super-resolution model will also be more accurate in improving the image resolution.

[0099] In specific implementation, the data volume of training samples can also be expanded in the following ways: randomly down-sampling the target images to be identified with higher resolution to generate new low-resolution target images to be identified, and pairing them with other high-resolution target images to be identified to generate new sample pairs, expand the training samples, and improve the accuracy of the training results.

[0100] In some embodiments, the initial super-resolution algorithm includes a learning algorithm based on a CNN structure and a training loss function, wherein the learning algorithm based on the CNN structure includes a feature extraction network, a feature reconstruction network, and an upsampling network; wherein:

[0101] The feature extraction network includes multiple convolutional layers, which extract features from input image samples. The features extracted by each convolutional layer are simultaneously used as input to the last convolutional layer. The features extracted by each convolutional layer are combined to obtain features of different scales of the image samples. Specifically, a feature extraction network with n convolutional layers is set up to extract features from image samples. After each layer is extracted, the extracted features are transmitted to the last layer for combination in the form of a residual neural network. This method can extract features of images of different scales and perform multi-dimensional combinations, which is more conducive to network learning.

[0102] The feature reconstruction network uses a deconvolution network, which performs feature reconstruction on the different-scale features extracted by the feature extraction network to improve the resolution of the different-scale features. Specifically, the feature reconstruction network uses a deconvolution neural network to reconstruct the features of the different-scale images obtained by the feature extraction network. The specific implementation method is: first, padding (the attribute defines the space between the element border and the element content) is expanded on the features of the different-scale images, and then a 1×1 convolution kernel is selected to perform a convolution operation on the features of the expanded different-scale images with a certain step size, and finally obtain the features of the upsampled high-resolution different-scale images.

[0103] The upsampling network performs interpolation processing on the image samples, and the interpolated image samples are superimposed with the different scale features with improved resolution output by the feature reconstruction network to obtain image samples with improved resolution; specifically, bicubic interpolation is performed on the image samples, and then added to the output result of the feature reconstruction network, and the output result of the network is finally obtained, which is a high-resolution image sample.

[0104] The image samples with increased resolution are used to train the training loss function, and the training results of the training loss function are used to adjust the weights of each network in the learning algorithm. Specifically, the L1 function can be used as the training loss function, and the high-resolution image samples generated in the above steps can be used as input for training to obtain a super-resolution model for a single image sample. The super-resolution model can be obtained by training the super-resolution model using various distances and different image samples.

[0105] In some embodiments, cropping the initial surveillance image to obtain a target area image containing the target area includes: obtaining a circumscribed rectangle of the target area in the initial surveillance image; cropping the circumscribed rectangle in the initial surveillance image to obtain the target area image; cropping the target image to be identified that contains the target to be identified in each of the captured images includes: obtaining a circumscribed rectangle of the target to be identified in each of the captured images; cropping the circumscribed rectangle in each of the captured images to obtain the target image to be identified. For example, the representation (x1, y1, x2, y2) is used as the coordinate representation of the upper left and lower right vertices of the circumscribed rectangle or the representation of the number of pixel rows and columns. Using a circumscribed rectangle for cropping can simplify the cropping method.

[0106] Preferably, in the above solution, the step S50 of using the super-resolution target area image to cover the area where the target area image is located in the initial monitoring image to obtain a final monitoring image with improved image quality of the local area includes:

[0107] S501: Using the position data of the edge points of the target area image in the initial monitoring image as a reference, normalize the position data of the edge points of the super-resolution target area image to determine the target position data of the edge points of the super-resolution target area image in the initial monitoring image.

[0108] If the circumscribed rectangle is used as the edge point of the target area image, the coordinates of the four vertices of the circumscribed rectangle can be used to determine the coordinates of the four vertices of the circumscribed rectangle, thereby determining the lengths of the four sides of the rectangle. After normalization, the changes in the coordinates of the four vertices of the super-resolution target area image after proportionally converting the four sides to the initial monitoring image can be determined.

[0109] S502: Overlay the super-resolution target area image in the area defined by the target position data in the initial monitoring image to obtain the final monitoring image.

[0110] After normalization, the super-resolution target area image has four sides that are consistent with the target area image, so it can be directly spliced ​​into the initial monitoring image, achieving: using the high-resolution, high-definition super-resolution target area image to cover the original low-resolution, low-definition target area image, thereby improving the image quality of the local area of ​​the monitoring image.

[0111] like Figure 5 As shown, the upper left figure is the hand movement recognition result from the initial monitoring image using the prior art, and the lower left figure is the gesture recognition result determined after analyzing the key points of the gesture movement recognition result from the upper left figure. The upper right figure is the hand movement recognition result from the final monitoring image using the technology of the present application, and the lower right figure is the gesture recognition result determined after analyzing the key points of the gesture movement recognition result from the upper right figure. Obviously, after using the solution of the present application to improve the local image quality, the gesture recognition result obtained is more accurate and more closely matches the actual gesture.

[0112] The embodiment of the present application also provides a device for improving the image quality of a local area of ​​an image, such as Figure 6 Shown, including:

[0113] The image acquisition module 10 is configured to acquire an initial monitoring image and determine a target area containing a target in the initial monitoring image;

[0114] An image cropping module 20 is configured to crop the initial monitoring image to obtain a target area image containing the target area;

[0115] a distance calculation module 30, configured to calculate the shooting distance of the target area image;

[0116] The super-resolution processing module 40 is configured to perform super-resolution processing on the target area image to obtain a super-resolution target area image if the shooting distance is greater than a set threshold; the set threshold is a parameter for determining whether the target in the initial monitoring image is clear;

[0117] The image covering module 50 is configured to cover the area where the target area image is located in the initial monitoring image with the super-resolution target area image, so as to obtain a final monitoring image with improved image quality of the local area.

[0118] The above device only performs resolution enhancement processing on target area images whose shooting distance is greater than a set threshold without performing super-resolution processing on the entire captured image, thereby reducing the data processing volume of super-resolution processing and improving processing efficiency.

[0119] Preferably, the distance calculation module 30 is used to calculate the average depth of all pixels in the target area image, and the average depth is used to represent the shooting distance; in the super-resolution processing module 40, if the shooting distance of the target area is greater than a set threshold, it includes: if the average depth is greater than a set depth threshold; wherein the set depth threshold is: the actual average depth of the target area image when the shooting distance is equal to the set threshold. Wherein, the calculation of the average depth of all pixels in the target area image includes: obtaining the depth matrix of the target area image, the depth matrix records the depth value of each pixel in the target area image; calculating the average value of all depth values ​​in the depth matrix as the average depth. The device in this solution simplifies the calculation method of the shooting distance.

[0120] Furthermore, the super-resolution processing module 40 is used to call a super-resolution model, and the super-resolution model is used to improve the resolution of the input image. The super-resolution model is trained by image sample pairs obtained by shooting the target to be identified at different shooting distances; the target area image is input into the super-resolution model for super-resolution processing to obtain the super-resolution target area image. The training process of the super-resolution model includes: obtaining images of the target to be identified at different test positions; wherein, when the target to be identified is at different test positions, the shooting distance from the target to be identified to the camera is different; cropping the target image to be identified that contains the target to be identified in each of the captured images, and dividing the target image to be identified collected at the same test position into a group of image samples; selecting the target image to be identified from any two groups of image samples to form a pair of image sample pairs, and the two groups of target images to be identified in each image sample pair have different resolutions; the super-resolution model is obtained after training the initial super-resolution algorithm using multiple pairs of image samples; the initial super-resolution algorithm includes a learning algorithm based on the CNN structure and a training loss function, and the learning algorithm based on the CNN structure includes a feature extraction network, a feature reconstruction network and an upsampling network; The feature extraction network includes multiple convolutional layers, which extract features from the input image samples. The features extracted by each convolutional layer are simultaneously used as the input of the last convolutional layer. The features extracted by each convolutional layer are combined to obtain different-scale features of the image sample. The feature reconstruction network uses a deconvolution network, which performs feature reconstruction on the different-scale features extracted by the feature extraction network to improve the resolution of the different-scale features. The upsampling network performs interpolation processing on the image samples, and the interpolated image samples are superimposed with the different-scale features with improved resolution output by the feature reconstruction network to obtain image samples with improved resolution. The image samples with improved resolution are used to train the training loss function, and the training results of the training loss function are used to adjust the weight values ​​of each network in the learning algorithm based on the CNN structure. The device in this solution uses a mature algorithm to implement model training to ensure the processing accuracy of the training model.

[0121] In the above scheme, the image cropping module 20 is used to obtain the circumscribed rectangle of the target area in the initial monitoring image; the circumscribed rectangle in the initial monitoring image is cropped to obtain the target area image; the super-resolution processing module 40 is used to obtain the circumscribed rectangle of the target to be identified in each of the captured images; and the circumscribed rectangle in each of the captured images is cropped to obtain the target image to be identified. Such a setting can simplify the cropping process.

[0122] Furthermore, the image overlay module 50 is configured to normalize the position data of the edge points of the super-resolution target area image using the position data of the edge points of the target area image in the initial surveillance image as a reference, determine the target position data of the edge points of the super-resolution target area image in the initial surveillance image, and overlay the super-resolution target area image within the area defined by the target position data in the initial surveillance image to obtain the final surveillance image. This solution can simplify the image stitching process.

[0123] An embodiment of the present application also provides a computer-readable storage medium, in which program information is stored. After a computer reads the program information, it executes the method for improving the image quality of a local area of ​​an image as described in any one of the above method embodiments.

[0124] An embodiment of the present application also provides a computer program product, including a computer program / instruction, characterized in that when the computer program / instruction is executed by a processor, it implements the method for improving the image quality of a local area of ​​an image as described in any of the above method embodiments.

[0125] The present application also provides an electronic device, such as Figure 7As shown, the electronic device includes at least one processor 71 and at least one memory 72. At least one of the memories 72 stores program information. After reading the program information, the at least one processor 71 executes the method for improving the image quality of a local region as described in any of the above method embodiments. The device may also include an input device 73 and an output device 74. The processor 71, memory 72, input device 73, and output device 74 are communicatively connected. Memory 72, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs, and modules. By running the non-volatile software programs, instructions, and modules stored in memory 72, the processor 71 executes various functional applications and data processing, thereby implementing the method for improving the image quality of a local region as described in any of the above methods. Memory 72 may include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function; the data storage area may store data generated by the method for improving the image quality of a local region. In addition, the memory 72 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 72 may optionally include a memory remotely located relative to the processor 71, and these remote memories may be connected to a device that executes the method for improving the image quality of a local area of ​​an image via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The input device 73 may receive input user clicks, and generate signal inputs related to user settings and function controls of the method for improving the image quality of a local area of ​​an image. The output device 74 may include a display device such as a display screen. When the one or more modules are stored in the memory 72 and are executed by the one or more processors 71, the method for improving the image quality of a local area of ​​an image in any of the above-mentioned method embodiments is executed.

[0126] As needed, the above technical solutions can be combined to achieve the best technical effect.

[0127] The above are only the principles and preferred embodiments of the present application. It should be noted that, for those skilled in the art, on the basis of the principles of the present application, several other modifications can be made, which should also be considered as the scope of protection of the present application.

Claims

1. A method for improving the image quality of a local area of ​​an image, characterized in that: include: Acquire an initial monitoring image, and determine a target area containing a target in the initial monitoring image; Cropping the initial monitoring image to obtain a target area image containing the target area; Calculating the shooting distance of the target area; If the shooting distance of the target area is greater than a set threshold, the target area image is super-resolved to obtain a super-resolution target area image; the set threshold is a parameter for determining whether the target in the initial monitoring image is clear; The super-resolution target area image is used to cover the area where the target area image is located in the initial monitoring image, so as to obtain a final monitoring image with improved image quality of the local area.

2. The method for improving the image quality of a local area according to claim 1, characterized in that: The calculating the shooting distance of the target area includes: calculating the average depth of all pixels in the target area image, and using the average depth as the shooting distance of the target area; If the shooting distance of the target area is greater than a set threshold, it includes: if the average depth is greater than a set depth threshold; wherein the set depth threshold is: the actual average depth of the target area image when the shooting distance is equal to the set threshold.

3. The method for improving the image quality of a local area according to claim 2, characterized in that: Calculating the average depth of all pixels in the target area image includes: Obtaining a depth matrix of the target area image, wherein the depth matrix records the depth value of each pixel in the target area image; An average value of all depth values ​​in the depth matrix is ​​calculated, and the average value is used as the average depth.

4. The method for improving the image quality of a local area according to claim 1, characterized in that: The super-resolution processing of the target area image to obtain a super-resolution target area image includes: Retrieving a super-resolution model, wherein the super-resolution model is used to improve the resolution of the input image, and the super-resolution model is trained by image sample pairs obtained by shooting the target to be identified at different shooting distances; The target area image is input into the super-resolution model for super-resolution processing to obtain the super-resolution target area image.

5. The method for improving the image quality of a local area of ​​an image according to claim 4, characterized in that: The training process of the super-resolution model includes: Acquire captured images of the target to be identified at different test positions; wherein, when the target to be identified is at different test positions, the shooting distance from the target to be identified to the camera is different; cropping the target image to be identified that contains the target to be identified in each of the captured images, and dividing the target images to be identified collected at the same test position into a group of image samples; Selecting the target images to be identified from any two groups of image samples to form a pair of image sample pairs, wherein the two target images to be identified in each image sample pair have different resolutions; The super-resolution model is obtained by training an initial super-resolution algorithm using multiple pairs of image samples.

6. The method for improving the image quality of a local area according to claim 5, characterized in that: The initial super-resolution algorithm includes a learning algorithm based on a CNN structure and a training loss function, wherein the learning algorithm based on a CNN structure includes a feature extraction network, a feature reconstruction network and an upsampling network; The feature extraction network includes multiple convolutional layers, which extract features from the input image samples. The features extracted by each convolutional layer are simultaneously used as input to the last convolutional layer. The features extracted by each convolutional layer are combined to obtain different scale features of the image samples. The feature reconstruction network adopts a deconvolution network, and the deconvolution network performs feature reconstruction on the different-scale features extracted by the feature extraction network to improve the resolution of the different-scale features; The upsampling network performs interpolation processing on the image samples, and splices the interpolated image samples with the different scale features with improved resolution output by the feature reconstruction network to obtain image samples with improved resolution; The image samples with increased resolution are used to train the training loss function, and the training results of the training loss function are used to adjust the network weight values ​​in the learning algorithm based on the CNN structure.

7. The method for improving the image quality of a local area according to claim 5, characterized in that: The step of cropping the initial monitoring image to obtain a target area image containing the target area includes: Obtaining a circumscribed rectangle of the target area in the initial monitoring image; cropping the circumscribed rectangle in the initial monitoring image to obtain the target area image; The step of cropping the target image to be identified that contains the target to be identified in each of the captured images comprises: Obtaining the circumscribed rectangle of the target to be identified in each of the captured images; The target image to be identified is obtained by cropping the circumscribed rectangle in each of the captured images.

8. The method for improving the image quality of a local area of ​​an image according to claim 1, wherein: The method of using the super-resolution target area image to cover the area where the target area image is located in the initial monitoring image to obtain a final monitoring image with improved image quality of the local area includes: normalizing the position data of the edge points of the target area image in the initial monitoring image using the position data of the edge points of the target area image in the initial monitoring image as a reference to determine the target position data of the edge points of the super-resolution target area image in the initial monitoring image; The super-resolution target area image is overlaid in the area defined by the target position data in the initial monitoring image to obtain the final monitoring image.

9. A device for improving the image quality of a local area of ​​an image, characterized in that: include: An image acquisition module is configured to acquire an initial monitoring image and determine a target area containing a target in the initial monitoring image; an image cropping module, configured to crop the initial monitoring image to obtain a target area image containing the target area; a distance calculation module, configured to calculate the shooting distance of the target area image; a super-resolution processing module configured to perform super-resolution processing on the target area image to obtain a super-resolution target area image if the shooting distance is greater than a set threshold; the set threshold is a parameter for determining whether the target in the initial monitoring image is clear; The image covering module is configured to use the super-resolution target area image to cover the area where the target area image is located in the initial monitoring image, so as to obtain a final monitoring image with improved image quality of the local area.

10. A computer-readable storage medium, characterized in that The storage medium stores program information, and the computer reads the program information and executes the method for improving the image quality of a local area of ​​an image according to any one of claims 1 to 8.

11. An electronic device, characterized in that: The electronic device includes at least one processor and at least one memory, at least one of the memories stores program information, and at least one of the processors reads the program information and executes the method for improving the image quality of a local area according to any one of claims 1 to 8.

12. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps of the method for improving the image quality of a local area of ​​an image according to any one of claims 1 to 8 are implemented.

Images