Method and system for improving monitoring screen resolution based on high-definition camera

Abstract

The present invention relates to the technical field of monitoring images, a method and system for improving the resolution of monitoring images based on a high-definition camera, comprising: accessing a storage system for storing monitoring images of a high-definition camera, wherein the storage system comprises a cloud storage system and a local storage system, searching and obtaining cloud monitoring images and local monitoring images from the cloud storage system and the local storage system, calculating the image similarity between the real-time monitoring image and the cloud monitoring image, and between the real-time monitoring image and the local monitoring image, obtaining a cloud image similarity value and a local image similarity value, and if the cloud image similarity value is less than or equal to the local image similarity value, performing a resolution improvement operation on the real-time monitoring image to obtain a high-quality monitoring image, storing the high-quality monitoring image in the cloud storage system, and generating a feedback instruction that the resolution has been improved. The present invention can solve the problem of improving the resolution of monitoring images while smoothly realizing monitoring by a high-definition camera.

Description

Technical Field

[0001] The present invention relates to the technical field of monitoring picture management, and in particular to a method, system, electronic device and computer-readable storage medium for improving the resolution of monitoring pictures based on a high-definition camera. Background Art

[0002] The development of modern surveillance systems is increasing day by day. With the improvement of security awareness, the demand for surveillance image resolution is also gradually increasing. High-resolution surveillance images can not only provide clearer and more detailed image information, which helps to improve the recognition accuracy of surveillance scenes, but also provide more accurate surveillance records, which are of great evidentiary significance for criminal investigations, accident investigations, etc. Therefore, improving the surveillance image resolution of high-definition cameras has important practical significance and technical requirements.

[0003] However, there are still some technical drawbacks in the current high-definition camera-based monitoring screen resolution improvement. The traditional method mainly improves the hardware conditions of the high-definition camera, such as lenses, modules, photosensitive elements, etc. to improve the monitoring screen resolution. This method will cause excessive resource investment, which will cause some users to be unable to choose it due to economic reasons, resulting in poor applicability. Another method relies on software optimization, such as building a deep learning model into the high-definition camera and using the deep learning model to improve the monitoring screen resolution. Although this method can achieve the purpose of improving the screen resolution, running the deep learning model requires a large amount of computing resources, which brings too high a computing burden to the high-definition camera, and it is very easy to cause the high-definition camera to run stuck or even crash.

[0004] Therefore, how to balance the high-definition camera to achieve smooth monitoring while improving the resolution of the monitoring picture is a technical problem that needs to be solved urgently. Summary of the invention

[0005] The present invention provides a monitoring picture resolution improvement method based on a high-definition camera and a computer-readable storage medium, the main purpose of which is to solve the problem of improving the monitoring picture resolution while the high-definition camera smoothly realizes monitoring.

[0006] To achieve the above object, the present invention provides a method for improving the resolution of a surveillance image based on a high-definition camera, comprising:

[0007] Receive a command to increase the resolution of a monitoring screen, connect to a high-definition camera according to the command to increase the resolution, and after successfully connecting to the high-definition camera, receive a real-time monitoring screen of the high-definition camera and a monitoring time point corresponding to the real-time monitoring screen, wherein the monitoring time point includes seconds, minutes, and hours;

[0008] Access the storage system of the high-definition camera for storing surveillance images, where the storage system includes a cloud storage system and a local storage system;

[0009] Using the monitoring time point as a search condition, searching historical images from the cloud storage system and the local storage system, and obtaining one or more cloud monitoring images and local monitoring images respectively, wherein the resolution of the cloud monitoring image is greater than that of the local monitoring image, but the number of the local monitoring images is greater than or equal to the number of the cloud monitoring images;

[0010] Calculate the image similarity between the real-time monitoring screen and the cloud monitoring screen, and between the real-time monitoring screen and the local monitoring screen, and obtain the cloud screen similarity value and the local screen similarity value;

[0011] Determine whether the cloud image similarity value is greater than the local image similarity value. If the cloud image similarity value is greater than the local image similarity value, directly store the real-time monitoring image in the local storage system and generate a feedback instruction that the resolution does not need to be improved;

[0012] If the cloud image similarity value is less than or equal to the local image similarity value, a resolution enhancement operation is performed on the real-time monitoring image to obtain a high-quality monitoring image;

[0013] Store high-quality surveillance footage in a cloud storage system and generate feedback instructions with enhanced resolution.

[0014] Optionally, the monitoring time point is used as a search condition to search for historical images from a cloud storage system and a local storage system to obtain one or more cloud monitoring images and local monitoring images respectively, including:

[0015] Receiving a preset time span value, using the time span value and the monitoring time point as a first search condition, searching for the number of historical images from the cloud storage system and the local storage system, respectively, to obtain the number of cloud monitoring images and the number of local monitoring images, respectively;

[0016] Calculate the search interval according to the number of cloud monitoring screens and the number of local monitoring screens, where the search interval consists of an earliest time point and a latest time point, and the structure of the earliest time point and the latest time point are both seconds-minutes-hours;

[0017] Using the earliest time point and the latest time point as the second search condition, historical images are searched in the cloud storage system and the local storage system respectively to obtain one or more cloud monitoring images and local monitoring images, wherein the image generation time of the cloud monitoring images and the local monitoring images are both within the search interval.

[0018] Optionally, calculating the search interval according to the number of cloud monitoring screens and the number of local monitoring screens includes:

[0019] Calculate the ratio of the number of cloud monitoring screens to the number of local monitoring screens to obtain the storage ratio value;

[0020] The search span value for the stored ratio value is calculated according to the following formula:

[0021]

[0022] Among them, D t represents the search span value at the monitoring time point t, M represents the weight factor for calculating the search span value, ρ represents the storage ratio value, and [] represents the rounding symbol;

[0023] With the monitoring time point t as the center and the search span value as the length, the search interval segment is constructed.

[0024] Optionally, respectively calculating the image similarities between the real-time monitoring screen and the cloud monitoring screen, and between the real-time monitoring screen and the local monitoring screen, to obtain a cloud screen similarity value and a local screen similarity value, includes:

[0025] Perform time segmentation based on the search interval segment to obtain one or more groups of time segmentation points;

[0026] Acquire monitoring screens with the same monitoring time point and time segmentation point from all cloud monitoring screens and local monitoring screens, and obtain cloud segmentation screens and local segmentation screens, wherein the number of cloud segmentation screens and the number of local segmentation screens are the same as the number of groups of time segmentation points;

[0027] Calculate the image similarity between the real-time monitoring screen and the cloud segmented screen, and between the real-time monitoring screen and the local segmented screen, and obtain one or more groups of cloud screen similarity values ​​and local screen similarity values;

[0028] The mean of one or more groups of cloud picture similarity values ​​and the mean of the local picture similarity values ​​are calculated respectively to obtain the cloud picture similarity value and the local picture similarity value.

[0029] Optionally, the calculating of image similarities between the real-time monitoring screen and the cloud split screen, and between the real-time monitoring screen and the local split screen, to obtain one or more groups of cloud screen similarity values ​​and local screen similarity values, includes:

[0030] Generate feature matrices of the real-time monitoring screen, cloud segmentation screen, and local segmentation screen, and obtain the real-time feature matrix, cloud segmentation feature matrix, and local segmentation feature matrix, respectively. The structure of the feature matrix is:

[0031] M i =C*N

[0032] Among them, M irepresents a real-time feature matrix, a cloud segmentation feature matrix, or a local segmentation feature matrix, C and N represent the row dimension and column dimension of the feature matrix respectively, when i is equal to 1, it is a real-time feature matrix, when i is equal to 2, it is a cloud segmentation feature matrix, when i is equal to 3, it is a local segmentation feature matrix, and the row dimension is equal to the number of channels of the real-time monitoring screen, the cloud segmentation screen, or the local segmentation screen, and the column dimension is equal to the product of the height and width of the real-time monitoring screen, the cloud segmentation screen, or the local segmentation screen;

[0033] Generate transposed matrices of the real-time feature matrix, the cloud segmentation feature matrix, and the local segmentation feature matrix respectively, and obtain a real-time transposed matrix, a cloud segmentation transposed matrix, and a local segmentation transposed matrix;

[0034] According to the corresponding relationship between the real-time feature matrix and the real-time transposed matrix, the cloud segmentation feature matrix and the cloud segmentation transposed matrix, and the local segmentation feature matrix and the local segmentation transposed matrix, a real-time integration matrix, a cloud segmentation integration matrix, and a local segmentation integration matrix are generated respectively;

[0035] The similarity between the real-time integrated matrix and the cloud segmented integrated matrix, and the similarity between the real-time integrated matrix and the local segmented integrated matrix are calculated to obtain the cloud screen similarity value and the local screen similarity value.

[0036] Optionally, the structures of the real-time integration matrix, the cloud segmentation integration matrix and the local segmentation integration matrix are:

[0037]

[0038] Among them, U i represents the real-time integration matrix, cloud segmentation integration matrix, and local segmentation integration matrix. When i is equal to 1, it is the real-time integration matrix; when i is equal to 2, it is the cloud segmentation integration matrix; when i is equal to 3, it is the local segmentation integration matrix. ij Indicates U i The total eigenvalues ​​of the i-th row and j-th column, and the maximum values ​​of i and j are both C, and u ij The calculation method is:

[0039]

[0040] Among them, m i represents the number of all features in the i-th row of the real-time feature matrix, cloud segmentation feature matrix, or local segmentation feature matrix, m j represents the number of all features in the jth row of the real-time feature matrix, cloud segmentation feature matrix, or local segmentation feature matrix, m i *m j It means that all the feature numbers in the i-th row are multiplied and added together with all the feature numbers in the j-th row. exp is an exponential function.

[0041] Optionally, the calculating the similarity between the real-time integrated matrix and the cloud segmented integrated matrix, and the real-time integrated matrix and the local segmented integrated matrix, to obtain the cloud screen similarity value and the local screen similarity value includes:

[0042] The cloud image similarity value and the local image similarity value are calculated according to the following formula:

[0043]

[0044]

[0045] Among them, S2 represents the cloud screen similarity value between the real-time integrated matrix and the cloud segmentation integrated matrix, S3 represents the local screen similarity value between the real-time integrated matrix and the local segmentation integrated matrix, U1·U2 represents the corresponding multiplication and addition of the eigenvalues ​​of U1 and U2, and U1·U3 represents the corresponding multiplication and addition of the eigenvalues ​​of U1 and U3. represents the addition of all eigenvalues ​​of the real-time integrated matrix.

[0046] Optionally, performing a resolution enhancement operation on the real-time monitoring picture to obtain a high-quality monitoring picture includes:

[0047] Acquire a real-time feature matrix of the real-time monitoring screen, and generate a search box according to the matrix dimension of the real-time feature matrix, wherein the row dimension of the search box is the same as the row dimension of the real-time feature matrix;

[0048] Confirm the local monitoring screen image corresponding to the maximum value among the local screen similarity values, and obtain a highly similar local screen;

[0049] Use the search box to obtain the pixel values ​​of the local area from the highly similar local picture to obtain the local pixel matrix;

[0050] The resolution of the real-time monitoring picture is improved by using a local pixel matrix to obtain the high-quality monitoring picture, wherein the resolution of the real-time monitoring picture is improved by using a local pixel matrix to obtain the high-quality monitoring picture, including:

[0051] Confirm the corresponding position of the local pixel matrix in the real-time monitoring screen to obtain the pixel enhancement area;

[0052] Calculate the difference between the pixel value in the local pixel matrix and the pixel value in the pixel to be promoted area to obtain the pixel difference;

[0053] When the pixel difference is greater than or equal to a preset threshold, a weighted average is performed on the pixel value in the local pixel matrix and the pixel value in the pixel to be promoted area to obtain a pixel replacement value;

[0054] The pixel replacement value is used to replace the pixel values ​​in the pixel improvement area whose pixel difference is greater than or equal to the preset threshold value until a high-quality monitoring picture is obtained.

[0055] Optionally, the structure of the search box is:

[0056] H=C*L

[0057] Among them, H represents the search box of the real-time monitoring screen, C represents the row dimension of the search box, which is the same as the row dimension of the real-time feature matrix, and L represents the column dimension of the search box, and the calculation method of the column dimension is:

[0058]

[0059] Among them, α represents the weight factor of the calculated column dimension, N represents the column dimension of the real-time feature matrix, and [] represents the rounding operation.

[0060] To achieve the above object, the present invention also provides a monitoring picture resolution enhancement system based on a high-definition camera, comprising:

[0061] A storage system access module, used to receive a resolution improvement instruction of a monitoring screen, connect to a high-definition camera according to the resolution improvement instruction, and after successfully connecting to the high-definition camera, receive the real-time monitoring screen of the high-definition camera and the monitoring time point corresponding to the real-time monitoring screen, wherein the monitoring time point includes seconds-minutes-hours, and access the storage system of the high-definition camera for storing the monitoring screen, wherein the storage system includes a cloud storage system and a local storage system;

[0062] A monitoring screen acquisition module, used to search for historical screens from the cloud storage system and the local storage system using the monitoring time point as a search condition, and obtain one or more cloud monitoring screens and local monitoring screens respectively, wherein the resolution of the cloud monitoring screen is greater than that of the local monitoring screen, but the number of local monitoring screens is greater than or equal to the number of cloud monitoring screens;

[0063] The image similarity value calculation module is used to calculate the image similarity between the real-time monitoring image and the cloud monitoring image, and between the real-time monitoring image and the local monitoring image, respectively, to obtain the cloud image similarity value and the local image similarity value, and to determine whether the cloud image similarity value is greater than the local image similarity value. If the cloud image similarity value is greater than the local image similarity value, the real-time monitoring image is directly stored in the local storage system, and a feedback instruction is generated that the resolution does not need to be improved;

[0064] The monitoring picture storage module is used to perform a resolution enhancement operation on the real-time monitoring picture if the cloud picture similarity value is less than or equal to the local picture similarity value to obtain a high-quality monitoring picture, store the high-quality monitoring picture in the cloud storage system, and generate a feedback instruction that the resolution has been enhanced.

[0065] In order to solve the above problem, the present invention further provides an electronic device, the electronic device comprising:

[0066] a memory storing at least one instruction; and

[0067] The processor executes the instructions stored in the memory to implement the above-mentioned method for improving the resolution of monitoring images based on a high-definition camera.

[0068] In order to solve the above problems, the present invention also provides a computer-readable storage medium, in which at least one instruction is stored. The at least one instruction is executed by a processor in an electronic device to implement the above-mentioned method for improving the resolution of monitoring images based on a high-definition camera.

[0069] The present invention solves the problem described in the background technology. The present invention utilizes a cloud storage system and a local storage system, combines the comparison of real-time monitoring images with historical monitoring images, and a screening mechanism based on similarity calculation to improve the resolution of monitoring images, avoid the use of deep learning models to bring a computing burden to the monitoring system where the high-definition camera is located, and thus ensure the process operation of the high-definition camera. In detail, the present invention makes full use of the resources of the cloud storage system and the local storage system. By combining with the cloud storage system and the local storage system, it can achieve optimization in the effective use of storage resources, and avoid bringing an overload burden to the storage system of the monitoring system where the high-definition camera is located; in addition, the present invention determines whether it is necessary to improve the resolution of the real-time monitoring image by comparing the real-time monitoring image with the historical monitoring image. If the real-time monitoring image has a high similarity with the historical monitoring image, the real-time monitoring image can be directly stored in the local storage system without the need for resolution improvement. This measure can selectively improve the resolution of the monitoring image, thereby preventing each frame from needing to improve the resolution, resulting in an excessive computing burden on the monitoring system; finally, the computational complexity of the similarity calculation method and the resolution improvement method used in the present invention is lower than that of the deep learning model, thereby further ensuring the process operation of the high-definition camera while improving the resolution of the monitoring image. Therefore, the present invention can solve the problem of improving the resolution of monitoring pictures while realizing smooth monitoring with a high-definition camera. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 A schematic diagram of a process for improving the resolution of a surveillance image based on a high-definition camera provided by an embodiment of the present invention;

[0071] Figure 2 A functional module diagram of a monitoring image resolution enhancement system based on a high-definition camera provided in one embodiment of the present invention;

[0072] Figure 3 A schematic diagram of the structure of an electronic device for implementing the method for improving the resolution of monitoring images based on a high-definition camera provided in one embodiment of the present invention.

[0073] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0074] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0075] The embodiment of the present application provides a method for improving the resolution of monitoring images based on a high-definition camera. The execution subject of the method for improving the resolution of monitoring images based on a high-definition camera includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the method provided by the embodiment of the present application. In other words, the method for improving the resolution of monitoring images based on a high-definition camera can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc.

[0076] Reference Figure 1 FIG. 1 is a flow chart of a method for improving the resolution of a surveillance image based on a high-definition camera provided by an embodiment of the present invention. In this embodiment, the method for improving the resolution of a surveillance image based on a high-definition camera includes:

[0077] S1. Receive a resolution enhancement command for a monitoring screen, connect to a high-definition camera according to the resolution enhancement command, and after successfully connecting to the high-definition camera, receive the real-time monitoring screen of the high-definition camera and the monitoring time point corresponding to the real-time monitoring screen, wherein the monitoring time point includes seconds-minutes-hours.

[0078] It should be explained that the command to increase the resolution of the surveillance images is generally initiated by the managers of the high-definition cameras. For example, Xiao Zhang is a security guard in a certain park. The surveillance equipment used in the park are all high-definition cameras. However, due to long-term monitoring, too many surveillance images are generated, so that the subsequent continuously generated surveillance images cannot be saved efficiently, and it also puts pressure on the storage system of the surveillance camera. Therefore, in order to ensure that the resolution of the surveillance images can help managers replay and trace the real-time effectiveness of historical events, and also to reduce the storage pressure of the storage system, Xiao Zhang initiated the resolution increase command.

[0079] Furthermore, after Xiao Zhang initiates a resolution enhancement command, the resolution enhancement command can be used to connect to the high-definition camera A of the park, so that the monitoring picture captured by the high-definition camera A can be obtained in real time, that is, the real-time monitoring picture.

[0080] It is understandable that the real-time monitoring screen corresponds to a monitoring time point, such as 2024-March 15-7:00-50:20. The embodiment of the present invention needs to focus on the seconds-minutes-hours included in the monitoring time point, which is of great significance to the subsequent real-time monitoring screen.

[0081] S2. Access a storage system of a high-definition camera for storing surveillance images, wherein the storage system includes a cloud storage system and a local storage system.

[0082] Exemplarily, when the connection to high-definition camera A is successful, a storage system for storing the monitoring images of high-definition camera A can be accessed, and the storage system includes a cloud storage system and a local storage system.

[0083] In the embodiment of the present invention, the real-time monitoring images captured by the high-definition camera are all high-definition images with a resolution of at least 1080P. However, it is understandable that long-term storage of high-definition images will cause storage burden on the storage system, so the local storage system has a large storage capacity and is used to store the real-time monitoring images captured by the high-definition camera. The cloud storage system is more secure and is used to store the monitoring images after the resolution of the real-time monitoring images is increased.

[0084] In addition, the images stored in the cloud storage system and the images stored in the local storage system are connected through monitoring time points. Assume that 2,000 real-time monitoring images are generated in 1 hour from 5:00 on June 1, 2024 to 6:00 on June 1, 2024, of which 1,800 are stored in the local storage system and 200 are stored in the cloud storage system. It is understandable that the 200 real-time monitoring images stored in the cloud storage system are all monitoring images with enhanced resolution. When Xiao Zhang wants to replay the monitoring images of this 1 hour, the embodiment of the present invention reorganizes the above 2,000 real-time monitoring images through the corresponding relationship of the monitoring time points, thereby achieving the purpose of monitoring playback.

[0085] S3. Using the monitoring time point as a search condition, search for historical images from the cloud storage system and the local storage system to obtain one or more cloud monitoring images and local monitoring images, respectively, wherein the resolution of the cloud monitoring image is greater than that of the local monitoring image, but the number of local monitoring images is greater than or equal to the number of cloud monitoring images.

[0086] In detail, the monitoring time point is used as a search condition to search for historical images from the cloud storage system and the local storage system to obtain one or more cloud monitoring images and local monitoring images respectively, including:

[0087] Receiving a preset time span value, using the time span value and the monitoring time point as a first search condition, searching for the number of historical images from the cloud storage system and the local storage system, respectively, to obtain the number of cloud monitoring images and the number of local monitoring images, respectively;

[0088] Calculate the search interval according to the number of cloud monitoring screens and the number of local monitoring screens, where the search interval consists of an earliest time point and a latest time point, and the structure of the earliest time point and the latest time point are both seconds-minutes-hours;

[0089] Using the earliest time point and the latest time point as the second search condition, historical images are searched in the cloud storage system and the local storage system respectively to obtain one or more cloud monitoring images and local monitoring images, wherein the image generation time of the cloud monitoring images and the local monitoring images are both within the search interval.

[0090] For example, assuming that the preset time span value is 2 minutes, and the real-time monitoring screen obtained by Xiao Zhang above corresponds to the monitoring time point of 2024-March 15-7:00-50:20 seconds, then 7:50:20 seconds and 2 minutes are used as the first search conditions, and the number of historical screens with a difference of 2 minutes before and after 7:50:20 seconds is obtained from the cloud storage system and the local storage system. It is assumed that the number of cloud monitoring screens is 50 and the number of local monitoring screens is 1000.

[0091] It should be explained that the number of cloud monitoring screens and the number of local monitoring screens reflect the storage ratio values ​​of the cloud storage system and the local storage system at the said monitoring time point. If the number of cloud monitoring screens is larger than the storage ratio at other monitoring time points, it means that the probability of the real-time monitoring screen at this monitoring time point being important is often greater.

[0092] Exemplarily, the search interval segment needs to be further calculated based on the number of 50 cloud monitoring images and the number of 1000 local monitoring images. Specifically, the search interval segment is calculated based on the number of cloud monitoring images and the number of local monitoring images, including:

[0093] Calculate the ratio of the number of cloud monitoring screens to the number of local monitoring screens to obtain the storage ratio value;

[0094] The search span value for the stored ratio value is calculated according to the following formula:

[0095]

[0096] Among them, D t represents the search span value at the monitoring time point t, M represents the weight factor for calculating the search span value, ρ represents the storage ratio value, and [] represents the rounding symbol;

[0097] With the monitoring time point t as the center and the search span value as the length, the search interval segment is constructed.

[0098] For example, the storage ratio of 50 cloud surveillance images to 1000 local surveillance images is 0.05, that is, ρ = 0.05, and then substituted into the above formula, assuming that the search span value D is calculated t is 8, wherein the value of M in the embodiment of the present invention is 10, 15 or 20.

[0099] Furthermore, if the monitoring time point t is 7:00-50:20, then with 7:50:20 as the center, the earliest time point is 7:46:20, and the latest time point is 7:54:20. That is, the structure of the earliest time point and the latest time point are both seconds-minutes-hours, and the search interval segment is [7:46:20, 7:54:20].

[0100] By way of example, using [7:46:20, 7:54:20] as the second search condition, historical images belonging to [7:46:20, 7:54:20] are searched in the cloud storage system and the local storage system respectively, and one or more cloud monitoring images and local monitoring images are obtained.

[0101] S4. Calculate the image similarity between the real-time monitoring screen and the cloud monitoring screen, and between the real-time monitoring screen and the local monitoring screen, respectively, to obtain a cloud screen similarity value and a local screen similarity value.

[0102] In detail, respectively calculating the image similarity between the real-time monitoring screen and the cloud monitoring screen, and between the real-time monitoring screen and the local monitoring screen, and obtaining the cloud screen similarity value and the local screen similarity value, includes:

[0103] Perform time segmentation based on the search interval segment to obtain one or more groups of time segmentation points;

[0104] Acquire monitoring screens with the same monitoring time point and time segmentation point from all cloud monitoring screens and local monitoring screens, and obtain cloud segmentation screens and local segmentation screens, wherein the number of cloud segmentation screens and the number of local segmentation screens are the same as the number of groups of time segmentation points;

[0105] Calculate the image similarity between the real-time monitoring screen and the cloud segmented screen, and between the real-time monitoring screen and the local segmented screen, and obtain one or more groups of cloud screen similarity values ​​and local screen similarity values;

[0106] The mean of one or more groups of cloud picture similarity values ​​and the mean of the local picture similarity values ​​are calculated respectively to obtain the cloud picture similarity value and the local picture similarity value.

[0107] It is understandable that using the search interval segment composed of [earliest time point, latest time point] as the second search condition may simultaneously search out multiple cloud monitoring screens and local monitoring screens. If the similarity between the real-time monitoring screen and each cloud monitoring screen and local monitoring screen is calculated in sequence, it will cause excessive waste of resources. Therefore, the embodiment of the present invention first performs a time segmentation operation. Exemplarily, the above [7:46:20, 7:54:20] is divided into 8 groups of time segmentation points, that is, at 7:46:20, 7:47:20, ..., 7:54:20, the corresponding monitoring screens are obtained from all cloud monitoring screens and local monitoring screens, and 8 groups of cloud segmentation screens and local segmentation screens are obtained. It can be seen that the number of cloud segmentation screens and the number of local segmentation screens are the same as the number of 8 groups of time segmentation points.

[0108] In addition, if no cloud split screen or local split screen corresponding to the time split point is found from the cloud monitoring screen or the local monitoring screen, the monitoring screen closest in time to the time split point can be selected as the cloud split screen or the local split screen.

[0109] The key point is that the traditional method of calculating image similarity is relatively cumbersome, that is, according to the pixel correspondence between the two pictures, the difference is calculated in turn and the similarity value is obtained by normalization mapping. Although this method can realize the calculation of image similarity, it is not applicable to monitoring equipment whose main purpose is timeliness due to excessive occupation of computing resources. Therefore, an embodiment of the present invention proposes a new similarity calculation method. In detail, the image similarity between the real-time monitoring screen and the cloud split screen, and the real-time monitoring screen and the local split screen is calculated to obtain one or more groups of cloud screen similarity values ​​and local screen similarity values, including:

[0110] Generate feature matrices of the real-time monitoring screen, cloud segmentation screen, and local segmentation screen, and obtain the real-time feature matrix, cloud segmentation feature matrix, and local segmentation feature matrix, respectively. The structure of the feature matrix is:

[0111] M i =C*N

[0112] Among them, M i represents a real-time feature matrix, a cloud segmentation feature matrix, or a local segmentation feature matrix, C and N represent the row dimension and column dimension of the feature matrix respectively, when i is equal to 1, it is a real-time feature matrix, when i is equal to 2, it is a cloud segmentation feature matrix, when i is equal to 3, it is a local segmentation feature matrix, and the row dimension is equal to the number of channels of the real-time monitoring screen, the cloud segmentation screen, or the local segmentation screen, and the column dimension is equal to the product of the height and width of the real-time monitoring screen, the cloud segmentation screen, or the local segmentation screen;

[0113] Generate transposed matrices of the real-time feature matrix, the cloud segmentation feature matrix, and the local segmentation feature matrix respectively, and obtain a real-time transposed matrix, a cloud segmentation transposed matrix, and a local segmentation transposed matrix;

[0114] According to the corresponding relationship between the real-time feature matrix and the real-time transposed matrix, the cloud segmentation feature matrix and the cloud segmentation transposed matrix, and the local segmentation feature matrix and the local segmentation transposed matrix, a real-time integration matrix, a cloud segmentation integration matrix, and a local segmentation integration matrix are generated respectively;

[0115] The similarity between the real-time integrated matrix and the cloud segmented integrated matrix, and the similarity between the real-time integrated matrix and the local segmented integrated matrix are calculated to obtain the cloud screen similarity value and the local screen similarity value.

[0116] For example, assuming that the structure of the real-time monitoring screen is: 3*10*5, it means that the number of channels of the real-time monitoring screen is 3, and the height and width are 10 and 5 respectively, so the corresponding real-time feature matrix M1=3*15 is generated.

[0117] In detail, the structures of the real-time integration matrix, the cloud segmentation integration matrix and the local segmentation integration matrix are as follows:

[0118]

[0119] Among them, U i represents the real-time integration matrix, cloud segmentation integration matrix, and local segmentation integration matrix. When i is equal to 1, it is the real-time integration matrix; when i is equal to 2, it is the cloud segmentation integration matrix; when i is equal to 3, it is the local segmentation integration matrix. ij Indicates U i The total eigenvalues ​​of the i-th row and j-th column, and the maximum values ​​of i and j are both C, and u ij The calculation method is:

[0120]

[0121] Among them, m i represents the number of all features in the i-th row of the real-time feature matrix, cloud segmentation feature matrix, or local segmentation feature matrix, m j represents the number of all features in the jth row of the real-time feature matrix, cloud segmentation feature matrix, or local segmentation feature matrix, m i *m j It means that all the feature numbers in the i-th row are multiplied and added together with all the feature numbers in the j-th row. exp is an exponential function.

[0122] It is understandable that the main purpose of generating the real-time transposed matrix, the cloud split transposed matrix and the local split transposed matrix is ​​to further control the dimension of the entire feature matrix. For example, if the above-mentioned real-time feature matrix M1=3*15, the dimension of the generated real-time transposed matrix is: 15*3. Therefore, if the real-time feature matrix and the real-time transposed matrix are multiplied, the final dimension can be changed to 3*3. It can be seen that the dimension of the entire feature matrix is ​​greatly reduced (that is, the matrix dimensions of the real-time integrated matrix, the cloud split integrated matrix and the local split integrated matrix are all 3*3), which is more conducive to calculating the similarity between the corresponding real-time monitoring screen and the cloud split screen, and the real-time monitoring screen and the local split screen.

[0123] In addition, compared with the traditional method of directly multiplying the transposed matrix and the original matrix, the embodiment of the present invention redefines the multiplication method, that is, u ij The generation process of adds the step of dividing the numerator and the denominator, so as to effectively control each feature number u in the real-time integration matrix, the cloud segmentation integration matrix and the local segmentation integration matrix ij The numerical value of ij Too large a value will put computing pressure on the HD camera.

[0124] Furthermore, the calculating of the similarity between the real-time integrated matrix and the cloud segmented integrated matrix, and the real-time integrated matrix and the local segmented integrated matrix, to obtain the cloud screen similarity value and the local screen similarity value includes:

[0125] The cloud image similarity value and the local image similarity value are calculated according to the following formula:

[0126]

[0127]

[0128] Among them, S2 represents the cloud screen similarity value between the real-time integrated matrix and the cloud segmentation integrated matrix, S3 represents the local screen similarity value between the real-time integrated matrix and the local segmentation integrated matrix, U1·U2 represents the corresponding multiplication and addition of the eigenvalues ​​of U1 and U2, and U1·U3 represents the corresponding multiplication and addition of the eigenvalues ​​of U1 and U3. represents the addition of all eigenvalues ​​of the real-time integrated matrix.

[0129] According to the above description, the embodiment of the present invention first constructs a real-time feature matrix, a cloud segmentation feature matrix and a local segmentation feature matrix according to the real-time monitoring screen, the cloud monitoring screen and the local monitoring screen, and generates a real-time integrated matrix, a cloud segmentation integrated matrix and a local segmentation integrated matrix with smaller dimensions based on the strategy that the matrix dimension can be reduced by multiplying the transposed matrix, and finally calculates the similarity based on the real-time integrated matrix and the cloud segmentation integrated matrix, and the real-time integrated matrix and the local segmentation integrated matrix. Compared with the traditional method, the calculation amount is greatly reduced while the calculation accuracy of the similarity can be guaranteed.

[0130] S5. Determine whether the cloud image similarity value is greater than the local image similarity value. If the cloud image similarity value is greater than the local image similarity value, directly store the real-time monitoring image in the local storage system and generate a feedback instruction that the resolution does not need to be improved.

[0131] It should be emphasized that the size relationship between the cloud image similarity value and the local image similarity value can directly indicate whether the real-time monitoring image is more similar to the cloud monitoring image or the local monitoring image. If the cloud image similarity value is greater than the local image similarity value, it means that the real-time monitoring image generated at this time has a higher similarity with the cloud monitoring image. Therefore, if there is a cloud monitoring image with a high similarity to the real-time monitoring image generated at this time in the cloud storage system, the real-time monitoring image can be directly stored in the local storage system.

[0132] S6. If the cloud image similarity value is less than or equal to the local image similarity value, a resolution enhancement operation is performed on the real-time monitoring image to obtain a high-quality monitoring image.

[0133] It is understandable that when the cloud image similarity value is less than or equal to the local image similarity value, it indicates that the real-time monitoring image generated at this time has a high similarity with the local monitoring image of the local storage system, but has a large difference with the cloud monitoring image in the cloud storage system. As described in the embodiments of the present invention, the cloud monitoring image in the cloud storage system is a security operation performed to prevent the loss of important monitoring images due to abnormal operation of the local storage system. Therefore, when the cloud image similarity value is less than or equal to the local image similarity value, the embodiments of the present invention need to perform a resolution enhancement operation on the real-time monitoring image.

[0134] In detail, performing a resolution enhancement operation on the real-time monitoring picture to obtain a high-quality monitoring picture includes:

[0135] Acquire a real-time feature matrix of the real-time monitoring screen, and generate a search box according to the matrix dimension of the real-time feature matrix, wherein the row dimension of the search box is the same as the row dimension of the real-time feature matrix;

[0136] Confirm the local monitoring screen image corresponding to the maximum value among the local screen similarity values, and obtain a highly similar local screen;

[0137] Use the search box to obtain the pixel values ​​of the local area from the highly similar local picture to obtain the local pixel matrix;

[0138] The resolution of the real-time monitoring picture is improved by using a local pixel matrix to obtain the high-quality monitoring picture, wherein the resolution of the real-time monitoring picture is improved by using a local pixel matrix to obtain the high-quality monitoring picture, including:

[0139] Confirm the corresponding position of the local pixel matrix in the real-time monitoring screen to obtain the pixel enhancement area;

[0140] Calculate the difference between the pixel value in the local pixel matrix and the pixel value in the pixel to be promoted area to obtain the pixel difference;

[0141] When the pixel difference is greater than or equal to a preset threshold, a weighted average is performed on the pixel value in the local pixel matrix and the pixel value in the pixel to be promoted area to obtain a pixel replacement value;

[0142] The pixel replacement value is used to replace the pixel values ​​in the pixel improvement area whose pixel difference is greater than or equal to the preset threshold value until a high-quality monitoring picture is obtained.

[0143] Exemplarily, the real-time feature matrix M1=3*15, which means that the row dimension of the generated search box is also 3. Therefore, in detail, the structure of the search box is:

[0144] H=C*L

[0145] Among them, H represents the search box of the real-time monitoring screen, C represents the row dimension of the search box, which is the same as the row dimension of the real-time feature matrix, and L represents the column dimension of the search box, and the calculation method of the column dimension is:

[0146]

[0147] Among them, α represents the weight factor of the calculated column dimension, N represents the column dimension of the real-time feature matrix, and [] represents the rounding operation.

[0148] For example, assuming that the calculated search box H = 3*9, it means that a 3*9 search box is used to obtain the local pixel matrix from left to right and from top to bottom in the high-similarity local picture. It can be seen that the dimension of the local pixel matrix is ​​also 3*9, and there are 27 groups of pixel values ​​in the local pixel matrix.

[0149] Furthermore, assuming that the local pixel matrix is ​​in the uppermost area on the left, that is, the area to be improved for pixels of the real-time monitoring screen is also the uppermost area on the left, then the pixel difference, threshold judgment and pixel replacement operations are calculated according to the above method until a high-quality monitoring screen is obtained.

[0150] In addition, the resolution enhancement operation described in the embodiment of the present invention also includes digital image processing methods such as image difference processing, denoising processing, and resampling, that is, the resolution of the real-time monitoring screen is improved by performing a series of digital image processing on the real-time monitoring screen. Compared with the image resolution enhancement method of the deep learning model, the computing resources occupied by this type of digital image processing method are not high, and it can ensure that the high-definition camera performs the resolution enhancement operation when operating normally.

[0151] S7. Storing the high-quality monitoring images in a cloud storage system and generating feedback instructions with improved resolution.

[0152] It should be understood that when the storage is completed, a feedback instruction indicating that the resolution does not need to be improved and a feedback instruction indicating that the resolution has been improved will be generated. However, in general, the feedback instruction indicating that the resolution does not need to be improved and the feedback instruction indicating that the subsequent resolution has been improved will be sent to the initiator of the resolution improvement instruction at the same time to tell the initiator of the resolution improvement instruction how many real-time monitoring images have been improved in the specified time period.

[0153] The present invention solves the problem described in the background technology. The present invention utilizes a cloud storage system and a local storage system, combines the comparison of real-time monitoring images with historical monitoring images, and a screening mechanism based on similarity calculation to improve the resolution of monitoring images, avoid the use of deep learning models to bring a computing burden to the monitoring system where the high-definition camera is located, and thus ensure the process operation of the high-definition camera. In detail, the present invention makes full use of the resources of the cloud storage system and the local storage system. By combining with the cloud storage system and the local storage system, it can achieve optimization in the effective use of storage resources, and avoid bringing an overload burden to the storage system of the monitoring system where the high-definition camera is located; in addition, the present invention determines whether it is necessary to improve the resolution of the real-time monitoring image by comparing the real-time monitoring image with the historical monitoring image. If the real-time monitoring image has a high similarity with the historical monitoring image, the real-time monitoring image can be directly stored in the local storage system without the need for resolution improvement. This measure can selectively improve the resolution of the monitoring image, thereby preventing each frame from needing to improve the resolution, resulting in an excessive computing burden on the monitoring system; finally, the computational complexity of the similarity calculation method and the resolution improvement method used in the present invention is lower than that of the deep learning model, thereby further ensuring the process operation of the high-definition camera while improving the resolution of the monitoring image. Therefore, the present invention can solve the problem of improving the resolution of monitoring pictures while realizing smooth monitoring with a high-definition camera.

[0154] like Figure 2 , which is a functional module diagram of a monitoring image resolution enhancement system based on a high-definition camera provided in one embodiment of the present invention.

[0155] The monitoring screen resolution enhancement system 100 based on a high-definition camera of the present invention can be installed in an electronic device. According to the functions to be implemented, the monitoring screen resolution enhancement system 100 based on a high-definition camera can include a storage system access module 101, a monitoring screen acquisition module 102, a screen similarity value calculation module 103 and a monitoring screen storage module 104. The module of the present invention can also be called a unit, which refers to a series of computer program segments that can be executed by a processor of an electronic device and can complete fixed functions, which are stored in the memory of the electronic device.

[0156] The storage system access module 101 is used to receive a resolution improvement instruction of a monitoring screen, connect to a high-definition camera according to the resolution improvement instruction, and after successfully connecting to the high-definition camera, receive the real-time monitoring screen of the high-definition camera and the monitoring time point corresponding to the real-time monitoring screen, wherein the monitoring time point includes seconds-minutes-hours, and access the storage system of the high-definition camera for storing the monitoring screen, wherein the storage system includes a cloud storage system and a local storage system;

[0157] The monitoring screen acquisition module 102 is used to search for historical screens from the cloud storage system and the local storage system using the monitoring time point as a search condition, and obtain one or more cloud monitoring screens and local monitoring screens respectively, wherein the resolution of the cloud monitoring screen is greater than that of the local monitoring screen, but the number of local monitoring screens is greater than or equal to the number of cloud monitoring screens;

[0158] The picture similarity value calculation module 103 is used to respectively calculate the image similarity between the real-time monitoring picture and the cloud monitoring picture, and between the real-time monitoring picture and the local monitoring picture, obtain the cloud picture similarity value and the local picture similarity value, and determine whether the cloud picture similarity value is greater than the local picture similarity value. If the cloud picture similarity value is greater than the local picture similarity value, the real-time monitoring picture is directly stored in the local storage system, and a feedback instruction is generated that the resolution does not need to be improved;

[0159] The monitoring picture storage module 104 is used to perform a resolution enhancement operation on the real-time monitoring picture if the cloud picture similarity value is less than or equal to the local picture similarity value to obtain a high-quality monitoring picture, store the high-quality monitoring picture in the cloud storage system, and generate a feedback instruction that the resolution has been enhanced.

[0160] In detail, the modules in the high-definition camera-based monitoring image resolution enhancement system 100 in the embodiment of the present invention are used in the same manner as described above. Figure 1 The same technical means as the method for improving the monitoring image resolution based on high-definition cameras described in the text can produce the same technical effects, so they will not be repeated here.

[0161] like Figure 3, is a schematic diagram of the structure of an electronic device for implementing a method for improving the resolution of a monitoring image based on a high-definition camera provided by an embodiment of the present invention.

[0162] The electronic device 1 may include a processor 10, a memory 11 and a bus 12, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as a monitoring image resolution enhancement method program based on a high-definition camera.

[0163] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. The memory 11 may be an internal storage unit of the electronic device 1 in some embodiments, such as a mobile hard disk of the electronic device 1. The memory 11 may also be an external storage device of the electronic device 1 in other embodiments, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 1. Further, the memory 11 also includes an internal storage unit of the electronic device 1 and an external storage device. The memory 11 can not only be used to store application software and various types of data installed in the electronic device 1, such as the code of the monitoring screen resolution enhancement method program based on a high-definition camera, but also can be used to temporarily store data that has been output or is to be output.

[0164] The processor 10 may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips, etc. The processor 10 is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes or executes programs or modules stored in the memory 11 (such as a monitoring screen resolution enhancement method program based on a high-definition camera, etc.), and calls data stored in the memory 11 to execute various functions of the electronic device 1 and process data.

[0165] The bus 12 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus 12 may be divided into an address bus, a data bus, a control bus, etc. The bus 12 is configured to realize connection and communication between the memory 11 and at least one processor 10, etc.

[0166] Figure 3 Only an electronic device with components is shown, and those skilled in the art will understand that Figure 3 The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0167] For example, although not shown, the electronic device 1 may also include a power source (such as a battery) for supplying power to each component. Preferably, the power source may be logically connected to the at least one processor 10 through a power management device, so that the power management device can realize functions such as charging management, discharging management, and power consumption management. The power source may also include any components such as one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, etc. The electronic device 1 may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

[0168] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device 1 and other electronic devices.

[0169] Optionally, the electronic device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), or a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device 1 and to display a visual user interface.

[0170] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0171] The monitoring picture resolution enhancement method program based on a high-definition camera stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When running in the processor 10, it can achieve:

[0172] Receive a command to increase the resolution of a monitoring screen, connect to a high-definition camera according to the command to increase the resolution, and after successfully connecting to the high-definition camera, receive a real-time monitoring screen of the high-definition camera and a monitoring time point corresponding to the real-time monitoring screen, wherein the monitoring time point includes seconds, minutes, and hours;

[0173] Access the storage system of the high-definition camera for storing surveillance images, where the storage system includes a cloud storage system and a local storage system;

[0174] Using the monitoring time point as a search condition, searching historical images from the cloud storage system and the local storage system, and obtaining one or more cloud monitoring images and local monitoring images respectively, wherein the resolution of the cloud monitoring image is greater than that of the local monitoring image, but the number of the local monitoring images is greater than or equal to the number of the cloud monitoring images;

[0175] Calculate the image similarity between the real-time monitoring screen and the cloud monitoring screen, and between the real-time monitoring screen and the local monitoring screen, and obtain the cloud screen similarity value and the local screen similarity value;

[0176] Determine whether the cloud image similarity value is greater than the local image similarity value. If the cloud image similarity value is greater than the local image similarity value, directly store the real-time monitoring image in the local storage system and generate a feedback instruction that the resolution does not need to be improved;

[0177] If the cloud image similarity value is less than or equal to the local image similarity value, a resolution enhancement operation is performed on the real-time monitoring image to obtain a high-quality monitoring image;

[0178] Store high-quality surveillance footage in a cloud storage system and generate feedback instructions with enhanced resolution.

[0179] Specifically, the specific implementation method of the processor 10 for the above instructions can refer to Figures 1 to 3 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0180] Furthermore, if the module / unit integrated in the electronic device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0181] The present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor of an electronic device, the computer program can implement:

[0182] Receive a command to increase the resolution of a monitoring screen, connect to a high-definition camera according to the command to increase the resolution, and after successfully connecting to the high-definition camera, receive a real-time monitoring screen of the high-definition camera and a monitoring time point corresponding to the real-time monitoring screen, wherein the monitoring time point includes seconds, minutes, and hours;

[0183] Access the storage system of the high-definition camera for storing surveillance images, where the storage system includes a cloud storage system and a local storage system;

[0184] Using the monitoring time point as a search condition, searching historical images from the cloud storage system and the local storage system, and obtaining one or more cloud monitoring images and local monitoring images respectively, wherein the resolution of the cloud monitoring image is greater than that of the local monitoring image, but the number of the local monitoring images is greater than or equal to the number of the cloud monitoring images;

[0185] Calculate the image similarity between the real-time monitoring screen and the cloud monitoring screen, and between the real-time monitoring screen and the local monitoring screen, and obtain the cloud screen similarity value and the local screen similarity value;

[0186] Determine whether the cloud image similarity value is greater than the local image similarity value. If the cloud image similarity value is greater than the local image similarity value, directly store the real-time monitoring image in the local storage system and generate a feedback instruction that the resolution does not need to be improved;

[0187] If the cloud image similarity value is less than or equal to the local image similarity value, a resolution enhancement operation is performed on the real-time monitoring image to obtain a high-quality monitoring image;

[0188] Store high-quality surveillance footage in a cloud storage system and generate feedback instructions with enhanced resolution.

[0189] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, systems and methods can be implemented in other ways. For example, the system embodiments described above are only illustrative, and actual implementation may have other division methods.

[0190] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0191] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0192] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0193] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the system claim can also be implemented by one unit or device through software or hardware. The second and other words are used to indicate names, but not to indicate any particular order.

[0194] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for improving the resolution of monitoring images based on a high-definition camera, characterized in that: The method comprises: Receive a command to increase the resolution of a monitoring screen, connect to a high-definition camera according to the command to increase the resolution, and after successfully connecting to the high-definition camera, receive a real-time monitoring screen of the high-definition camera and a monitoring time point corresponding to the real-time monitoring screen, wherein the monitoring time point includes seconds, minutes, and hours; Access the storage system of the high-definition camera for storing surveillance images, where the storage system includes a cloud storage system and a local storage system; Using the monitoring time point as a search condition, searching historical images from the cloud storage system and the local storage system, and obtaining one or more cloud monitoring images and local monitoring images respectively, wherein the resolution of the cloud monitoring image is greater than that of the local monitoring image, but the number of the local monitoring images is greater than or equal to the number of the cloud monitoring images; Calculate the image similarity between the real-time monitoring screen and the cloud monitoring screen, and between the real-time monitoring screen and the local monitoring screen, and obtain the cloud screen similarity value and the local screen similarity value; Determine whether the cloud image similarity value is greater than the local image similarity value. If the cloud image similarity value is greater than the local image similarity value, directly store the real-time monitoring image in the local storage system and generate a feedback instruction that the resolution does not need to be improved; If the cloud image similarity value is less than or equal to the local image similarity value, a resolution enhancement operation is performed on the real-time monitoring image to obtain a high-quality monitoring image; Store high-quality surveillance footage in a cloud storage system and generate feedback instructions with enhanced resolution.

2. The method for improving the monitoring image resolution based on a high-definition camera according to claim 1, characterized in that: The method uses the monitoring time point as a search condition, searches for historical images from the cloud storage system and the local storage system, and obtains one or more cloud monitoring images and local monitoring images respectively, including: Receiving a preset time span value, using the time span value and the monitoring time point as a first search condition, searching for the number of historical images from the cloud storage system and the local storage system, respectively, to obtain the number of cloud monitoring images and the number of local monitoring images, respectively; Calculate the search interval according to the number of cloud monitoring screens and the number of local monitoring screens, where the search interval consists of an earliest time point and a latest time point, and the structure of the earliest time point and the latest time point are both seconds-minutes-hours; Using the earliest time point and the latest time point as the second search condition, historical images are searched in the cloud storage system and the local storage system respectively to obtain one or more cloud monitoring images and local monitoring images, wherein the image generation time of the cloud monitoring images and the local monitoring images are both within the search interval.

3. The method for improving the monitoring image resolution based on a high-definition camera as claimed in claim 2, characterized in that: The step of calculating the search interval according to the number of cloud monitoring screens and the number of local monitoring screens includes: Calculate the ratio of the number of cloud monitoring screens to the number of local monitoring screens to obtain the storage ratio value; The search span value for the stored ratio value is calculated according to the following formula: Among them, D t represents the search span value at the monitoring time point t, M represents the weight factor for calculating the search span value, ρ represents the storage ratio value, and [] represents the rounding symbol; With the monitoring time point t as the center and the search span value as the length, the search interval segment is constructed.

4. The method for improving the monitoring image resolution based on a high-definition camera as claimed in claim 3, characterized in that: The method of respectively calculating the image similarity between the real-time monitoring screen and the cloud monitoring screen, and between the real-time monitoring screen and the local monitoring screen, and obtaining the cloud screen similarity value and the local screen similarity value, includes: Perform time segmentation based on the search interval segment to obtain one or more groups of time segmentation points; Acquire monitoring screens with the same monitoring time point and time segmentation point from all cloud monitoring screens and local monitoring screens, and obtain cloud segmentation screens and local segmentation screens, wherein the number of cloud segmentation screens and the number of local segmentation screens are the same as the number of groups of time segmentation points; Calculate the image similarity between the real-time monitoring screen and the cloud segmented screen, and between the real-time monitoring screen and the local segmented screen, and obtain one or more groups of cloud screen similarity values ​​and local screen similarity values; The mean of one or more groups of cloud picture similarity values ​​and the mean of the local picture similarity values ​​are calculated respectively to obtain the cloud picture similarity value and the local picture similarity value.

5. The method for improving the monitoring image resolution based on a high-definition camera as claimed in claim 4, characterized in that: The calculating of the image similarity between the real-time monitoring screen and the cloud segmented screen, and the real-time monitoring screen and the local segmented screen, to obtain one or more groups of cloud screen similarity values ​​and local screen similarity values, includes: Generate feature matrices of the real-time monitoring screen, cloud segmentation screen, and local segmentation screen, and obtain the real-time feature matrix, cloud segmentation feature matrix, and local segmentation feature matrix, respectively. The structure of the feature matrix is: M i =C*N Among them, M i represents a real-time feature matrix, a cloud segmentation feature matrix, or a local segmentation feature matrix, C and N represent the row dimension and column dimension of the feature matrix respectively, when i is equal to 1, it is a real-time feature matrix, when i is equal to 2, it is a cloud segmentation feature matrix, when i is equal to 3, it is a local segmentation feature matrix, and the row dimension is equal to the number of channels of the real-time monitoring screen, the cloud segmentation screen, or the local segmentation screen, and the column dimension is equal to the product of the height and width of the real-time monitoring screen, the cloud segmentation screen, or the local segmentation screen; Generate transposed matrices of the real-time feature matrix, the cloud segmentation feature matrix, and the local segmentation feature matrix respectively, and obtain a real-time transposed matrix, a cloud segmentation transposed matrix, and a local segmentation transposed matrix; According to the corresponding relationship between the real-time feature matrix and the real-time transposed matrix, the cloud segmentation feature matrix and the cloud segmentation transposed matrix, and the local segmentation feature matrix and the local segmentation transposed matrix, a real-time integration matrix, a cloud segmentation integration matrix, and a local segmentation integration matrix are generated respectively; The similarity between the real-time integrated matrix and the cloud segmented integrated matrix, and the similarity between the real-time integrated matrix and the local segmented integrated matrix are calculated to obtain the cloud screen similarity value and the local screen similarity value.

6. The method for improving the monitoring image resolution based on a high-definition camera as claimed in claim 5, characterized in that: The structures of the real-time integration matrix, cloud segmentation integration matrix and local segmentation integration matrix are as follows: Among them, U i represents the real-time integration matrix, cloud segmentation integration matrix, and local segmentation integration matrix. When i is equal to 1, it is the real-time integration matrix; when i is equal to 2, it is the cloud segmentation integration matrix; when i is equal to 3, it is the local segmentation integration matrix. ij Indicates U i The total eigenvalues ​​of the i-th row and j-th column, and the maximum values ​​of i and j are both C, and u ij The calculation method is: Among them, m i represents the number of all features in the i-th row of the real-time feature matrix, cloud segmentation feature matrix, or local segmentation feature matrix, m j represents the number of all features in the jth row of the real-time feature matrix, cloud segmentation feature matrix, or local segmentation feature matrix, m i *m j It means that all the feature numbers in the i-th row are multiplied and added together with all the feature numbers in the j-th row. exp is an exponential function.

7. The method for improving the monitoring picture resolution based on a high-definition camera as claimed in claim 6, characterized in that: The calculating the similarity between the real-time integrated matrix and the cloud segmented integrated matrix, and the real-time integrated matrix and the local segmented integrated matrix, to obtain the cloud screen similarity value and the local screen similarity value includes: The cloud image similarity value and the local image similarity value are calculated according to the following formula: Among them, S2 represents the cloud screen similarity value between the real-time integrated matrix and the cloud segmentation integrated matrix, S3 represents the local screen similarity value between the real-time integrated matrix and the local segmentation integrated matrix, U1·U2 represents the corresponding multiplication and addition of the eigenvalues ​​of U1 and U2, and U1·U3 represents the corresponding multiplication and addition of the eigenvalues ​​of U1 and U3. represents the addition of all eigenvalues ​​of the real-time integrated matrix.

8. The method for improving the monitoring picture resolution based on a high-definition camera as claimed in claim 7, characterized in that: The performing of a resolution enhancement operation on the real-time monitoring picture to obtain a high-quality monitoring picture includes: Acquire a real-time feature matrix of the real-time monitoring screen, and generate a search box according to the matrix dimension of the real-time feature matrix, wherein the row dimension of the search box is the same as the row dimension of the real-time feature matrix; Confirm the local monitoring screen image corresponding to the maximum value among the local screen similarity values, and obtain a highly similar local screen; Use the search box to obtain the pixel values ​​of the local area from the highly similar local picture to obtain the local pixel matrix; The resolution of the real-time monitoring picture is improved by using a local pixel matrix to obtain the high-quality monitoring picture, wherein the resolution of the real-time monitoring picture is improved by using a local pixel matrix to obtain the high-quality monitoring picture, including: Confirm the corresponding position of the local pixel matrix in the real-time monitoring screen to obtain the pixel enhancement area; Calculate the difference between the pixel value in the local pixel matrix and the pixel value in the pixel to be promoted area to obtain the pixel difference; When the pixel difference is greater than or equal to a preset threshold, a weighted average is performed on the pixel value in the local pixel matrix and the pixel value in the pixel to be promoted area to obtain a pixel replacement value; The pixel replacement value is used to replace the pixel values ​​in the pixel improvement area whose pixel difference is greater than or equal to the preset threshold value until a high-quality monitoring picture is obtained.

9. The method for improving the monitoring picture resolution based on a high-definition camera as claimed in claim 8, characterized in that: The structure of the search box is: H=C*L Among them, H represents the search box of the real-time monitoring screen, C represents the row dimension of the search box, which is the same as the row dimension of the real-time feature matrix, and L represents the column dimension of the search box, and the calculation method of the column dimension is: Among them, α represents the weight factor of the calculated column dimension, N represents the column dimension of the real-time feature matrix, and [] represents the rounding operation.

10. A monitoring image resolution enhancement system based on a high-definition camera, characterized in that: The system comprises: A storage system access module, used to receive a resolution improvement instruction of a monitoring screen, connect to a high-definition camera according to the resolution improvement instruction, and after successfully connecting to the high-definition camera, receive the real-time monitoring screen of the high-definition camera and the monitoring time point corresponding to the real-time monitoring screen, wherein the monitoring time point includes seconds-minutes-hours, and access the storage system of the high-definition camera for storing the monitoring screen, wherein the storage system includes a cloud storage system and a local storage system; A monitoring screen acquisition module, used to search for historical screens from the cloud storage system and the local storage system using the monitoring time point as a search condition, and obtain one or more cloud monitoring screens and local monitoring screens respectively, wherein the resolution of the cloud monitoring screen is greater than that of the local monitoring screen, but the number of local monitoring screens is greater than or equal to the number of cloud monitoring screens; The image similarity value calculation module is used to calculate the image similarity between the real-time monitoring image and the cloud monitoring image, and between the real-time monitoring image and the local monitoring image, respectively, to obtain the cloud image similarity value and the local image similarity value, and to determine whether the cloud image similarity value is greater than the local image similarity value. If the cloud image similarity value is greater than the local image similarity value, the real-time monitoring image is directly stored in the local storage system, and a feedback instruction is generated that the resolution does not need to be improved; The monitoring picture storage module is used to perform a resolution enhancement operation on the real-time monitoring picture if the cloud picture similarity value is less than or equal to the local picture similarity value to obtain a high-quality monitoring picture, store the high-quality monitoring picture in the cloud storage system, and generate a feedback instruction that the resolution has been enhanced.

Images