A monitoring image generation method and device based on artificial intelligence technology

Abstract

The present invention proposes a monitoring image generation method and device based on artificial intelligence technology, which generates a low-resolution background image of the monitoring area based on the real-time image of the monitoring area captured by a monitoring camera, identifies dynamic elements in the monitoring area in the last frame of real-time image captured by the monitoring camera, generates a low-resolution foreground image of the monitoring area corresponding to the capture moment of the last frame of real-time image according to the position of the dynamic elements in the last frame of real-time image, inputs the low-resolution background image into a pre-trained background image reconstruction model to generate a corresponding super-resolution background image, inputs the low-resolution foreground image into a pre-trained foreground image reconstruction model to generate a corresponding super-resolution foreground image, and uses the super-resolution background image and the super-resolution foreground image to synthesize the monitoring image, which can effectively use artificial intelligence technology to improve the image quality of the monitoring image.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a monitoring image generation method and device based on artificial intelligence technology. Background Art

[0002] Video surveillance is one of the main components of the security system. It uses high-definition cameras to capture real-time images of the monitoring area and display them on the display screen of the monitoring center, so that monitoring personnel far away in the monitoring center can monitor multiple monitoring areas at the same time, without having to arrange manpower for long-term duty in each security monitoring area. At the same time, the stored image data also provides the possibility of image tracing in the security monitoring area. Limited by the hardware parameters of the camera such as resolution, type and size of the photosensitive element, processing power of the monitoring chip, quality and performance of the lens, and also limited by the ambient brightness and the distance of the monitored object, the clarity of the monitored object in the monitoring image often cannot meet the requirements. For example, the clarity of the monitored objects such as faces or license plates in the monitoring screen is low, making it difficult to achieve applications such as searching for people, objects or vehicle tracking through the monitoring screen. The traditional use of image enhancement algorithms to optimize the contrast of monitoring images, sharpen image details and edges, etc., has not significantly improved the clarity of monitoring images. In recent years, with the development of artificial intelligence technology, people have studied the super-resolution reconstruction of images through machine learning, which can greatly improve the clarity of images. However, machine learning models for super-resolution reconstruction of images require a large number of precisely paired high-resolution and low-resolution images as training sample data for training. In the field of surveillance, there are no truly effective and precisely paired high-resolution and low-resolution images, and it is impossible to specifically train super-resolution reconstruction models for surveillance images. Therefore, how to improve the image quality optimization of surveillance images remains a difficult problem. Summary of the invention

[0003] Based on the above problems, the present invention proposes a monitoring image generation method and device based on artificial intelligence technology, which can effectively utilize artificial intelligence technology to improve the image quality of monitoring images.

[0004] In view of this, a first aspect of the present invention proposes a monitoring image generation method based on artificial intelligence technology, comprising:

[0005] Obtain each frame of real-time image of the monitoring area captured by the surveillance camera;

[0006] Generate a low-resolution background image of the monitoring area based on the real-time image, wherein the low-resolution background image is an image composed of images of static elements located in the monitoring area;

[0007] Identifying dynamic elements within the monitoring area in the last frame of real-time image captured by the monitoring camera;

[0008] Generate a low-resolution foreground image of the monitoring area corresponding to the capture moment of the last frame of real-time image according to the position of the dynamic element in the last frame of real-time image, wherein the number of the foreground images corresponds to the number of dynamic elements in the last frame of real-time image;

[0009] Inputting the low-resolution background image into a pre-trained background image reconstruction model to generate a corresponding super-resolution background image;

[0010] Inputting the low-resolution foreground image into a pre-trained foreground image reconstruction model to generate a corresponding super-resolution foreground image;

[0011] A monitoring image is synthesized using the super-resolution background image and the super-resolution foreground image.

[0012] Furthermore, the step of generating a low-resolution background image of the monitoring area based on the real-time image specifically includes:

[0013] configuring a background image generation cycle for generating the low-resolution background image;

[0014] At the beginning of each background image generation cycle, extract at least one frame of original background image from the previous background image generation cycle;

[0015] The low-resolution background image is generated based on the original background image.

[0016] Furthermore, at the beginning of each background image generation cycle, the step of extracting at least one frame of original background image from the previous background image generation cycle specifically includes:

[0017] Extracting the last frame of real-time image in the previous background image generation cycle as the basic image in the original background image;

[0018] Determining whether there is a dynamic element in the basic image;

[0019] When there is no dynamic element in the basic image, the step of generating the low-resolution background image based on the original background image specifically includes determining the basic image as the low-resolution background image.

[0020] Furthermore, after the step of determining whether there is a dynamic element in the basic image, the method further includes:

[0021] When there are dynamic elements in the basic image, generating an identifier of each dynamic element on the basic image;

[0022] Traverse each dynamic element on the base image and perform the following steps:

[0023] Determine the currently traversed dynamic element as the target dynamic element;

[0024] Determining a target image area covered by the target dynamic element on the basic image;

[0025] Analyze the real-time image in the previous background image generation cycle frame by frame from back to front, so as to determine the first frame of the real-time image in which the target image area does not have a dynamic element as the original background image corresponding to the target dynamic element;

[0026] The original background image corresponding to the target dynamic element is associated with the identifier of the target dynamic element and stored.

[0027] Furthermore, the step of analyzing the real-time image in the previous background image generation cycle frame by frame from back to front to determine the first frame of the real-time image in which the target image area does not have a dynamic element as the original background image corresponding to the target dynamic element specifically includes:

[0028] Arrange the other real-time images except the basic image captured by the monitoring camera in the previous background image generation cycle in reverse order in time sequence to construct a reverse order real-time image queue;

[0029] Traverse each frame of real-time image in the reverse real-time image queue to perform the following steps:

[0030] Determine a frame of real-time image currently traversed as the target real-time image;

[0031] Identifying dynamic elements on the target real-time image;

[0032] Determine whether there is any dynamic element on the target real-time image that has an overlapping area with the target image area;

[0033] When there is no dynamic element on the target real-time image that has an overlapping area with the target image area, determining the target real-time image as the original background image corresponding to the target dynamic element;

[0034] End the traversal of the reverse order real-time image queue.

[0035] Furthermore, the step of generating the low-resolution background image based on the original background image specifically includes:

[0036] Filling the image area covered by each dynamic element on the basic image with a grayscale mask to generate a grayscale mask area corresponding to each dynamic element on the basic image;

[0037] Determining the original background image corresponding to the grayscale mask area according to the corresponding relationship between each dynamic element and the original background image;

[0038] The image content corresponding to the grayscale mask area is extracted from the original background image and filled into the grayscale mask area on the basic image.

[0039] Furthermore, before the step of inputting the low-resolution background image into a pre-trained background image reconstruction model to generate a corresponding super-resolution background image, the method further includes:

[0040] Obtain a pre-prepared high-resolution image dataset containing high-resolution environment images;

[0041] Inputting the high-resolution image dataset into an image degradation model to generate a low-resolution environment image corresponding to each high-resolution environment image in the high-resolution image dataset;

[0042] Constructing a first paired training dataset from high-resolution environment images and corresponding low-resolution environment images in the high-resolution image dataset;

[0043] Constructing a first super-resolution image reconstruction network, wherein the first super-resolution image reconstruction network is a generative adversarial network;

[0044] The first paired training data set is input into the first super-resolution image reconstruction network to train the background image reconstruction model.

[0045] Furthermore, before the step of inputting the low-resolution foreground image into a pre-trained foreground image reconstruction model to generate a corresponding super-resolution foreground image, the method further includes:

[0046] Acquire a pre-prepared low-resolution image data set including the low-resolution foreground image;

[0047] Inputting the low-resolution image dataset into an image enhancement model to generate a high-resolution foreground image corresponding to each low-resolution foreground image in the low-resolution image dataset;

[0048] Constructing a second paired training data set from low-resolution foreground images and corresponding high-resolution foreground images in the low-resolution image data set;

[0049] Constructing a second super-resolution image reconstruction network, wherein the second super-resolution image reconstruction network is a generative adversarial network;

[0050] The second paired training data set is used to input into the second super-resolution image reconstruction network to train the foreground image reconstruction model.

[0051] Furthermore, the step of generating a low-resolution foreground image of the monitoring area corresponding to the capture moment of the last frame of real-time image according to the position of the dynamic element in the last frame of real-time image specifically includes:

[0052] Extracting a local image of each dynamic element from the last frame of real-time image;

[0053] Obtaining the image size of the low-resolution background image;

[0054] The local image is expanded to the same image size as the low-resolution background image to generate the low-resolution foreground image.

[0055] The second aspect of the present invention proposes a monitoring image generation device based on artificial intelligence technology, including a memory and a processor, wherein the processor executes a computer program stored in the memory to implement the monitoring image generation method based on artificial intelligence technology as described in any one of the first aspect of the present invention.

[0056] The present invention proposes a monitoring image generation method and device based on artificial intelligence technology, which generates a low-resolution background image of the monitoring area based on the real-time image of the monitoring area captured by a monitoring camera, identifies dynamic elements in the monitoring area in the last frame of real-time image captured by the monitoring camera, generates a low-resolution foreground image of the monitoring area corresponding to the capture moment of the last frame of real-time image according to the position of the dynamic elements in the last frame of real-time image, inputs the low-resolution background image into a pre-trained background image reconstruction model to generate a corresponding super-resolution background image, inputs the low-resolution foreground image into a pre-trained foreground image reconstruction model to generate a corresponding super-resolution foreground image, and uses the super-resolution background image and the super-resolution foreground image to synthesize the monitoring image, which can effectively use artificial intelligence technology to improve the image quality of the monitoring image. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 is a flow chart of a monitoring image generation method based on artificial intelligence technology provided by an embodiment of the present invention;

[0058] Figure 2 It is a schematic diagram of a reverse real-time image queue provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0060] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0061] In the description of the present invention, the term "multiple" refers to two or more. Unless otherwise clearly defined, the orientation or positional relationship indicated by the terms "upper", "lower", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation of the present invention. The terms "connection", "installation", "fixation", etc. should be understood in a broad sense. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be directly connected or indirectly connected through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances. In addition, the terms "first", "second", etc. are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first", "second", etc. can explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, the meaning of "multiple" is two or more.

[0062] In the description of this specification, the description of the terms "one embodiment", "some implementations", "specific embodiments", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0063] A monitoring image generation method and device based on artificial intelligence technology provided according to some embodiments of the present invention will be described below with reference to the accompanying drawings.

[0064] like Figure 1 As shown, the first aspect of the present invention proposes a monitoring image generation method based on artificial intelligence technology, comprising:

[0065] Obtain each frame of real-time image of the monitoring area captured by the surveillance camera;

[0066] Generate a low-resolution background image of the monitoring area based on the real-time image, wherein the low-resolution background image is an image composed of images of static elements located in the monitoring area;

[0067] Identifying dynamic elements within the monitoring area in the last frame of real-time image captured by the monitoring camera;

[0068] Generate a low-resolution foreground image of the monitoring area corresponding to the capture moment of the last frame of real-time image according to the position of the dynamic element in the last frame of real-time image, wherein the number of the foreground images corresponds to the number of dynamic elements in the last frame of real-time image;

[0069] Inputting the low-resolution background image into a pre-trained background image reconstruction model to generate a corresponding super-resolution background image;

[0070] Inputting the low-resolution foreground image into a pre-trained foreground image reconstruction model to generate a corresponding super-resolution foreground image;

[0071] A monitoring image is synthesized using the super-resolution background image and the super-resolution foreground image.

[0072] The monitoring area refers to the field of view of the monitoring camera, that is, the area covered by the monitoring screen of the monitoring camera. Unless otherwise specified, the monitoring area referred to in the present invention refers to the monitoring area of ​​a monitoring camera, that is, each monitoring camera corresponds to a monitoring area, and similarly, one monitoring area corresponds to a low-resolution background image. The static elements are objects that do not move autonomously within the monitoring area, including the ground and buildings, decorations and plants fixed or planted on the ground, and cables, pipes, etc. fixed on buildings such as walls.

[0073] The step of identifying the dynamic elements in the monitoring area in the last frame of real-time image captured by the monitoring camera specifically includes:

[0074] Configure dynamic element recognition step size ;

[0075] The surveillance camera captures each Frame real-time image, that is, the image captured by the surveillance camera The dynamic element recognition is performed on the last frame of the live image.

[0076] The dynamic element recognition step The dynamic element recognition step length can be configured according to actual implementation needs. The size of is positively correlated with the hardware parameters of the monitoring image generating device, including processing capability, cache space, etc.

[0077] The step of generating a foreground image of the monitored area corresponding to the moment of capturing the last frame of real-time image according to the position of the dynamic element in the last frame of real-time image is specifically to generate a foreground image corresponding to each dynamic element, that is, each dynamic element corresponds to one or more foreground images, and at the moment of capturing the last frame of real-time image, each dynamic element corresponds to a foreground image.

[0078] Furthermore, the step of generating a low-resolution background image of the monitoring area based on the real-time image specifically includes:

[0079] configuring a background image generation cycle for generating the low-resolution background image;

[0080] At the beginning of each background image generation cycle, extract at least one frame of original background image from the previous background image generation cycle;

[0081] The low-resolution background image is generated based on the original background image.

[0082] Although the static elements in the monitoring area will not move autonomously, they are not completely unchanged. For example, due to the influence of human factors or natural factors such as wind or rain, including the growth of plants, the background image in the monitoring area may change, but its change speed is much slower than the change of the low-resolution foreground image composed of the dynamic elements in the monitoring area. Therefore, it is necessary to configure a background image generation cycle to regenerate the background image of the monitoring area in each background image generation cycle.

[0083] The background image generation cycle is a time period of a specific length, and adjacent background image generation cycles are connected end to end to form a continuous timeline, so the start time of the background image generation cycle is the end time of the previous background image generation cycle. The original background image is one or more frames of real-time images of the monitoring area captured by the monitoring camera in the previous background image generation cycle.

[0084] Furthermore, at the beginning of each background image generation cycle, the step of extracting at least one frame of original background image from the previous background image generation cycle specifically includes:

[0085] Extracting the last frame of real-time image in the previous background image generation cycle as the basic image in the original background image;

[0086] Determining whether there is a dynamic element in the basic image;

[0087] When there is no dynamic element in the basic image, the step of generating the low-resolution background image based on the original background image specifically includes determining the basic image as the low-resolution background image.

[0088] In the technical solution of the above-mentioned embodiment, when there are no dynamic elements in the basic image, the basic image itself is a complete low-resolution background image, and there is no need to continue to extract other original background images from the previous background image generation cycle to generate the low-resolution background image. The basic image is used as the only original background image to execute the step of generating the low-resolution background image based on the original background image.

[0089] Furthermore, after the step of determining whether there is a dynamic element in the basic image, the method further includes:

[0090] When there are dynamic elements in the basic image, generating an identifier of each dynamic element on the basic image;

[0091] Traverse each dynamic element on the base image and perform the following steps:

[0092] Determine the currently traversed dynamic element as the target dynamic element;

[0093] Determining a target image area covered by the target dynamic element on the basic image;

[0094] Analyze the real-time image in the previous background image generation cycle frame by frame from back to front, so as to determine the first frame of the real-time image in which the target image area does not have a dynamic element as the original background image corresponding to the target dynamic element;

[0095] The original background image corresponding to the target dynamic element is associated with the identifier of the target dynamic element and stored.

[0096] The identifier of the dynamic element is a unique identifier of the dynamic element on the basic image. The corresponding dynamic element on the basic image can be uniquely identified by the identifier. The identifier of the dynamic element can be a digital number or a character code, etc.

[0097] Each frame of real-time image captured by the monitoring camera in the last background image generation cycle has the same image size and resolution. The target image area covered by the target dynamic element determined on the base image mentioned in the above embodiment can determine a corresponding image area in any other frame of real-time image except the base image captured by the monitoring camera in the last background image generation cycle, that is, the image area with the same position coordinates of the edge line on the image. The target image area mentioned in the step of analyzing the real-time images in the last background image generation cycle frame by frame from back to front to determine the first frame of real-time image in which there is no dynamic element in the target image area as the original background image corresponding to the target dynamic element refers to the image area in each frame of real-time image that corresponds to the target image area covered by the target element on the base image.

[0098] The original background image corresponding to the target dynamic element is associated with the identifier of the target dynamic element and stored, and an association relationship between the target dynamic element, the target image area covered by the target dynamic element and the original background image corresponding to the target dynamic element is established through the identifier of the target dynamic element.

[0099] Furthermore, the step of analyzing the real-time image in the previous background image generation cycle frame by frame from back to front to determine the first frame of the real-time image in which the target image area does not have a dynamic element as the original background image corresponding to the target dynamic element specifically includes:

[0100] Arrange the other real-time images except the basic image captured by the monitoring camera in the previous background image generation cycle in reverse order in time sequence to construct a reverse order real-time image queue;

[0101] Traverse each frame of real-time image in the reverse real-time image queue to perform the following steps:

[0102] Determine a frame of real-time image currently traversed as the target real-time image;

[0103] Identifying dynamic elements on the target real-time image;

[0104] Determine whether there is any dynamic element on the target real-time image that has an overlapping area with the target image area;

[0105] When there is no dynamic element on the target real-time image that has an overlapping area with the target image area, determining the target real-time image as the original background image corresponding to the target dynamic element;

[0106] End the traversal of the reverse order real-time image queue.

[0107] See below Figure 2 The steps of arranging the other real-time images except the basic image captured by the monitoring camera in the previous background image generation cycle in reverse chronological order to construct a reverse real-time image queue are described as follows.

[0108] Taking each background image generation cycle as an example, each frame of real-time image in the previous background image generation cycle is numbered as f1, f2, f3, ..., f n-1 , f n , then the step of extracting the last frame of real-time image in the previous background image generation cycle as the basic image in the original background image is specifically to extract the last frame of real-time image numbered f in the previous background image generation cycle n The real-time image of is determined as the basic image in the original background image. The number of real-time images in the reverse real-time image queue is one frame less than the number of real-time images in the previous background image generation cycle, that is, when the number of real-time images in the previous background image generation cycle is n, the number of real-time images in the reverse real-time image queue is n-1, such as Figure 2 As shown in FIG. 1 , the real-time image in the reverse real-time image queue is numbered from f n-1 The real-time images are arranged in reverse chronological order starting from the real-time image numbered f1.

[0109] Furthermore, after the step of determining whether there is any dynamic element on the target real-time image that has an overlapping area with the target image area, the method further includes:

[0110] When there are dynamic elements on the target real-time image that overlap with the target image area, the reverse real-time image queue is traversed until a frame of real-time image is found that satisfies the condition that there is no dynamic element that overlaps with the target image area.

[0111] Furthermore, after traversing all the real-time images in the reverse real-time image queue, if no real-time image is found to satisfy the condition that there is no overlapping area between any dynamic element and the target image area, it is determined that the generation of the low-resolution background image of the current background image generation cycle has failed, the generation of the low-resolution background image of the current background image generation cycle is abandoned, the last generated low-resolution background image is used to execute subsequent processing steps, and wait for the next background image generation cycle to re-execute the step of generating a low-resolution background image of the monitored area based on the real-time image.

[0112] Furthermore, the step of generating the low-resolution background image based on the original background image specifically includes:

[0113] Filling the image area covered by each dynamic element on the basic image with a grayscale mask to generate a grayscale mask area corresponding to each dynamic element on the basic image;

[0114] Determining the original background image corresponding to the grayscale mask area according to the corresponding relationship between each dynamic element and the original background image;

[0115] The image content corresponding to the grayscale mask area is extracted from the original background image and filled into the grayscale mask area on the basic image.

[0116] Similarly, each grayscale mask area is associated with the dynamic element through the identifier of the corresponding dynamic element, and is also associated with the image area covered by the dynamic element and the original background image corresponding to the dynamic element. Therefore, it can be determined that each grayscale mask area has a one-to-one correspondence with each original background image, so that the image content at the same position as the grayscale mask area can be extracted from the corresponding original background image and filled into the basic image to complete the background image content at the position occupied by the dynamic element on the basic image.

[0117] Furthermore, before the step of inputting the low-resolution background image into a pre-trained background image reconstruction model to generate a corresponding super-resolution background image, the method further includes:

[0118] Obtain a pre-prepared high-resolution image dataset containing high-resolution environment images;

[0119] Inputting the high-resolution image dataset into an image degradation model to generate a low-resolution environment image corresponding to each high-resolution environment image in the high-resolution image dataset;

[0120] Constructing a first paired training dataset from high-resolution environment images and corresponding low-resolution environment images in the high-resolution image dataset;

[0121] Constructing a first super-resolution image reconstruction network, wherein the first super-resolution image reconstruction network is a generative adversarial network;

[0122] The first paired training data set is input into the first super-resolution image reconstruction network to train the background image reconstruction model.

[0123] The high-resolution image dataset is a sample dataset consisting of a large number of high-resolution environment images that are pre-photographed or acquired through other means, including the ground and buildings, decorations and plants fixed or planted on the ground, and static elements such as cables and pipes fixed on the walls of buildings. The low-resolution environment image has the same size and resolution as the low-resolution background image.

[0124] The image degradation model can be any image degradation model such as the trained IKC (Iterative kernel correction) model, DRL-DASR (Degradation Representation Learning-Domain distanceAware Super-Resolution) model, etc. The image degradation model generates a corresponding low-quality image by simulating the image degradation process during the generation, processing, and storage of the image, such as noise pollution, compression coding, resolution reduction, etc.

[0125] In the technical solution of the above-mentioned embodiment, the first super-resolution image reconstruction network is a generative adversarial network, and the first super-resolution image reconstruction network includes a generator and a discriminator, the generator is used to generate a super-resolution image, and the discriminator is used to judge the authenticity of the generated super-resolution image. Specifically, in the step of using the first paired training data set to input into the first super-resolution image reconstruction network to train the background image reconstruction model, the first paired training data set is input into the first super-resolution image reconstruction network for training, the generator generates a super-resolution environment image based on the low-resolution environment image in the first paired training data set, the generated super-resolution environment image and the high-resolution environment image in the first paired training data set are input into the discriminator to judge whether the generated super-resolution environment image is "real", and the two are iteratively trained to continuously improve the super-resolution image feature reconstruction capability of the generator.

[0126] Furthermore, before the step of inputting the low-resolution foreground image into a pre-trained foreground image reconstruction model to generate a corresponding super-resolution foreground image, the method further includes:

[0127] Acquire a pre-prepared low-resolution image data set including the low-resolution foreground image;

[0128] Inputting the low-resolution image dataset into an image enhancement model to generate a high-resolution foreground image corresponding to each low-resolution foreground image in the low-resolution image dataset;

[0129] Constructing a second paired training data set from low-resolution foreground images and corresponding high-resolution foreground images in the low-resolution image data set;

[0130] Constructing a second super-resolution image reconstruction network, wherein the second super-resolution image reconstruction network is a generative adversarial network;

[0131] The second paired training data set is used to input into the second super-resolution image reconstruction network to train the foreground image reconstruction model.

[0132] The image enhancement model may be any image enhancement model such as a trained RealSR (Real-World Super-Resolution) model, a Real-ESRGAN (Enhanced Super-Resolution Generative Adversarial Network) model, a BSRGAN (Blind Super-Resolution Generative Adversarial Network) model, or the like.

[0133] In the technical solution of the above-mentioned embodiment, the second super-resolution image reconstruction network is a generative adversarial network, and the second super-resolution image reconstruction network includes a generator and a discriminator, the generator is used to generate a super-resolution image, and the discriminator is used to judge the authenticity of the generated super-resolution image. Specifically, in the step of using the second paired training data set to input into the second super-resolution image reconstruction network to train the foreground image reconstruction model, the second paired training data set is input into the second super-resolution image reconstruction network for training, the generator generates a super-resolution foreground image based on the low-resolution foreground image in the second paired training data set, the generated super-resolution foreground image and the high-resolution foreground image in the second paired training data set are input into the discriminator to judge whether the generated super-resolution foreground image is "real", and the two are iteratively trained with each other to continuously improve the super-resolution image feature reconstruction capability of the generator.

[0134] Furthermore, the step of generating a low-resolution foreground image of the monitoring area corresponding to the capture moment of the last frame of real-time image according to the position of the dynamic element in the last frame of real-time image specifically includes:

[0135] Extracting a local image of each dynamic element from the last frame of real-time image;

[0136] Obtaining the image size of the low-resolution background image;

[0137] The local image is expanded to the same image size as the low-resolution background image to generate the low-resolution foreground image.

[0138] In the step of extracting a partial image of each dynamic element from the last frame of real-time image, the partial image has the size and resolution of the dynamic element in the last frame of real-time image. The partial image, the low-resolution foreground image and the low-resolution background image have the same resolution, which is the original resolution of the real-time image of the monitoring area captured by the monitoring camera.

[0139] The step of expanding the partial image to the same image size as the low-resolution background image to generate the low-resolution foreground image specifically includes filling monochrome pixels around the partial image to expand the partial image to the same image size as the low-resolution background image, wherein the monochrome pixels may be white pixels, black pixels, or gray pixels of any other restoration value. In the process of filling monochrome pixels around the partial image, the size and resolution of the partial image are kept unchanged.

[0140] The second aspect of the present invention proposes a monitoring image generation device based on artificial intelligence technology, including a memory and a processor, wherein the processor executes a computer program stored in the memory to implement the monitoring image generation method based on artificial intelligence technology as described in any one of the first aspect of the present invention.

[0141] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0142] According to the embodiments of the present invention as described above, these embodiments do not describe all the details in detail, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and changes can be made based on the above description. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can make good use of the present invention and the modified use based on the present invention. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A monitoring image generation method based on artificial intelligence technology, characterized in that: include: Obtain each frame of real-time image of the monitoring area captured by the surveillance camera; Generate a low-resolution background image of the monitoring area based on the real-time image, wherein the low-resolution background image is an image composed of images of static elements located in the monitoring area; Identifying dynamic elements within the monitoring area in the last frame of real-time image captured by the monitoring camera; Extracting a local image of each dynamic element from the last frame of real-time image; Obtaining the image size of the low-resolution background image; Expanding the partial image to the same image size as the low-resolution background image to generate a low-resolution foreground image of the monitoring area corresponding to the capture moment of the last frame of real-time image, wherein the number of the partial images corresponds to the number of dynamic elements in the last frame of real-time image; Inputting the low-resolution background image into a pre-trained background image reconstruction model to generate a corresponding super-resolution background image; Inputting the low-resolution foreground image into a pre-trained foreground image reconstruction model to generate a corresponding super-resolution foreground image; synthesizing a monitoring image using a super-resolution background image and the super-resolution foreground image; The step of generating a low-resolution background image of the monitoring area based on the real-time image specifically includes: configuring a background image generation cycle for generating the low-resolution background image; At the beginning of each background image generation cycle, extract at least one frame of original background image from the previous background image generation cycle; Generate the low-resolution background image based on the original background image; The step of extracting at least one frame of original background image from the previous background image generation cycle at the beginning of each background image generation cycle specifically includes: Extract the last frame of real-time image of the previous background image generation cycle as the basic image in the original background image. When there are no dynamic elements in the basic image, the basic image itself is a complete low-resolution background image, and it is no longer necessary to continue to extract other original background images from the previous background image generation cycle to generate the low-resolution background image. The basic image is used as the only original background image to perform the step of generating the low-resolution background image based on the original background image; Determining whether there is a dynamic element in the basic image; When there is no dynamic element in the basic image, the step of generating the low-resolution background image based on the original background image specifically comprises determining the basic image as the low-resolution background image; After the step of determining whether there is a dynamic element in the basic image, the method further includes: When there are dynamic elements in the basic image, generating an identifier of each dynamic element on the basic image; Traverse each dynamic element on the base image and perform the following steps: Determine the currently traversed dynamic element as the target dynamic element; Determining a target image area covered by the target dynamic element on the basic image; Analyze the real-time image in the previous background image generation cycle frame by frame from back to front, so as to determine the first frame of the real-time image in which the target image area does not have a dynamic element as the original background image corresponding to the target dynamic element; The original background image corresponding to the target dynamic element is associated with the identifier of the target dynamic element and stored.

2. The monitoring image generation method based on artificial intelligence technology according to claim 1 is characterized in that: The step of analyzing the real-time image in the previous background image generation cycle frame by frame from back to front to determine the first frame of the real-time image in which the target image area does not have a dynamic element as the original background image corresponding to the target dynamic element specifically includes: Arrange the other real-time images except the basic image captured by the monitoring camera in the last background image generation cycle in reverse order in time sequence to construct a reverse order real-time image queue; Traverse each frame of real-time image in the reverse real-time image queue to perform the following steps: Determine a frame of real-time image currently traversed as the target real-time image; Identifying dynamic elements on the target real-time image; Determine whether there is any dynamic element on the target real-time image that has an overlapping area with the target image area; When there is no dynamic element on the target real-time image that has an overlapping area with the target image area, determining the target real-time image as the original background image corresponding to the target dynamic element; End the traversal of the reverse order real-time image queue.

3. The monitoring image generation method based on artificial intelligence technology according to claim 1 is characterized in that: The step of generating the low-resolution background image based on the original background image specifically includes: Filling the image area covered by each dynamic element on the basic image with a grayscale mask to generate a grayscale mask area corresponding to each dynamic element on the basic image; Determining the original background image corresponding to the grayscale mask area according to the corresponding relationship between each dynamic element and the original background image; The image content corresponding to the grayscale mask area is extracted from the original background image and filled into the grayscale mask area on the basic image.

4. The monitoring image generation method based on artificial intelligence technology according to claim 1 is characterized in that: Before the step of inputting the low-resolution background image into a pre-trained background image reconstruction model to generate a corresponding super-resolution background image, the method further includes: Obtain a pre-prepared high-resolution image dataset containing high-resolution environment images; Inputting the high-resolution image dataset into an image degradation model to generate a low-resolution environment image corresponding to each high-resolution environment image in the high-resolution image dataset; Constructing a first paired training dataset from high-resolution environment images and corresponding low-resolution environment images in the high-resolution image dataset; Constructing a first super-resolution image reconstruction network, wherein the first super-resolution image reconstruction network is a generative adversarial network; The first paired training data set is input into the first super-resolution image reconstruction network to train the background image reconstruction model.

5. The monitoring image generation method based on artificial intelligence technology according to claim 1 is characterized in that: Before the step of inputting the low-resolution foreground image into a pre-trained foreground image reconstruction model to generate a corresponding super-resolution foreground image, the method further includes: Acquire a pre-prepared low-resolution image data set including the low-resolution foreground image; Inputting the low-resolution image dataset into an image enhancement model to generate a high-resolution foreground image corresponding to each low-resolution foreground image in the low-resolution image dataset; constructing a second paired training data set from low-resolution foreground images and corresponding high-resolution foreground images in the low-resolution image data set; Constructing a second super-resolution image reconstruction network, wherein the second super-resolution image reconstruction network is a generative adversarial network; The second paired training data set is used to input into the second super-resolution image reconstruction network to train the foreground image reconstruction model.

6. A monitoring image generation device based on artificial intelligence technology, characterized in that: It includes a memory and a processor, and the processor executes a computer program stored in the memory to implement the monitoring image generation method based on artificial intelligence technology as described in any one of claims 1 to 5.

Images