Method, device and equipment for removing rain and snow in image, medium and product

Abstract

The invention discloses a method, a device and equipment for removing rain and snow in an image, a medium and a product. The method for removing rain and snow in the image comprises the following steps: acquiring an image frame to be compared in railway monitoring video data; determining a reconstruction pixel value of each pixel point according to a pixel minimum value of multiple frames of images of the to-be-compared image frame at the same pixel point; and reconstructing the rain and snow filtered image according to the reconstructed pixel value of each pixel point. According to the technical scheme of the embodiment of the invention, rain and snow interference in the image can be quickly and well filtered out, the calculated amount is small, and a large number of paired rain and snow images in an actual scene are not needed to serve as training data.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method, device, equipment, medium and product for removing rain and snow in an image. Background Art

[0002] The quality of collected pictures is often poor due to weather conditions such as rain and snow. With the development of digital image processing technology, removing the influence of rain and snow in images has become an important research direction in image processing.

[0003] Existing rain and snow removal methods are divided into algorithms based on traditional mathematical models and algorithms based on deep learning. Algorithms based on traditional mathematical models mainly include indirect precipitation models and statistical filtering methods. Algorithms based on deep learning mainly include GAN (Generative Adversarial Networks) algorithms and CNN (Convolutional Neural Networks) algorithms.

[0004] The indirect precipitation model is a cloud projection model that treats raindrops as free-falling bodies in the air, colliding with the ground and splashing light from a certain intensity of light source. The indirect precipitation model simplifies the impact of raindrops to a droplet pixel generated for each element in the cloud layer, and then uses a filter to separate the cloud layer and eliminate the droplet pixels. Finally, the reflected light is added to the filtered result to obtain an image without precipitation. The statistical filtering method treats raindrops as disks on the project and uses differential equations to describe the speed and position of the disk to achieve the effect of removing rain and snow. This method can better simulate the physical process of real raindrops, but it also has disadvantages such as large amount of calculation and long calculation time.

[0005] CNN is one of the most widely used deep learning algorithms. It uses local filters in the convolution layer to extract features from the image, and finally maps the features to the output layer through the fully connected layer for classification or regression. In terms of rain and snow removal, researchers use different network results and add modules such as convolution layers and regularization layers between the input layer and the output layer for rain and snow removal training. GAN is another solution based on deep learning. It is trained through two neural networks. The Generator is responsible for generating samples, and the Discriminator is responsible for distinguishing the generated samples from the real samples. In terms of rain and snow removal, real rain and snow images and images without precipitation are input into the network, and the generative model is trained to generate output images, and the generative model is optimized in the back-propagation algorithm.

[0006] Although traditional precipitation models, statistical filtering models, and differential equation models have certain use value, their algorithm accuracy and efficiency need to be optimized. Among deep learning algorithms, CNN, GAN and other methods have achieved good results in removing rain and snow, and their training accuracy is constantly improving. However, their computational complexity is large, and since it is impossible to obtain real paired rain and snow image training samples, the generalization of deep learning-based algorithms is poor. Summary of the invention

[0007] The present invention provides a method, device, equipment, medium and product for removing rain and snow from images to solve the problem that the current method of removing rain and snow from images has large amount of calculation, difficult modeling and limited training samples, resulting in poor performance of the algorithm for removing rain and snow from images.

[0008] According to one aspect of the present invention, a method for removing rain and snow in an image is provided, comprising:

[0009] Obtaining image frames to be compared in railway monitoring video data;

[0010] Determine the reconstructed pixel value of each pixel point according to the minimum pixel value of the multiple frames of the image frame to be compared at the same pixel point;

[0011] The rain and snow filtered image is reconstructed according to the reconstructed pixel value of each pixel.

[0012] According to another aspect of the present invention, a device for removing rain and snow in an image is provided, comprising:

[0013] An image frame acquisition module is used to acquire image frames to be compared in railway monitoring video data;

[0014] A reconstructed pixel value determination module is used to determine the reconstructed pixel value of each pixel point according to the minimum pixel value of multiple frames of the image frame to be compared at the same pixel point;

[0015] The image reconstruction module is used to reconstruct the rain and snow filtered image according to the reconstructed pixel value of each pixel point.

[0016] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0017] at least one processor; and

[0018] a memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the method for removing rain and snow from an image described in any embodiment of the present invention.

[0020] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for removing rain and snow in an image as described in any embodiment of the present invention when executed.

[0021] According to another aspect of the present invention, a computer program product is provided, characterized in that the computer program product comprises a computer program, and when the computer program is executed by a processor, the method for removing rain and snow in an image described in any embodiment of the present invention is implemented.

[0022] The technical solution of the embodiment of the present invention obtains the image frame to be compared in the railway monitoring video data, thereby determining the reconstructed pixel value of each pixel point according to the minimum pixel value of multiple frames of the image frame to be compared at the same pixel point, and then reconstructs the rain and snow filtered image according to the reconstructed pixel value of each pixel point. In this solution, there is no need to perform difficult modeling of rain and snow, and in the case of not needing to obtain rain and snow pictures of actual scenes, it is still possible to obtain a good rain and snow removal effect in various different scenes, and the calculation method is simple and does not require a large amount of calculation. It solves the problem of poor performance of the rain and snow removal algorithm in the image caused by the large amount of calculation, difficult modeling and limited training samples in the current method of removing rain and snow from the image, and can quickly and well filter out the rain and snow interference in the image, with a small amount of calculation and without requiring a large number of paired rain and snow images in actual scenes as training data.

[0023] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 A flowchart of a method for removing rain and snow from an image provided by Embodiment 1 of the present invention;

[0026] Figure 2 A comparison diagram of an image before and after rain and snow filtering provided in the first embodiment of the present invention;

[0027] Figure 3 A flowchart of a method for removing rain and snow from an image provided by Embodiment 2 of the present invention;

[0028] Figure 4 A schematic diagram of the structure of a device for removing rain and snow from an image provided by Embodiment 3 of the present invention;

[0029] Figure 5 A schematic diagram of the structure of an electronic device that can be used to implement an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] Embodiment 1

[0033] Figure 1 This is a flow chart of a method for removing rain and snow from an image provided by the first embodiment of the present invention. This embodiment is applicable to the case of filtering out rain and snow interference in an image. The method can be performed by a device for removing rain and snow from an image. The device for removing rain and snow from an image can be implemented in the form of hardware and / or software. The device for removing rain and snow from an image can be configured in an electronic device. Figure 1 As shown, the method includes:

[0034] Step 110: Obtain image frames to be compared in railway monitoring video data.

[0035] The railway monitoring video data may be monitoring videos collected by cameras arranged along the railway, and the image frames to be compared may be continuous image frames in the railway monitoring video data.

[0036] In an embodiment of the present invention, railway monitoring video data can be collected based on cameras arranged along the railway, and based on a preset sampling frequency, the image frames to be compared that currently need to be processed are determined from the railway monitoring video data.

[0037] Step 120 : Determine the reconstructed pixel value of each pixel point according to the minimum pixel value of the multiple frames of the image frame to be compared at the same pixel point.

[0038] The reconstructed pixel value may be a pixel value of a pixel point reconstructed in a newly created image frame based on the image frame to be compared.

[0039] In an embodiment of the present invention, pixel points are used as comparison units, and pixel values ​​at the same pixel points of multiple frames of images included in the image frame to be compared are compared to determine the minimum pixel value of each pixel point in different images, and the minimum pixel value of each pixel point in the multiple frames of images is used as the reconstructed pixel value of the corresponding pixel point.

[0040] Rain and snow often appear in very obvious white in monitoring, and the brightness of local pixels changes rapidly. The shape and brightness characteristics of rain and snow can be well displayed in the image through pixel values. That is, the pixel points with rain and snow interference in the image can be well located through the minimum pixel value.

[0041] Step 130: reconstruct the rain and snow filtered image according to the reconstructed pixel value of each pixel point.

[0042] The rain and snow filtered image may be a new image frame reconstructed based on the reconstructed pixel values ​​and matching the image frame to be compared.

[0043] In the embodiment of the present invention, a new image frame may be reconstructed based on the reconstructed pixel value of each pixel point as a rain and snow filtered image.

[0044] In the railway monitoring scene, except for high-speed trains, other interferences including pedestrians, rain, snow, flying insects, etc. are all in the category of foreign objects. For normal high-speed rail targets, pedestrian targets and fixed environmental facilities, due to their continuity in the picture, the shape characteristics of the targets can still be retained by taking the minimum pixel of continuous frames; and for interference targets such as rain and snow that stay briefly at local pixel points, they can be well filtered out by the method of continuous frame minimum pixel reconstruction. Based on this method, other short-stay interferences (such as flying insects, etc.) can also be effectively filtered out, that is, while filtering out rain and snow, other short-stay interference targets can also be filtered out synchronously.

[0045] In a specific example, the image before rain and snow removal is as follows: Figure 2 As shown in the left picture of , after processing based on this scheme, the image after rain and snow filtering is completed is as follows Figure 2Shown in the picture on the right.

[0046] The technical solution of the embodiment of the present invention obtains the image frame to be compared in the railway monitoring video data, thereby determining the reconstructed pixel value of each pixel point according to the minimum pixel value of multiple frames of the image frame to be compared at the same pixel point, and then reconstructs the rain and snow filtered image according to the reconstructed pixel value of each pixel point. In this solution, there is no need to perform difficult modeling of rain and snow, and in the case of not needing to obtain rain and snow pictures of actual scenes, it is still possible to obtain a good rain and snow removal effect in various different scenes, and the calculation method is simple and does not require a large amount of calculation. It solves the problem of poor performance of the rain and snow removal algorithm in the image caused by the large amount of calculation, difficult modeling and limited training samples in the current method of removing rain and snow from the image, and can quickly and well filter out the rain and snow interference in the image, with a small amount of calculation and without requiring a large number of paired rain and snow images in actual scenes as training data.

[0047] Embodiment 2

[0048] Figure 3 This is a flowchart of a method for removing rain and snow from an image provided by the second embodiment of the present invention. This embodiment is specific based on the above embodiment and provides a specific optional implementation method for determining the minimum value of a pixel point before determining the reconstructed pixel value of each pixel point based on the minimum value of the pixel at the same pixel point of multiple frames of the image frame to be compared. Figure 3 As shown, the method includes:

[0049] Step 210: Obtain image frames to be compared in the railway monitoring video data.

[0050] In an optional embodiment of the present invention, obtaining the image frames to be compared in the railway monitoring video data may include: determining the image frame sampling frequency; and performing image frame sampling on the railway monitoring video data according to the image frame sampling frequency to obtain the image frames to be compared.

[0051] The image frame sampling frequency may be a sampling frequency of railway monitoring video data, and the image frame sampling frequency may be a preset default value, or may be adjusted according to user needs, or may be automatically and dynamically adjusted based on preset rules.

[0052] In an embodiment of the present invention, the image frame sampling frequency can be configured as a fixed value, or automatically adjusted based on preset rules, or can be adjusted in real time by the user, so that based on the currently set image frame sampling frequency, continuous image frames in the railway monitoring video data are sampled to obtain image frames to be compared.

[0053] In an optional embodiment of the present invention, determining the image frame sampling frequency may include: determining the current image pixel change ratio of the railway monitoring video data based on a pixel-by-pixel comparison method; when the current image pixel change ratio is less than the target preset ratio, using the first sampling frequency as the image frame sampling frequency; when the current image pixel change ratio is not less than the target preset ratio, using the second sampling frequency as the image frame sampling frequency.

[0054] The current image pixel change ratio can be used to describe the ratio of the pixel points whose pixel values ​​change between the current image frame and the previous image frame in the railway monitoring video data to the total number of pixel points in all image frames. The target preset ratio can be a preset ratio value. The first sampling frequency and the second sampling frequency can be two preset sampling frequencies, and the first sampling frequency is less than the second sampling frequency.

[0055] In an embodiment of the present invention, the pixel points whose pixel values ​​change between the current image frame and the previous image frame of the railway monitoring video data can be determined based on a pixel-by-pixel comparison method, thereby calculating the current image pixel point change ratio, and further comparing the current image pixel point change ratio with a first preset ratio. If the current image pixel point change ratio is less than the first preset ratio, the first sampling frequency is used as the image frame sampling frequency. If the current image pixel point change ratio is not less than the target preset ratio, the second sampling frequency is used as the image frame sampling frequency.

[0056] Step 220 , using an image open source library function, compare multiple frames of the image frame to be compared pixel by pixel, and determine the minimum value of the pixel points at the same pixel point of the multiple frames of the image frame to be compared.

[0057] The image open source library function may be an existing library function that supports pixel value comparison pixel by pixel in multiple image frames.

[0058] In an embodiment of the present invention, an image open source library function can be selected, and based on the selected image open source library function, multiple frames of the image frame to be compared are compared pixel by pixel to obtain the minimum value of the pixel points at the same pixel point of the multiple frames of the image frame to be compared.

[0059] Step 230 : Determine the reconstructed pixel value of each pixel point according to the minimum pixel value of the multiple frames of the image frame to be compared at the same pixel point.

[0060] Step 240: reconstruct the rain and snow filtered image according to the reconstructed pixel value of each pixel point.

[0061] In an optional embodiment of the present invention, reconstructing the rain and snow filtered image according to the reconstructed pixel value of each pixel point may include: reconstructing the rain and snow filtered image according to the reconstructed pixel value of each pixel point and the pixel point position of each pixel point. After reconstructing the rain and snow filtered image, it may also include: displaying the rain and snow filtered image.

[0062] In an embodiment of the present invention, the position of the pixel point corresponding to each reconstructed pixel value in the image frame can be determined, so that at the pixel point position of each pixel point, a new image frame is reconstructed according to the corresponding reconstructed pixel value to obtain a rain and snow filtered image, and the rain and snow filtered image is displayed.

[0063] In an optional embodiment of the present invention, after reconstructing the rain and snow filtered image based on the reconstructed pixel value of each pixel and the pixel position of each pixel, it can also include: performing semantic segmentation and / or target detection on the rain and snow filtered image to obtain target recognition data.

[0064] The target recognition data may be analysis data of the rain and snow filtered image.

[0065] In an embodiment of the present invention, the method for removing rain and snow from an image can be transplanted into the preprocessing stage of various algorithms. For example, the method for removing rain and snow from an image can be transplanted into a semantic segmentation algorithm and / or a target detection algorithm to further perform semantic segmentation and / or target detection on the rain and snow filtered image to obtain target recognition data, thereby providing a data basis for subsequent image analysis.

[0066] The technical solution of the embodiment of the present invention obtains the image frame to be compared in the railway monitoring video data, and then uses the image open source library function to compare the multiple frames of the image frame to be compared pixel by pixel, determine the minimum value of the pixel points of the multiple frames of the image frame to be compared at the same pixel point, and then determine the reconstructed pixel value of each pixel point according to the minimum value of the pixel points of the multiple frames of the image frame to be compared at the same pixel point, and reconstruct the rain and snow filtered image according to the reconstructed pixel value of each pixel point. In this solution, it is not necessary to perform high-difficulty modeling on rain and snow, and in the case of not needing to obtain rain and snow pictures of actual scenes, it is still possible to obtain a good rain and snow removal effect in various different scenes, and the calculation method is simple, and does not require a large amount of calculation, which solves the problem that the current method of removing rain and snow from images has a large amount of calculation, difficult modeling and limited training samples, resulting in poor performance of the rain and snow removal algorithm in the image, and can quickly and well filter out rain and snow interference in the image, with a small amount of calculation and without requiring a large number of paired rain and snow images in actual scenes as training data.

[0067] Embodiment 3

[0068] Figure 4 FIG. 1 is a schematic diagram of a rain and snow removal device in an image provided by Embodiment 3 of the present invention. Figure 4 As shown, the device comprises:

[0069] The image frame acquisition module 310 is used to acquire the image frame to be compared in the railway monitoring video data;

[0070] A reconstructed pixel value determination module 320, for determining a reconstructed pixel value of each pixel point according to the minimum pixel value of the multiple frames of the image frame to be compared at the same pixel point;

[0071] The image reconstruction module 330 is used to reconstruct the rain and snow filtered image according to the reconstructed pixel value of each pixel point.

[0072] The technical solution of the embodiment of the present invention obtains the image frame to be compared in the railway monitoring video data, thereby determining the reconstructed pixel value of each pixel point according to the minimum pixel value of multiple frames of the image frame to be compared at the same pixel point, and then reconstructs the rain and snow filtered image according to the reconstructed pixel value of each pixel point. In this solution, there is no need to perform difficult modeling of rain and snow, and in the case of not needing to obtain rain and snow pictures of actual scenes, it is still possible to obtain a good rain and snow removal effect in various different scenes, and the calculation method is simple and does not require a large amount of calculation. It solves the problem of poor performance of the rain and snow removal algorithm in the image caused by the large amount of calculation, difficult modeling and limited training samples in the current method of removing rain and snow from the image, and can quickly and well filter out the rain and snow interference in the image, with a small amount of calculation and without requiring a large number of paired rain and snow images in actual scenes as training data.

[0073] Optionally, the image frame acquisition module 310 is specifically used to determine an image frame sampling frequency; and perform image frame sampling on the railway monitoring video data according to the image frame sampling frequency to obtain the image frame to be compared.

[0074] Optionally, the image frame acquisition module 310 is specifically used to determine the current image pixel change ratio of the railway monitoring video data based on a pixel-by-pixel comparison method; when the current image pixel change ratio is less than a target preset ratio, the first sampling frequency is used as the image frame sampling frequency; when the current image pixel change ratio is not less than a target preset ratio, the second sampling frequency is used as the image frame sampling frequency.

[0075] Optionally, the rain and snow removal device in the image also includes a pixel minimum value determination module, which is used to use the image open source library function to compare the multiple frames of the image frames to be compared pixel by pixel, and determine the minimum pixel value of the multiple frames of the image frames to be compared at the same pixel point.

[0076] Optionally, the image reconstruction module 330 is specifically used to reconstruct the rain and snow filtered image according to the reconstructed pixel value of each pixel and the pixel position of each pixel. The rain and snow removal device in the image also includes an image display module for displaying the rain and snow filtered image.

[0077] Optionally, the device for removing rain and snow from an image further includes a data processing module for performing semantic segmentation and / or target detection on the rain and snow filtered image to obtain target recognition data.

[0078] The device for removing rain and snow from an image provided by an embodiment of the present invention can execute the method for removing rain and snow from an image provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0079] Embodiment 4

[0080] Figure 5 A schematic diagram of an electronic device that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also be represented as a camera. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0081] like Figure 5 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0082] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0083] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as a method for removing rain and snow from an image.

[0084] In some embodiments, the rain and snow removal method in the image can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the rain and snow removal method in the image described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the rain and snow removal method in the image in any other suitable manner (for example, by means of firmware).

[0085] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0086] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0087] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0088] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0089] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0090] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of traditional physical hosts and VPS servers, which are difficult to manage and have weak business scalability.

[0091] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0092] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for removing rain and snow from an image, characterized in that: include: Obtaining image frames to be compared in railway monitoring video data; Determine the reconstructed pixel value of each pixel point according to the minimum pixel value of the multiple frames of the image frame to be compared at the same pixel point; The rain and snow filtered image is reconstructed according to the reconstructed pixel value of each pixel.

2. The method according to claim 1, characterized in that The step of obtaining the image frame to be compared in the railway monitoring video data comprises: Determine the image frame sampling frequency; Image frame sampling is performed on the railway monitoring video data according to the image frame sampling frequency to obtain the image frame to be compared.

3. The method according to claim 2, characterized in that The determining of the image frame sampling frequency comprises: Determine the current image pixel change ratio of the railway monitoring video data based on a pixel-by-pixel comparison method; When the change ratio of the pixels of the current image is less than the target preset ratio, using the first sampling frequency as the image frame sampling frequency; When the change ratio of the pixels of the current image is not less than the target preset ratio, the second sampling frequency is used as the image frame sampling frequency.

4. The method according to claim 1, characterized in that Before determining the reconstructed pixel value of each pixel point according to the minimum pixel value of the multiple frames of the image frame to be compared at the same pixel point, the method further includes: By using an image open source library function, the multiple frames of the image frame to be compared are compared pixel by pixel, and the minimum value of the pixel points of the multiple frames of the image frame to be compared at the same pixel point is determined.

5. The method according to claim 1, characterized in that The step of reconstructing the rain and snow filtered image according to the reconstructed pixel value of each pixel point includes: Reconstructing the rain and snow filtered image according to the reconstructed pixel value of each pixel and the pixel position of each pixel; After reconstructing the rain and snow filtered image, the method further includes: The rain and snow filtered image is displayed.

6. The method according to claim 5, characterized in that After reconstructing the rain and snow filtered image according to the reconstructed pixel value of each pixel and the pixel position of each pixel, the method further includes: The rain and snow filtered image is subjected to semantic segmentation and / or target detection to obtain target recognition data.

7. A device for removing rain and snow from an image, characterized in that: include: An image frame acquisition module is used to acquire image frames to be compared in railway monitoring video data; A reconstructed pixel value determination module, used to determine the reconstructed pixel value of each pixel point according to the minimum pixel value of the multiple frames of the image frame to be compared at the same pixel point; The image reconstruction module is used to reconstruct the rain and snow filtered image according to the reconstructed pixel value of each pixel point.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the method for removing rain and snow from an image according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for removing rain and snow from an image according to any one of claims 1 to 6 when executed.

10. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the method for removing rain and snow in an image according to any one of claims 1 to 6.

Images