Image enhancement method, device and computer equipment

Abstract

The present disclosure relates to the technical field of image processing, and specifically discloses an image enhancement method and device and a computer device. The method comprises: collecting a target area's inspection original image; inputting the inspection original image into an image enhancement model and obtaining an inspection enhanced image output by the image enhancement model; wherein the image enhancement model is obtained by training a deep learning network, and the training sample of the image enhancement model comprises a standard inspection image set and a fuzzy inspection image set of a target area, and the fuzzy inspection image in the fuzzy inspection image set has a corresponding standard inspection image in the standard inspection image set. The present disclosure greatly improves the clarity of the inspection image collected by the inspection robot, and improves the image quality of the inspection under the premise of not increasing the overall cost and maintaining the normal inspection speed.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to an image enhancement method, apparatus, and computer equipment. Background Technology

[0002] In applications such as intelligent monitoring, surveillance, security, and remote centralized data management, inspection robots can undertake repetitive, high-frequency, and high-intensity inspection tasks through intelligent technology, freeing up manpower. This has led to rapid development in the image processing technology of inspection robots. Especially in complex environments, inspection robots are prone to significant bumps during movement, causing camera shake and resulting in blurred images.

[0003] In related technologies, shaking is often mitigated by reducing the inspection travel speed or adding corresponding image stabilization devices to the bottom of the camera. However, reducing the inspection travel speed significantly reduces inspection efficiency and places higher demands on the inspection robot's battery life; while adding image stabilization devices can only alleviate image blur to a certain extent, and the stabilization effect has limitations, and it greatly increases the hardware maintenance cost of the inspection robot. Summary of the Invention

[0004] Therefore, it is necessary to provide an image enhancement method, apparatus, computer device, storage medium, and computer program product to address the aforementioned technical problems.

[0005] In a first aspect, this disclosure provides an image enhancement method. The method includes:

[0006] Collect raw images of the target area during inspection;

[0007] The original inspection image is input into the image enhancement model, and the enhanced inspection image output by the image enhancement model is obtained.

[0008] The image enhancement model is trained through a deep learning network. The training samples of the image enhancement model include a standard inspection image set and a fuzzy inspection image set of the target region. The fuzzy inspection images in the fuzzy inspection image set are standard inspection images with corresponding positional relationships in the standard inspection image set.

[0009] In one embodiment, the image enhancement model is constructed in the following manner:

[0010] Acquire standard inspection videos and blurred inspection videos of the target area;

[0011] Based on the acquisition location information of the standard inspection image in the standard inspection video and the blurred inspection image in the blurred inspection video, the standard inspection image and the blurred inspection image corresponding to the acquisition location information are extracted one by one to obtain the standard inspection image set and the blurred inspection image set.

[0012] The image enhancement model is trained on the standard inspection image set and the blurred inspection image set based on the deep learning network to obtain a mature image enhancement model.

[0013] In one embodiment, acquiring the standard inspection video and the blurred inspection video of the target area includes:

[0014] Standard inspection images are acquired within the target area at a standard inspection speed to obtain the standard inspection video;

[0015] A blurred inspection image is acquired within the target area at a target inspection speed to obtain the blurred inspection video; the standard inspection speed is less than the target inspection speed.

[0016] In one embodiment, the deep learning network employs a generative adversarial network (GAN), which includes a generator network and a decision network. The step of training the standard inspection image set and the fuzzy inspection image set based on the deep learning network to obtain a mature image enhancement model includes:

[0017] The blurred inspection images of the blurred inspection image set are input into the generator network to obtain the pseudo-enhanced images output by the generator network;

[0018] The judgment network compares and judges the pseudo-enhanced image of the blurred inspection image with the standard inspection image corresponding to the blurred inspection image, and obtains the error parameters of the pseudo-enhanced image and the standard inspection image output by the judgment network.

[0019] The generator network is optimized and adjusted according to the error parameters. The process of inputting the blurred inspection images of the blurred inspection image set into the generator network is repeated until the error parameters meet the training conditions of the generator network.

[0020] In one embodiment, the training conditions for the generated network include the generation network not reaching a preset number of iterations, and the error parameter converging and being less than or equal to a preset error threshold; or the generation network reaching the preset number of iterations, and the error parameter diverging or being greater than the preset error threshold.

[0021] Secondly, this disclosure also provides an image enhancement apparatus. The apparatus includes:

[0022] The image acquisition module is used to acquire raw images of the target area during inspection.

[0023] The image enhancement module is used to input the original inspection image into the image enhancement model and obtain the inspection enhancement image output by the image enhancement model;

[0024] The image enhancement model is trained through a deep learning network. The training samples of the image enhancement model include a standard inspection image set and a fuzzy inspection image set of the target region. The fuzzy inspection images in the fuzzy inspection image set are standard inspection images with corresponding positional relationships in the standard inspection image set.

[0025] In one embodiment, the device further includes:

[0026] The model training module is used to construct the image enhancement model;

[0027] The model training module includes:

[0028] Training sample unit, used to acquire standard inspection video and blurred inspection video of the target area;

[0029] The image correspondence unit is used to extract the standard inspection image and the fuzzy inspection image that correspond one-to-one with the acquisition location information of the standard inspection image in the standard inspection video and the fuzzy inspection image in the fuzzy inspection video, so as to obtain the standard inspection image set and the fuzzy inspection image set.

[0030] The deep learning unit is used to train the standard inspection image set and the fuzzy inspection image set based on the deep learning network to obtain a well-trained image enhancement model.

[0031] In one embodiment, the training sample unit includes:

[0032] A standard inspection video subunit is used to acquire the standard inspection video collected in the target area at a standard inspection speed;

[0033] A fuzzy inspection video subunit is used to acquire the fuzzy inspection video collected within the target area at a target inspection speed; the standard inspection speed is less than the target inspection speed.

[0034] In one embodiment, the deep learning network employs a generative adversarial network (GAN), which includes a generator network and a decision network. The deep learning unit includes:

[0035] The pseudo-enhancement subunit is used to input the fuzzy inspection image of the fuzzy inspection image set into the generator network and obtain the pseudo-enhancement image output by the generator network.

[0036] The judgment subunit is used to compare and judge the pseudo-enhanced image of the blurred inspection image with the standard inspection image corresponding to the blurred inspection image through the judgment network, and to obtain the error parameters of the pseudo-enhanced image and the standard inspection image output by the judgment network.

[0037] An optimization subunit is used to optimize and adjust the generator network according to the error parameters, and to repeatedly input the fuzzy inspection images of the fuzzy inspection image set into the generator network until the error parameters meet the training conditions of the generator network.

[0038] In one embodiment, the training conditions for the generated network include the generation network not reaching a preset number of iterations, and the error parameter converging and being less than or equal to a preset error threshold; or the generation network reaching the preset number of iterations, and the error parameter diverging or being greater than the preset error threshold.

[0039] Thirdly, this disclosure also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the above-described image enhancement method.

[0040] Fourthly, this disclosure also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the above-described image enhancement method.

[0041] Fifthly, this disclosure also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the above-described image enhancement method.

[0042] The above-mentioned image enhancement methods, apparatus, computer equipment, storage media, and computer program products have at least the following beneficial effects:

[0043] This disclosure achieves image enhancement processing of the original inspection images by inputting the original inspection images collected by the inspection robot into the image enhancement model and obtaining a clear inspection enhanced image output by the image enhancement model. This effectively eliminates the image blurring caused by the random shaking of the inspection robot during the inspection process, greatly improves the clarity of the inspection images collected by the inspection robot, and improves the image quality of the inspection without increasing the overall cost and maintaining the normal inspection speed. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments or conventional technologies of this disclosure, the accompanying drawings used in the description of the embodiments or conventional technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a diagram illustrating the application environment of an image enhancement method in one embodiment;

[0046] Figure 2 This is a flowchart illustrating an image enhancement method in one embodiment;

[0047] Figure 3 This is a flowchart illustrating the steps involved in constructing an image enhancement model in one embodiment.

[0048] Figure 4 This is a flowchart illustrating the steps for obtaining fuzzy inspection videos and standard inspection videos in one embodiment.

[0049] Figure 5 This is a flowchart illustrating the training steps of an image enhancement model in one embodiment;

[0050] Figure 6 This is a schematic diagram of deep learning for an image enhancement model in one embodiment;

[0051] Figure 7 This is a structural block diagram of an image enhancement device in one embodiment;

[0052] Figure 8 This is a structural block diagram of the image enhancement device in another embodiment;

[0053] Figure 9 This is a block diagram of the model training module in one embodiment;

[0054] Figure 10 Here is a block diagram of the training sample unit in one embodiment;

[0055] Figure 11 This is a block diagram of the structure of a deep learning unit in one embodiment;

[0056] Figure 12 This is a block diagram of the internal structure of a computer device in one embodiment. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of this disclosure.

[0059] The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims. The terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitation, the presence of additional identical or equivalent elements in a process, method, product, or apparatus that includes said elements is not excluded. For example, the use of terms such as "first," "second," etc., is to denote names and does not indicate any particular order.

[0060] When used herein, the singular forms of “a,” “an,” and “the” may also include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms “comprising / including” or “having,” etc., specify the presence of the stated features, wholes, steps, operations, components, parts, or combinations thereof, but do not preclude the possibility of the presence or addition of one or more other features, wholes, steps, operations, components, parts, or combinations thereof. Meanwhile, in this specification, the term “and / or” includes any and all combinations of the associated listed items.

[0061] In applications such as intelligent monitoring, surveillance, security, and remote centralized data management, inspection robots can leverage intelligent technology to handle repetitive, high-frequency, and high-intensity inspection tasks, freeing up manpower. For example... Figure 1In the illustrated application environment, the inspection robot 102 can communicate with the server 104 via a network. A data storage system can store the data that the server 104 needs to process. The data storage system can be integrated into the server 104 or placed in the cloud or on another network server. The inspection robot 102 is equipped with a controller and an image acquisition device. The inspection robot 102 can acquire inspection images of the target area using the image acquisition device (e.g., a camera), perform image enhancement processing on the inspection images, and then send them to the server 104 for monitoring and management. Image enhancement processing can also be implemented through the server 104. The server 104 can be a standalone server or a server cluster consisting of multiple servers. In some embodiments, the inspection robot may not need to wirelessly communicate with the server; the data storage system can be integrated into the robot's body, acquiring inspection images of the target area through its own image acquisition device (e.g., a camera), performing image enhancement processing on the inspection images, and then monitoring and managing them. However, in complex environments, the inspection robot often experiences significant bumps during movement, causing camera shake and resulting in blurred images. Some related technologies often mitigate shaking by reducing the inspection robot's travel speed or adding corresponding image stabilization devices to the bottom of the camera. However, reducing the inspection travel speed significantly reduces inspection efficiency and places higher demands on the robot's battery life; while adding image stabilization devices can only alleviate image blur to a certain extent, and the stabilization effect has limitations, while also greatly increasing the hardware maintenance costs of the inspection robot.

[0062] To alleviate the problem that inspection robots often experience significant bumps during movement, causing camera shake and resulting in blurry images. For example... Figure 2 As shown, in some embodiments of this disclosure, an image enhancement method is provided, which includes the following steps:

[0063] Step 210: Collect the original inspection images of the target area.

[0064] For example, the image acquisition device of the inspection robot acquires raw inspection images of the target area as the robot travels along the inspection path. These raw images can be directly acquired images that require image enhancement, or images that have already undergone preprocessing but still require image enhancement. The target area can be a pre-defined area that the inspection robot is instructed to inspect; for example, the target area could be a tunnel with a complex environment and slopes, causing the robot to experience bumps during inspection. In some embodiments, a server can issue inspection commands to the robot, instructing it to inspect within the target area.

[0065] Step 220: Input the original inspection image into the image enhancement model and obtain the inspection enhancement image output by the image enhancement model; wherein, the image enhancement model is trained by a deep learning network, and the training samples of the image enhancement model include a standard inspection image set and a fuzzy inspection image set of the target area, wherein the fuzzy inspection images in the fuzzy inspection image set are standard inspection images with corresponding positional relationships in the standard inspection image set.

[0066] For example, the original inspection images collected by the inspection robot are processed by an image enhancement model. The inspection robot can input the original inspection images into the image enhancement model and obtain the enhanced inspection images output by the model. The image enhancement model can perform image enhancement processing on the input original inspection images, outputting clearer enhanced inspection images.

[0067] The image enhancement model can be trained using a deep learning network. In this embodiment, the training samples for the image enhancement model include a standard inspection image set and a blurred inspection image set of the target region. That is, the image enhancement model is trained using deep learning based on the standard inspection image set and the blurred inspection image set of the target region.

[0068] In some implementations, the fuzzy inspection images in the fuzzy inspection image set typically refer to blurry images acquired by the inspection robot that do not meet the sharpness requirements, while the standard inspection images in the standard inspection image set are typically standard images acquired by the inspection robot that meet the sharpness requirements. Each fuzzy inspection image in the fuzzy inspection image set has a corresponding standard inspection image in the standard inspection image set. This correspondence is a positional relationship during the inspection process; that is, the fuzzy inspection images and standard images acquired by the inspection robot at the same location have a corresponding positional relationship.

[0069] In the aforementioned image enhancement method, a correlation is established between blurry inspection images and standard inspection images collected by the inspection robot at the same location based on their corresponding positional relationships. An image correspondence method is used to generate a corresponding sequence of blurry and standard inspection images. Then, a deep learning model is trained based on this sequence to obtain the image enhancement model. Furthermore, during actual inspections, the original inspection images collected by the inspection robot can be input into the image enhancement model, and a clear enhanced inspection image can be obtained from the model's output. This effectively eliminates image blurring caused by random shaking of the inspection robot during inspections, significantly improving the clarity of the inspection images collected by the robot. This enhances the image quality of the inspection without increasing overall costs and while maintaining normal inspection speed.

[0070] In some embodiments of this disclosure, such as Figure 3 As shown, the methods for constructing image enhancement models include:

[0071] Step 310: Obtain the standard inspection video and the blurred inspection video of the target area.

[0072] For example, the inspection robot can acquire learning samples for training the image enhancement model. First, it acquires a standard inspection video and a blurry inspection video of the target area. The standard inspection video can refer to the standard inspection video of the target area that meets the clarity requirements, and the blurry inspection video can refer to the standard inspection video of the target area that does not meet the clarity requirements.

[0073] Step 320: Based on the acquisition location information of the standard inspection image in the standard inspection video and the fuzzy inspection image in the fuzzy inspection video, extract the standard inspection image and the fuzzy inspection image that correspond one-to-one with the acquisition location information to obtain the standard inspection image set and the fuzzy inspection image set.

[0074] For example, after acquiring standard inspection video and blurry inspection video, the inspection robot extracts corresponding standard inspection images and blurry inspection images from the standard and blurry inspection videos respectively, based on the acquisition location information, thus obtaining a standard inspection image set and a blurry inspection image set. That is, by extracting standard inspection images and blurry inspection images located at the same position on the inspection path from the standard and blurry inspection videos respectively, a correspondence is bound between the standard and blurry inspection images located at the same position on the inspection path. This results in a standard inspection image set and a blurry inspection image set, where the acquisition location information of the standard inspection images in the standard inspection image set and the blurry inspection images in the blurry inspection image set correspond one-to-one.

[0075] Optionally, in some embodiments, the inspection robot is also equipped with a positioning device, such as a meter-counting wheel or GPS, which can be used to obtain the location information of the inspection images collected by the robot. The frame rate of the inspection video recorded by the robot during the acquisition of standard and blurry inspection videos is usually kept consistent. The frequency at which the inspection robot acquires real-time location information is not lower than the frame rate of the recorded inspection video, and the frequency is an integer multiple of the frame rate. For example, if the inspection robot acquires real-time location information at a frequency of 100 Hz and the frame rate of the recorded inspection video is 25 frames per second, accurate location information can be obtained for each frame of the inspection image. This location information allows for the mapping between standard and blurry inspection images; that is, for each blurry inspection image, a clear standard inspection image can be found to correspond to it.

[0076] Step 330: Train the standard inspection image set and the blurred inspection image set based on the deep learning network to obtain a well-trained image enhancement model.

[0077] For example, by using the acquired standard inspection image set and blurred inspection image set as learning samples, and training based on a deep learning network, a well-trained image enhancement model can be obtained.

[0078] This embodiment pre-collects standard inspection videos and blurred inspection videos, and extracts standard and blurred inspection images with one-to-one positional correspondences from the standard and blurred inspection videos based on the collected location information, thus obtaining a standard inspection image set and a blurred inspection image set. This quickly establishes the correspondence between the standard and blurred inspection images and obtains an effective training sample set of standard and blurred inspection images. Therefore, in some embodiments, using the above-described image enhancement model construction method only requires pre-training the inspection robot on the inspection path to be inspected. In subsequent use, it can perform inspections at normal speed, even in environments with bumps caused by racks or foreign objects such as sand and gravel, thus acquiring clear images, improving inspection speed, and ensuring image quality. Image training involves the inspection robot acquiring standard inspection videos at low speeds, then acquiring blurred inspection videos at normal speeds. Based on positional correspondence, a correlation is established between the standard inspection image set of the standard inspection video and the blurred inspection image set of the blurred inspection video. This allows the robot to use the standard inspection images extracted from the standard inspection image set to perform image enhancement processing on the corresponding blurred images, ensuring image quality.

[0079] In some embodiments of this disclosure, such as Figure 4 As shown, step 310 includes:

[0080] Step 312: Acquire the standard inspection video collected in the target area at the standard inspection speed.

[0081] For example, standard inspection video can be obtained by instructing an inspection robot to inspect the target area at a standard inspection speed and collect standard inspection images. It is important to note that when the inspection robot performs inspections at a standard speed, the collected standard inspection video meets the clarity requirements. The standard inspection speed can be a relatively slow speed, such as 0.2 m / s, 0.1 m / s, or other suitable speeds. However, in practical applications, a slower inspection speed is often insufficient to meet the efficiency requirements when the inspection robot performs daily inspection tasks, and a faster inspection speed is often necessary.

[0082] Step 314: Acquire the blurred inspection video collected in the target area at the target inspection speed; the standard inspection speed is less than the target inspection speed.

[0083] For example, obtaining blurry inspection video can be achieved by instructing an inspection robot to inspect a target area at a target inspection speed and collect blurry inspection images. The target inspection speed refers to the inspection speed adopted by the inspection robot when performing the inspection task, which is often greater than the standard inspection speed.

[0084] This embodiment uses the normal inspection speed during inspection as the target inspection speed, acquires fuzzy inspection images, and acquires clear standard inspection images at a slow standard inspection speed. This not only obtains a set of standard inspection images and a set of fuzzy inspection images that correspond one-to-one with the positions on the inspection trajectory, but also allows the target inspection speed to be determined according to the requirements of the inspection task. For example, in practical applications, the inspection speed of the inspection robot can be determined by combining the characteristics of the inspection trajectory and the inspection time requirements, thereby determining the target inspection speed. This ensures that the speed of the inspection robot when performing the inspection task does not exceed the target inspection speed, thus obtaining the corresponding set of fuzzy inspection images. The learning samples of the fuzzy inspection image set are more adaptable and accurate, thereby improving the image clarity quality.

[0085] In some embodiments, the set of blurred inspection images during the training process of the image enhancement model is obtained based on the target inspection speed. In step 210, the inspection robot can be controlled to perform the inspection task at a speed not exceeding the target inspection speed, and to collect the original inspection image of the target area.

[0086] Optionally, the target inspection speed includes at least a first speed and a second speed. A first fuzzy inspection image set is obtained based on the first speed, and a first image enhancement model is trained based on the first fuzzy inspection image set and a standard inspection image set. A second fuzzy inspection image set is obtained based on the second speed, and a second image enhancement model is trained based on the second fuzzy inspection image set and a standard image set. The first speed is greater than zero and less than the second speed. The training steps for the first and second image enhancement models can refer to steps 310-330 above, and will not be repeated here.

[0087] When performing inspection tasks, the inspection robot can generate corresponding fuzzy inspection image sets for the same inspection trajectory using two or more target inspection speeds, which can then be used to generate corresponding image enhancement models. For example:

[0088] Based on the inspection task information, the actual inspection speed for executing the inspection task is determined, and the inspection robot is controlled to collect a set of fuzzy inspection images along the inspection trajectory at the actual inspection speed. The image enhancement model corresponding to the actual inspection speed is determined, and the set of fuzzy inspection images collected along the inspection trajectory at the actual inspection speed is input into the corresponding image enhancement model. For example, if the actual inspection speed is greater than zero and less than or equal to a first speed, the image enhancement model corresponding to this actual inspection speed is the first image enhancement model; if the actual inspection speed is greater than the first speed and less than or equal to a second speed, the image enhancement model corresponding to this actual inspection speed is the second image enhancement model.

[0089] In this embodiment, the inspection robot can flexibly determine the actual inspection speed by combining the specific inspection trajectory and inspection requirements, and determine the corresponding image enhancement model by the speed range of the actual inspection speed, which further improves the accuracy of image enhancement; and when inspecting the same inspection section, the inspection speed can be flexibly adjusted while improving image clarity.

[0090] In other embodiments, the inspection robot's inspection trajectory has different inspection zones when performing inspection tasks. The robot can be controlled to acquire raw inspection images at different inspection speeds in different inspection zones, and then the raw inspection images acquired in different inspection zones can be processed using different image enhancement models to obtain enhanced inspection images. For example:

[0091] Based on the inspection task information, it can be determined that the inspection trajectory includes at least a first inspection zone and a second inspection zone. The target inspection speed corresponding to the first inspection zone is the first speed, and the target inspection speed corresponding to the second inspection zone is the second speed. The first speed is greater than zero and less than the second speed. The position information of the inspection robot is acquired in real time, and the current inspection zone of the inspection robot is determined based on this information. If the inspection robot is determined to be currently in the first inspection zone, it is controlled to perform the inspection task in the first inspection zone at a speed greater than zero and less than or equal to the first speed. The original inspection image acquired in the first inspection zone is processed through a first image enhancement model to obtain an enhanced inspection image. If the inspection robot is determined to be currently in the second inspection zone, it is controlled to perform the inspection task in the second inspection zone at a speed greater than the first speed and less than or equal to the second speed. The original inspection image acquired in the second inspection zone is processed through a second image enhancement model to obtain an enhanced inspection image.

[0092] It is important to note that when the inspection trajectory includes a first inspection zone and a second inspection zone, each of the first and second inspection zones can have one or more segments. That is, the inspection trajectory can consist of several segments of the first inspection zone and several segments of the second inspection zone. Furthermore, the inspection trajectory can also include three or more inspection zones. Different inspection zones have corresponding target inspection speeds and corresponding image enhancement models, based on the same principle as an inspection trajectory including a first and second inspection zone.

[0093] This embodiment divides the inspection trajectory based on specific inspection paths, with each inspection zone having a corresponding target inspection speed. This allows the inspection robot to determine the inspection speed according to the inspection requirements of each zone, further improving inspection efficiency. Simultaneously, different inspection zones obtain corresponding image enhancement models according to their target inspection speeds. These image enhancement models not only improve the targeting of the blurred inspection image set and greatly enhance the accuracy of the enhancement, but also allow the image enhancement model to be determined according to the inspection requirements of different inspection zones, improving the adaptability to inspection tasks.

[0094] In some embodiments of this disclosure, such as Figure 5 As shown, step 330 includes:

[0095] Step 332: Input the fuzzy inspection image of the fuzzy inspection image set into the generator network to obtain the pseudo-enhanced image output by the generator network.

[0096] The image enhancement model can be trained using a Generative Adversarial Network (GAN), which is a training model typically found in deep learning. A GAN consists of a generator network and a decision network. The generator network generates images; in this embodiment, it's used to generate sharper pseudo-enhanced images based on blurry inspection images. The decision network, a discriminative network, determines whether an image meets the sharpness requirements; in this embodiment, it compares the pseudo-enhanced image with the corresponding standard inspection image and determines the error parameters between the pseudo-enhanced image and the standard inspection image. It's important to note that a GAN can be replaced by other deep learning networks that learn from standard inspection images to enhance blurry inspection images.

[0097] For example, combined Figure 6 As shown, the blurred inspection images in the obtained blurred inspection image set are input into the generator network, so that the generator network generates a clearer pseudo-enhanced image corresponding to the blurred inspection image.

[0098] Step 334: The judgment network compares the pseudo-enhanced image of the blurred inspection image with the standard inspection image corresponding to the blurred inspection image, and obtains the error parameters of the pseudo-enhanced image and the standard inspection image output by the judgment network.

[0099] For example, the pseudo-enhanced image output by the generating network and the corresponding standard inspection image are input into the judgment network. The judgment network compares and judges the pseudo-enhanced image and the standard inspection image to obtain the error parameters between the pseudo-enhanced image and the standard inspection image.

[0100] Step 336: Optimize and adjust the generator network according to the error parameters, and repeat the step of inputting the fuzzy inspection images of the fuzzy inspection image set into the generator network until the error parameters meet the training conditions of the generator network.

[0101] For example, the generator network is optimized and adjusted based on the error parameters output by the judgment network. For example, the neuron parameters in the generator network are adjusted, and steps 332 to 336 above are repeated. That is, the blurred inspection image is input into the optimized generator network to obtain a new pseudo-enhanced image. The judgment network judges the new pseudo-enhanced image and the standard inspection image to obtain new error parameters. The generator network is optimized and adjusted based on the new error parameters until the error parameters meet the training conditions of the generator network. The repetition stops, and a well-trained generator network is obtained as an image enhancement model.

[0102] This embodiment continuously generates pseudo-enhanced images of blurred inspection images through a generative adversarial network (GAN). A judgment network then compares the pseudo-enhanced images with the corresponding standard inspection images to obtain their error parameters. These error parameters are then fed back to the generative network for optimization, resulting in a well-trained generative network that can efficiently generate pseudo-enhanced images and accurately distinguish between them and standard inspection images, greatly improving the model's training efficiency and accuracy.

[0103] In some embodiments of this disclosure, the training conditions for the generated network include the convergence of the error parameter and its being less than a preset error threshold, or the number of optimization iterations of the generated network reaching a preset number of iterations.

[0104] For example, during the repetition of steps 332-336 above, the number of times the generator network can be optimized and adjusted is limited, and the repetition can be stopped when any of the following training conditions are met.

[0105] (1) The number of optimization iterations of the generated network has not reached the preset number of iterations, and the error parameter has converged and is less than or equal to the preset error threshold.

[0106] (2) The number of optimizations of the generated network reaches the preset number of iterations, and the error parameter diverges or the error parameter is greater than the preset error threshold.

[0107] For example, when the generative network obtains error parameters based on a fuzzy inspection image set in a certain iteration, it can determine whether the accumulated number of iterations has reached a preset number of iterations. If the preset number of iterations has not been reached, it can determine whether the error parameters have converged based on the error parameters of previous iterations, and whether the obtained error parameters are less than a preset error threshold. If the error parameters have converged and are less than or equal to the preset error threshold, the repeated iterations stop, and the generative network is considered mature. Otherwise, the optimization iteration of the generative network continues.

[0108] If the preset number of iterations is reached, the convergence of the error parameters can be determined based on the error parameters from each iteration, and it can be checked whether the obtained error parameters are less than a preset error threshold. If the error parameters are determined to be divergent, or if the error parameters are greater than the preset error threshold, the repeated iterations are stopped. At this point, the network training fails, and the training samples for the image enhancement model can be re-determined, and training can be repeated.

[0109] This embodiment stops repeating when the generator network can effectively generate pseudo-enhanced images that meet the sharpness requirements, or when the generator network reaches the preset number of optimization iterations, the error parameters diverge, or the generated pseudo-enhanced images still do not meet the sharpness requirements. The generator network parameters or input / output are then verified, and training is performed again.

[0110] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0111] Based on the same inventive concept, this disclosure also provides an image enhancement apparatus for implementing the image enhancement method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, specific limitations in one or more image enhancement apparatus embodiments provided below can be found in the limitations of the image enhancement method described above, and will not be repeated here.

[0112] In some embodiments of this disclosure, such as Figure 7 As shown, an image enhancement device is provided. The device 700 includes:

[0113] Image acquisition module 710 is used to acquire raw images of the target area during inspection;

[0114] The image enhancement module 720 is used to input the original inspection image into the image enhancement model and obtain the inspection enhancement image output by the image enhancement model;

[0115] The image enhancement model is trained through a deep learning network. The training samples of the image enhancement model include a standard inspection image set and a fuzzy inspection image set of the target region. The fuzzy inspection images in the fuzzy inspection image set are standard inspection images with corresponding positional relationships in the standard inspection image set.

[0116] In some embodiments of this disclosure, such as Figure 8 As shown, the device 700 further includes:

[0117] Model training module 730 is used to construct the image enhancement model;

[0118] like Figure 9 As shown, the model training module 730 includes:

[0119] Training sample unit 732 is used to acquire standard inspection videos and blurred inspection videos of the target area;

[0120] The image correspondence unit 734 is used to extract the standard inspection image and the fuzzy inspection image that correspond one-to-one with the acquisition location information of the standard inspection image in the standard inspection video and the fuzzy inspection image in the fuzzy inspection video, so as to obtain the standard inspection image set and the fuzzy inspection image set.

[0121] The deep learning unit 736 is used to train the standard inspection image set and the fuzzy inspection image set based on the deep learning network to obtain a well-trained image enhancement model.

[0122] In some embodiments of this disclosure, such as Figure 10 As shown, the training sample unit 732 includes:

[0123] The standard inspection video subunit 7322 is used to acquire the standard inspection video collected in the target area at a standard inspection speed;

[0124] The fuzzy inspection video subunit 7324 is used to acquire the fuzzy inspection video collected in the target area at a target inspection speed; the standard inspection speed is less than the target inspection speed.

[0125] In some embodiments of this disclosure, such as Figure 11 As shown, the deep learning network employs a generative adversarial network (GAN), which includes a generator network and a decision network. The deep learning unit 736 includes:

[0126] The pseudo-enhancement subunit 7362 is used to input the fuzzy inspection image of the fuzzy inspection image set into the generator network and obtain the pseudo-enhancement image output by the generator network.

[0127] The judgment subunit 7364 is used to compare and judge the pseudo-enhanced image of the blurred inspection image with the standard inspection image corresponding to the blurred inspection image through the judgment network, and to obtain the error parameters of the pseudo-enhanced image and the standard inspection image output by the judgment network.

[0128] The optimization subunit 7366 is used to optimize and adjust the generator network according to the error parameters, and to repeatedly input the fuzzy inspection images of the fuzzy inspection image set into the generator network until the error parameters meet the training conditions of the generator network.

[0129] In one embodiment, the training conditions for the generated network include the generation network not reaching a preset number of iterations, and the error parameter converging and being less than or equal to a preset error threshold; or the generation network reaching the preset number of iterations, and the error parameter diverging or being greater than the preset error threshold.

[0130] Each module in the aforementioned image enhancement device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module. It should be noted that the module division in this embodiment is illustrative and only represents a logical functional division; in actual implementation, other division methods may be used.

[0131] Based on the aforementioned embodiments of the image enhancement method, in another embodiment provided in this disclosure, a computer device is provided. This computer device may be a terminal, and its internal structure diagram may be as follows: Figure 12As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements an image enhancement method. The display unit is used to form a visually visible image, which can be displayed through an external display screen, projection device, or virtual reality imaging device. The display screen can be a liquid crystal display screen or an e-ink display screen, and the input device can be an image acquisition device.

[0132] Those skilled in the art will understand that Figure 12 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0133] Based on the aforementioned embodiments of the image enhancement methods, in another embodiment provided in this disclosure, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps in the above-described method embodiments.

[0134] Based on the foregoing description of the image enhancement methods, in another embodiment provided in this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0135] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0136] In the description of this specification, references to terms such as "some embodiments," "other embodiments," and "ideal embodiments" indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative descriptions of the above terms do not necessarily refer to the same embodiments or examples.

[0137] It is understood that the various embodiments of the methods described in this specification are presented in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. Relevant details can be found in the descriptions of other method embodiments.

[0138] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features of the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0139] The embodiments described above are merely illustrative of several implementations of this disclosure, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this disclosure, and these all fall within the scope of protection of this disclosure. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. An image enhancement method characterized by, The method comprises: collecting a target area inspection original image; the inspection original image is an inspection image directly collected by an inspection robot during the process of traveling along an inspection path, which needs to be processed by image enhancement, or an inspection image that has been preprocessed and still needs to be processed by image enhancement; the target area is an area that the inspection robot needs to inspect in advance; inputting the inspection original image into an image enhancement model, and obtaining a clear inspection enhanced image output by the image enhancement model; wherein the image enhancement model is obtained by training a deep learning network, the training samples of the image enhancement model include a standard inspection image set and a fuzzy inspection image set of the target area, the fuzzy inspection images in the fuzzy inspection image set are fuzzy images collected by the inspection robot that do not meet the clarity requirement, the standard inspection images in the standard inspection image set are standard images collected by the inspection robot that meet the clarity requirement, and the fuzzy inspection images in the fuzzy inspection image set have corresponding standard inspection images in the standard inspection image set; wherein the acquisition of the standard inspection image set and the fuzzy inspection image set comprises: acquiring a standard inspection video and a fuzzy inspection video collected on the inspection path of the target area; extracting standard inspection images and fuzzy inspection images corresponding to the acquisition position information according to the acquisition position information of the standard inspection video and the fuzzy inspection video, binding the standard inspection images and the fuzzy inspection images located at the same position on the inspection path, and obtaining the standard inspection image set and the fuzzy inspection image set; wherein the acquisition position information is obtained synchronously by a positioning device carried by the inspection robot when collecting the video; the frame rate of the inspection video recorded by the inspection robot during the collection of the standard inspection video and the fuzzy inspection video remains consistent; the frequency of obtaining real-time position information by the inspection robot is not lower than the frame rate of recording the inspection video, and the frequency of obtaining real-time position information by the inspection robot is an integer multiple of the frame rate of recording the inspection video.

2. The method of claim 1, wherein, The construction method of the image enhancement model comprises: training the standard inspection image set and the fuzzy inspection image set based on the deep learning network to obtain a mature image enhancement model.

3. The method of claim 2, wherein, The acquisition of the standard inspection video and the fuzzy inspection video of the target area comprises: acquiring the standard inspection video collected in the target area at a standard inspection speed; acquiring the fuzzy inspection video collected in the target area at a target inspection speed; the standard inspection speed is less than the target inspection speed.

4. The method of claim 2, wherein, The deep learning network adopts a generative adversarial network, which comprises a generation network and a judgment network, and the training of the standard inspection image set and the fuzzy inspection image set based on the deep learning network to obtain a mature image enhancement model comprises: inputting the fuzzy inspection images in the fuzzy inspection image set into the generation network to obtain pseudo-enhanced images output by the generation network; The pseudo-enhanced image of the fuzzy inspection image is compared with the standard inspection image corresponding to the fuzzy inspection image by the judgment network, and an error parameter of the pseudo-enhanced image and the standard inspection image output by the judgment network is obtained; The generation network is adjusted according to the error parameter, and the step of inputting the fuzzy inspection image of the fuzzy inspection image set into the generation network is repeated until the error parameter meets the training condition of the generation network.

5. The method of claim 4, wherein, The training condition of the generation network includes that the number of optimization of the generation network does not reach a preset iteration number, and the error parameter converges and is less than or equal to a preset error threshold; or the number of optimization of the generation network reaches the preset iteration number, and the error parameter diverges or the error parameter is greater than the preset error threshold.

6. An image enhancement device, characterized by The device comprises: An image acquisition module is configured to acquire an inspection original image of a target region; the inspection original image is an inspection image directly acquired by an inspection robot during travel along an inspection path and requiring image enhancement processing, or an inspection image that has been preprocessed and still requires image enhancement processing; the target region is a region that the inspection robot needs to inspect according to a pre-indication; An image enhancement module is configured to input the inspection original image into an image enhancement model to obtain a clear inspection enhanced image output by the image enhancement model; The image enhancement model is trained by a deep learning network, and training samples of the image enhancement model include a standard inspection image set and a fuzzy inspection image set of the target region; the fuzzy inspection image in the fuzzy inspection image set is a fuzzy image that does not meet the clarity requirement and is acquired by the inspection robot; the standard inspection image in the standard inspection image set is a standard image that meets the clarity requirement and is acquired by the inspection robot; the fuzzy inspection image in the fuzzy inspection image set has a standard inspection image with a corresponding position relationship in the standard inspection image set; the standard inspection image set and the fuzzy inspection image set are obtained by acquiring a standard inspection video and a fuzzy inspection video acquired on the inspection path of the target region; a standard inspection image and a fuzzy inspection image corresponding to the acquisition position information are extracted according to the acquisition position information of the standard inspection video and the fuzzy inspection video, the standard inspection image and the fuzzy inspection image located at the same position on the inspection path are bound to a corresponding relationship, and the standard inspection image set and the fuzzy inspection image set are obtained; the acquisition position information is synchronously obtained by a positioning device carried by the inspection robot when the video is acquired; the frame rate of the inspection video recorded by the inspection robot during acquisition of the standard inspection video and the fuzzy inspection video is consistent; the frequency of acquiring real-time position information by the inspection robot is not less than the frame rate of recording the inspection video, and the frequency of acquiring real-time position information by the inspection robot is an integer multiple of the frame rate of recording the inspection video.

7. The apparatus of claim 6, wherein, The device further comprises: A deep learning unit is configured to train the standard inspection image set and the fuzzy inspection image set based on the deep learning network to obtain a mature image enhancement model. 8.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-7. The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 5.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 5.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1 to 5.

Images