Image processing device and method for analyzing dangerous object on road

Abstract

The present specification relates to computer vision technology. A method by which an image processing device processes an image for object analysis involves: receiving an input of at least one image for a space; determining, in consideration of the image characteristics of a road surface included in the input image, whether pre-processing is required; improving image degradation of the road surface included in a set target image according to the result of determining whether the pre-processing is required; and outputting one or more images including the improved target image to an artificial intelligence model for object analysis.

Description

Image processing device and method for analyzing road hazards

[0001] This specification relates to computer vision technology, and more particularly, to a device, method, and recording medium recording the method for processing an image captured of a space in which a vehicle is driving, such as a road, before analyzing the presence of a hazardous material in the space.

[0002] On roads, there are hazards such as potholes (localized holes created by the collapse or collapse of a portion of the road surface), pavement cracks, and fallen objects. If an unexpected hazard suddenly appears while driving, it can be difficult for drivers to respond appropriately, such as slowing down or changing direction to avoid the hazard, increasing the risk of traffic accidents.

[0003] Recently, technologies have emerged that support drivers' safer driving by providing them with advance information about road hazards. Research is actively underway on technologies that collect images of road hazards and the location information of the locations where the images were captured, using vehicles equipped with information collection terminals. These images are then provided to road management agencies or shared with other drivers. Korean Patent No. 10-2147540, "Information Sharing Server for Enabling Sharing of Road Condition Information Based on Vehicle Location Information and Driving Condition Information and Operating Method Thereof," introduces a technology for sharing road condition information based on vehicle location information and driving condition information.

[0004] However, prior to image-based analysis of road hazards, even well-designed AI models can struggle to perform their intended functions due to the poor quality of the images fed into the model. For example, low-light conditions, excessive light scattering, blur, or noise in the input image can prevent the AI ​​model from properly performing functions such as object detection, tracking, recognition, or classification.

[0005] Therefore, it is necessary to improve the quality of images input to artificial intelligence models through a preprocessing process to make them more suitable for image analysis. In particular, image preprocessing can utilize a wide variety of element technologies depending on the purpose of the image processing. In this case, image processing suitable for analyzing road images must be possible depending on the purpose of recognizing specific objects, such as hazardous materials, within road images acquired from a moving vehicle. Furthermore, the presentation of technologies that match the application area or environment (road space) of the technology and the characteristics of the target image (road image) is required.

[0006] The technical problems that the embodiments of the present specification seek to solve are: resolving the problem that it is difficult to analyze road surface images using an artificial intelligence model depending on the road environment or weather conditions when acquiring real-time road images through a camera installed in a driving vehicle; overcoming the limitation that image quality improvement technologies targeting general subjects do not properly reflect the image characteristics of road surfaces; and resolving the weakness that a method of batch-correcting a large number of continuously input images actually causes inappropriate errors in road surface image analysis.

[0007] In order to solve the above technical problem, a method for processing an image for object analysis by an image processing device having at least one processor according to an embodiment of the present specification includes the steps of: receiving, by the image processing device, at least one image for a space; determining, by the image processing device, whether preprocessing is necessary in consideration of image characteristics of a road surface included in the input image; improving, by the image processing device, image deterioration of a road surface included in a set target image according to a result of the determination of whether preprocessing is necessary; and outputting, by the image processing device, one or more images including the improved target image as an artificial intelligence model for object analysis.

[0008] In an image processing method according to one embodiment, the image characteristics of the road surface may be determined from a correlation between the variables and the detection performance when the detection performance of the artificial intelligence model for the road surface itself, a hazardous material or damage on the road surface changes due to at least one or more variables among the moving speed of the image processing device, the shutter speed of the image sensor provided in the image processing device, the illuminance, and whether it is day or night.

[0009] In an image processing method according to one embodiment, the step of determining whether preprocessing is necessary may include the steps of: generating a histogram representing a distribution of each pixel value for the input image; counting the number of pixels having an illuminance value smaller than an illuminance threshold for a road surface image on the histogram; and setting the image as a target image for preprocessing when the ratio of the number of counted pixels to the total number of pixels of the image is greater than the ratio threshold.

[0010] In an image processing method according to one embodiment, the step of improving image degradation may include a step of reducing noise of a road surface included in a target image by referring to surrounding pixel values ​​for a target image set to require preprocessing; and a step of non-linearly adjusting a brightness level of a filtered target image to improve contrast of a road surface included in the target image.

[0011] Furthermore, the following provides a computer-readable recording medium having recorded thereon a program for executing the image processing method described above on a computer.

[0012] In order to solve the above technical problem, an image processing device according to one embodiment of the present specification includes a memory that stores software code for processing an image for object analysis; and a processor that executes the software code, wherein the software code includes a command for receiving at least one image for a space, determining whether preprocessing is necessary by considering the image characteristics of a road surface included in the input image, improving image deterioration of a road surface included in a set target image according to a result of the determination of whether preprocessing is necessary, and outputting one or more images including the improved target image as an artificial intelligence model for object analysis.

[0013] In an image processing device according to one embodiment, the image characteristics of the road surface may be determined from a correlation between the variables and the detection performance when the detection performance of the artificial intelligence model for the road surface itself, a hazardous material or damage on the road surface changes due to at least one or more variables among a moving speed of the image processing device, a shutter speed of an image sensor provided in the image processing device, illuminance, and whether it is day or night.

[0014] In an image processing device according to one embodiment, the software code may include a command for generating a histogram representing a distribution of each pixel value for the input image, counting the number of pixels having an illuminance value smaller than an illuminance threshold value for a road surface image on the histogram, and setting the image as a target image for preprocessing when a ratio of the number of counted pixels to the total number of pixels of the image is greater than a ratio threshold value.

[0015] In an image processing device according to one embodiment, the software code may include instructions for reducing noise of a road surface included in a target image by referring to surrounding pixel values ​​for a target image set to require preprocessing, and for non-linearly adjusting a brightness level of a filtered target image to enhance contrast of a road surface included in the target image.

[0016] Embodiments of the present specification determine whether preprocessing is necessary by considering the image characteristics of the road surface included in the image, while comprehensively considering the illuminance value of each pixel and the distribution ratio of pixels that are not suitable for analysis within the image area, thereby reducing the burden on the system by performing preprocessing on only some images among a plurality of images continuously input in real time, thereby improving the performance of road surface image analysis. In addition, when improving image deterioration to be suitable for artificial intelligence analysis, noise is reduced by preserving the boundaries of the image by referring to the surrounding pixel values, and contrast is improved by non-linearly adjusting the brightness level of the image, thereby significantly improving the analysis performance for road surface areas where the environment and weather conditions change.

[0017] Figure 1 is an exemplary diagram illustrating a situation in which a hazardous material detected in a road environment in which embodiments of the present specification are implemented is analyzed.

[0018] FIG. 2 is a diagram showing a schematic process for processing images for road hazard analysis proposed by embodiments of the present specification.

[0019] FIG. 3 is a flowchart illustrating a method for processing an image for object analysis according to one embodiment of the present specification.

[0020] Figure 4 is a flowchart illustrating in more detail the process of determining whether preprocessing is necessary in the embodiment of Figure 3 for processing an image for object analysis.

[0021] Figures 5 and 6 are diagrams illustrating comparison of histogram analysis results for road images captured during the day and at night, respectively.

[0022] FIG. 7 is a flowchart illustrating in more detail the process of improving image degradation in the embodiment of FIG. 3 for processing images for object analysis.

[0023] FIG. 8 is a drawing illustrating a process for reducing noise on a road surface in an image processing method according to one embodiment of the present specification.

[0024] FIG. 9 is a drawing for explaining a process for improving the contrast of a road surface in an image processing method according to one embodiment of the present specification.

[0025] FIG. 10 and FIG. 11 are diagrams for explaining a process of optimizing a control factor used in a preprocessing process in an image processing method according to one embodiment of the present specification.

[0026] FIG. 12 is a block diagram illustrating an image processing device for object analysis according to one embodiment of the present specification.

[0027] <Explanation of symbols>

[0028] 90: Image processing unit 10: Memory

[0029] 30: Processor 50: Image Sensor

[0030] Hereinafter, embodiments of the present specification will be described in detail with reference to the drawings. However, detailed descriptions of well-known functions or components that may obscure the gist of the embodiments in the following description and the attached drawings will be omitted. Additionally, throughout the specification, the term "including" a component does not exclude other components, unless specifically stated otherwise, but rather implies the inclusion of other components.

[0031] The terminology used herein is for the purpose of describing specific embodiments only and is not intended to be limiting of the present disclosure. The singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, it should be understood that the terms "comprise" or "have" indicate the presence of a described feature, number, step, operation, component, part, or combination thereof, but do not preclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0032] Unless specifically defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art to which this specification pertains. Terms defined in commonly used dictionaries should be interpreted to have a meaning consistent with their meaning in the context of the relevant technology, and will not be interpreted in an idealized or overly formal sense unless explicitly defined herein.

[0033] Figure 1 is an exemplary diagram illustrating a situation in which hazards detected in a road environment are analyzed in which embodiments of the present disclosure are implemented. It illustrates a situation in which a vehicle is driving on a road at night, but streetlights are partially present within the space. Furthermore, the reference vehicle is equipped with an image sensor (camera) that captures road images, thereby acquiring continuous images (e.g., video) of the vehicle's surroundings (e.g., forward).

[0034] If you want to input images acquired from a vehicle into an AI model for analysis, you should be able to detect potholes, as illustrated in the lower right of Figure 1. However, while analysis of the acquired images can be performed well when using a high-quality image sensor, in an environment with sufficient daylight or light, or when the vehicle is moving at low speeds, in the opposite case, the image quality itself is low, which increases the possibility of errors in image analysis. There are numerous factors that negatively impact the analysis of road images using computer vision. In a given environment, there is a problem with blurring due to high-speed vehicle driving, light bleed caused by lighting such as streetlights or headlights of vehicles in the opposite lane in urban areas, and problems with background blur caused by shutter speed control of the camera in low-light environments or increased noise due to sensitivity control have been discovered.

[0035] Above all, when driving at high speeds, it is inevitable to increase the shutter speed or image sensor sensitivity to secure sharpness in the background area. However, this presents problems in road environments that are different from those of conventional photography. Specifically, various objects within the image area captured by a camera typically have different levels of importance, and manipulation of various camera control factors is often designed to highlight the subject. However, in the embodiments of the present specification for monitoring road conditions, the background area (particularly the road surface) is given much greater importance than the subject, which is considered important in conventional photography. Therefore, manipulation of control factors for cameras or image sensors, which is typically employed during high-speed or nighttime driving, inadvertently complicates the analysis of road surface information. In particular, road surfaces have unique characteristics, such as color, contrast, brightness, shape, and structure, requiring a different approach than typical photographic image preprocessing.

[0036] FIG. 2 is a diagram illustrating a schematic process for processing images for road hazard analysis proposed by embodiments of the present disclosure. Road images are acquired through an image sensor (camera) mounted on a vehicle, and format conversion such as ISP, YUV, RGB, etc. can be performed. However, this can be appropriately configured depending on the implementation environment or requirements, and is not limited to the embodiments of the present disclosure. The embodiments of the present disclosure assume that a number of continuous images are input in real time, and aim to perform preprocessing (210) on some of these images and then input them into an artificial intelligence model. At this time, a standard for determining which images are subject to preprocessing (210) is required. In addition, it is necessary to determine which technical means will be used to manipulate images that meet the standard.

[0037] As previously explained, the embodiments of the present specification analyze road surface images. Therefore, the detection performance of an artificial intelligence model for the road surface itself, hazards, or damage on the road surface varies depending on at least one or more variables, including the moving speed of the image processing device, the shutter speed of the image sensor equipped with the image processing device, the level of illumination, and whether it is day or night. In this case, a correlation appears between various variables and detection performance, and it is necessary to perform preprocessing by thoroughly understanding these characteristics. The embodiments of the present specification establish criteria for determining preprocessing targets by considering these characteristics of road images, and also specify a preprocessing method to enhance the processing performance of the artificial intelligence model.

[0038] Furthermore, unlike typical single-image processing, road condition monitoring requires rapid processing of video consisting of multiple frames input in real time while a vehicle is driving on the road. For example, uniform preprocessing of approximately 30 images per second for a fast-moving vehicle in bad weather conditions or low-light / night conditions still places a burden on hardware resources. Therefore, it is necessary to quickly determine whether preprocessing is necessary for multiple images input continuously in real time, and this process can be achieved according to the judgment criteria established in the embodiments of the present specification. Referring to Fig. 2, a target image requiring preprocessing (210) is selected from among a plurality of RGB images, and then improved through the preprocessing process to be suitable for road analysis. Here, "improvement" does not mean changing the quality of the image to make it more pleasing to the eye, but rather means processing the image so that an artificial intelligence model can perform analysis on the road or road surface image.

[0039] FIG. 3 is a flowchart illustrating a method for processing an image for object analysis according to one embodiment of the present specification, wherein an image processing device having at least one processor can perform the following series of processing steps. Here, the image processing device may be a device installed in a moving object (e.g., a moving vehicle) to receive and analyze images of the surroundings (e.g., the front) of the vehicle acquired through an image sensor (e.g., a camera). However, the image processing device does not necessarily have to be physically connected to the image sensor or located in the vehicle, and it is sufficient if it is a device that is connected to the vehicle via wired or wireless communication means and can analyze road images transmitted from the vehicle in real time.

[0040] At step S310, the image processing device receives at least one image for space.

[0041] In step S330, the image processing device determines whether preprocessing is necessary by considering the image characteristics of the road surface included in the image input through step S310. Here, the image characteristics of the road surface can be determined from the correlation between the variables and the detection performance when the detection performance of the artificial intelligence model for the road surface itself, hazardous materials, or damage on the road surface changes due to at least one or more variables among the moving speed of the image processing device, the shutter speed of the image sensor provided in the image processing device, the illuminance, and whether it is day or night.

[0042] In this process, the determination of whether preprocessing is necessary is basically performed by referring to the histogram of the image. The histogram represents the distribution of each pixel value in the image and provides a basis for judgment for tasks such as adjusting the brightness or contrast of the image. Since the embodiments of the present specification analyze road images (particularly road surfaces), they have the problem of difficulty in recognizing cracks or damage within asphalt, which is composed of monotonous colors based on gray or black. In addition, road images have the weakness of uneven distribution of pixel values ​​or lack of clear contrast. Therefore, considering the image characteristics of road surfaces, the necessity of preprocessing can be determined based on how clearly the weakness of road images that makes image analysis difficult is revealed. To this end, the embodiments of the present specification determine whether preprocessing is necessary by comprehensively considering the illuminance value of each pixel and the distribution status (distribution ratio) of these unsuitable pixels within the image area. A more specific implementation process will be described later with reference to FIG. 4.

[0043] In step S350, the image processing device improves the image deterioration of the road surface included in the set target image based on the result of the judgment as to whether preprocessing is necessary. The image determined to require preprocessing in the previous step S330 is set as the target image, and only the target image is selectively preprocessed among a plurality of images continuously input in real time. In other words, it has the advantage of being able to preprocess only the absolutely necessary frames while reducing the burden on system resources considering the situation of monitoring road conditions. In particular, if all consecutive frames are corrected in batches, the deviations in each scene that appear during road driving may actually have a negative impact on the analysis when input to the artificial intelligence model. Therefore, it is very important to determine only the images unsuitable for road surface analysis among a plurality of images and selectively perform preprocessing.

[0044] Meanwhile, as explained above, when improving the image deterioration of the road surface included in the target image, 'improvement' means processing the image so that the artificial intelligence model can perform analysis well on the road or road surface image.

[0045] In step S370, the image processing device outputs one or more images, including the target image improved through step S350, to an artificial intelligence model for object analysis. Since the previous process generated images processed to be suitable for analysis by the artificial intelligence model, both the processed images suitable for analysis and the original images suitable for analysis are provided as analysis targets.

[0046] Through the above process, it can be seen that the preprocessing process proposed by the embodiments of the present specification is to detect target images that are unsuitable for the analysis task of the artificial intelligence model. In addition, it can be seen that the improvement of image deterioration performed by performing the preprocessing process is also to process the target image so that it is suitable for the analysis task of the artificial intelligence model. Therefore, it can be seen that the control factors involved in the target and execution of the preprocessing process are all values ​​set to be suitable for the artificial intelligence model to be used later. In particular, being suitable for the artificial intelligence model means whether the artificial intelligence model can perform the analysis of the road image well and accurately recognize the hazards or road surface damage contained in the image. To this end, the success rate or accuracy of any one of detection, tracking, recognition, or classification of an object using the artificial intelligence model can be set as a judgment index for optimizing the control factors for the analysis of the artificial intelligence model.

[0047] FIG. 4 is a flowchart illustrating in more detail the process (step S330) of determining whether preprocessing is necessary in the embodiment of FIG. 3 for processing an image for object analysis.

[0048] In step S331, a histogram representing the distribution of each pixel value for the input image is generated.

[0049] In step S333, the number of pixels having an illuminance value lower than an illuminance threshold for a road surface image is counted on the histogram. If a pixel has an illuminance value lower than a preset illuminance threshold on the histogram, it means that the pixel is not suitable for road surface image analysis, and the number of pixels below the threshold is calculated to determine how distributed such pixels are within the entire image area.

[0050] In step S335, if the ratio of the number of counted pixels to the total number of pixels in the image is greater than a ratio threshold, the image can be set as a target image for preprocessing. A greater number of pixels below the threshold indicates that the image is unsuitable for road surface image analysis using an artificial intelligence model, and therefore, the ratio of unsuitable pixels needs to be calculated. If the calculated ratio of unsuitable pixels is greater than a preset ratio threshold, the image can be determined as a target image requiring preprocessing.

[0051] Here, the illuminance threshold value and the ratio threshold value can be set so that the detection performance of the artificial intelligence model for a hazardous material object included in a road surface image is greater than a specific performance standard value.

[0052] Figures 5 and 6 are diagrams illustrating comparison of histogram analysis results for road images captured during the day and at night, respectively.

[0053] Referring to Figure 5, the daytime image demonstrates sufficient light, making the captured image quality suitable for analysis using an AI model even at high speeds or with increased camera shutter speeds. Specifically, the histogram illustrated in Figure 5 demonstrates a wide range of brightness values, a uniform distribution of pixel intensities, high mid-tone intensities, and clear image contrast.

[0054] In contrast, referring to Fig. 6, which shows a night image or a low-light image, it can be seen that the brightness distribution is uneven, there is insufficient contrast for the road surface area, and there is a partial adverse effect due to light reflection.

[0055] Therefore, the criteria for determining the image to be preprocessed can be confirmed through the difference in these histograms. When determining the preprocessing target, if the determination is made simply using the average of the pixel values ​​of all pixels, the determination result may be unsuitable for the analysis of road surface images. On the other hand, the embodiments of the present specification comprehensively consider the luminance value of individual pixels and the distribution status (distribution ratio) of these unsuitable pixels within the image area to determine whether preprocessing is necessary. The method for determining whether preprocessing is necessary proposed by the embodiments of the present specification enables robust determination by responding relatively well to uneven brightness distributions when compared to a low-light classification method that simply uses the average of the pixel values ​​of all pixels. For example, even in an image with an intermediate average brightness, if pixels are concentrated in a specific range of brightness values, the actual visual experience may be perceived as low-light. The method for determining whether preprocessing is necessary proposed by the embodiments of the present specification can selectively adjust the brightness threshold considered as low-light and the ratio of low-light pixels to all pixels. This adjustment method allows for more precise control over different lighting conditions and environments, increasing the flexibility and accuracy of low-light judgment.

[0056] FIG. 7 is a flowchart illustrating in more detail the process of improving image degradation (step S350) in the embodiment of FIG. 3 for processing images for object analysis.

[0057] In step S351, for a target image that is set to require preprocessing, the noise of the road surface included in the target image is reduced by referring to the surrounding pixel values. In this process, filtering may be performed by sorting the pixel values ​​of the surrounding pixels by the filter size of each pixel in the target image, and selecting the value located at the center of the sorted pixel values ​​to replace the corresponding pixel value. Here, the filter size means the distance from one pixel to the surrounding pixels to be compared together, and the larger the filter size, the greater the number of surrounding pixels to be compared together. However, an excessively small filter size has a small noise reduction effect, while an excessively large filter size may lose the unique morphological characteristics of the image. Therefore, it is preferable to set an appropriate value in consideration of the morphological and structural characteristics of the target image.

[0058] FIG. 8 is a diagram illustrating a process for reducing road surface noise (step S351 of FIG. 7) in an image processing method according to one embodiment of the present specification. Referring to FIG. 8, noise reduction is performed on each pixel of a target image (810), and a process is shown for changing the pixel value of a specific pixel (the central pixel) by referring to the pixel values ​​of the surrounding pixels based on the current specific pixel. It is illustrated that the pixel values ​​of the surrounding pixels of the central pixel (pixel value '102') of the target image (810) are brought in and aligned, and then the median value (pixel value '72') is selected and replaced with the pixel value, thereby outputting the resulting image (820).

[0059] In addition to the noise reduction method proposed in one embodiment of this specification, various noise reduction filtering methods exist. However, in the road condition monitoring environment where this embodiment is implemented, it is necessary to preserve the boundaries of road surface images. Therefore, the proposed noise reduction method using median values ​​was confirmed to be more suitable for object (hazardous material or damage) detection in road surface images than other methods (e.g., Gaussian Blur).

[0060] Returning to FIG. 7, in step S353, the brightness level of the target image filtered through step S351 is nonlinearly adjusted to enhance the contrast of the road surface included in the target image. In this process, the ratio of each pixel value to the maximum range of pixel values ​​is calculated for each pixel of the target image, and the value obtained by exponentiating the calculated ratio by a gamma correction constant and multiplying the result by the maximum range of pixel values ​​is output as the pixel value. This nonlinear adjustment of the brightness level can be implemented using the following mathematical formula.

[0061]

[0062] Here, I is the input pixel value of the original image, O is the output pixel value after gamma correction, and γ(gamma) is a gamma correction constant used to adjust the contrast of the image. The value 255 set in the numerator and denominator represents the maximum range of image pixel values, and is an example of a value commonly used in 8-bit images. Therefore, it may be set to a different value depending on the characteristics of the image.

[0063] FIG. 9 is a diagram for explaining a process for enhancing the contrast of a road surface (step S353 of FIG. 7) in an image processing method according to one embodiment of the present disclosure, showing a method for non-linearly adjusting the brightness level of a pixel using a gamma value (γ). When the gamma value is less than '1', the brightness of dark areas increases relatively more, brightening the entire image and making details stand out better. Conversely, when the gamma value is greater than '1', the brightness of bright areas is relatively emphasized and dark areas are darkened more, strengthening the contrast of the image. This image adjustment method is particularly useful for images with low contrast in low-light environments, which are the target of the embodiments of the present disclosure.

[0064] Meanwhile, the size of the filter for reducing noise on the road surface and the gamma correction constant for enhancing contrast on the road surface can be set so that the detection performance of the artificial intelligence model for hazardous objects included in the road surface image is higher than a specific performance standard value.

[0065] FIG. 10 and FIG. 11 are diagrams for explaining a process of optimizing a control factor used in a preprocessing process in an image processing method according to one embodiment of the present specification.

[0066] Above, four control factors involved in the image preprocessing process have been presented through the embodiments of this specification. Each factor's name has been arbitrarily assigned for convenience, and the meaning of each control factor is summarized and introduced below.

[0067] - Alpha (α, low_light_threshold): If the number of pixels below a specific luminance threshold (α) in the histogram exceeds a given ratio (β), the image is classified as low-light. This allows for more robust low-light identification than traditional mean-based classification methods, as it takes into account the actual brightness distribution of each image.

[0068] - Beta (β, low_light_ratio): Indicates the ratio of low-light pixels to all pixels, which enables fine adjustments for various lighting conditions and environments, thereby improving the flexibility and accuracy of low-light image identification.

[0069] - Mu (μ, mesh_size): The size of the filter used in the noise reduction filter. It affects the nonlinear filtering technique that removes noise and preserves the boundaries of the image by selecting the median value based on the surrounding pixel values.

[0070] - Gamma (γ, gamma_correction): Nonlinearly adjusts the brightness level of each pixel to increase the brightness of dark areas relatively according to the gamma value, or to emphasize the brightness of bright areas, thereby making details of the entire image more visible or enhancing contrast.

[0071] These control factors can be adjusted to maximize the performance of an AI model for object detection, for example, by utilizing optimization algorithms such as genetic algorithms or gradient descent, thereby maximizing the detection performance of road hazards. The optimization process is repeatedly performed through a large number of sample images, which improves the generalization ability of the model and ensures the reliability of hazard detection in actual driving environments. The image processing technology proposed in the embodiments of this specification focuses on designing a pipeline for image preprocessing in low-light environments and optimizing control factors, thereby aiming to significantly improve the performance of road hazard detection in nighttime and low-light environments.

[0072] Referring to Fig. 10, which shows a process of optimizing control factors of a preprocessing step using a genetic algorithm, a plurality of test images are provided and an improved image is output through a preprocessing (1010) step. At this time, a result image can be generated through a combination of various control factors, and as many combinations as the number of genetic algorithm objects N (N is a natural number) can be generated. The result images generated for each combination of control factors are each input into an artificial intelligence model (1020) to evaluate the accuracy of detecting hazards on the target road surface. Here, the top N / 2 objects with high accuracy among the evaluation results can be selected as excellent objects for the next generation.

[0073] Referring to Figure 11, which illustrates a series of steps for optimizing control factors in the preprocessing process using a genetic algorithm, a dataset of images captured and labeled under various road conditions is first input into a preprocessing pipeline. This pipeline has control factors of alpha, beta, mu, and gamma, and each object is assigned a unique combination of control factor values. The preprocessed image dataset is input into a hazard detection AI model, and the accuracy of each object is analyzed based on the inferred results. Among the evaluation results for multiple objects, the top N / 2 objects with relatively high accuracy can be selected.

[0074] Then, based on the selected objects, new combinations of control factors can be generated through crossover and mutation operations. These new combinations of control factors can be fed back into the preprocessing pipeline, and a series of performance evaluations can be repeated until a preset termination condition is reached. This process aims to observe how each combination of control factors changes the performance of the AI ​​model for hazard detection in image data obtained from real-world road conditions, and to continuously adjust the control factors based on the changing results. This genetic algorithm-based control factor optimization method can enable the AI ​​model to achieve more robust detection capabilities across a variety of road conditions.

[0075] FIG. 12 is a block diagram illustrating an image processing device for object analysis according to one embodiment of the present specification, which reconstructs the image processing method of FIG. 3 from the perspective of hardware configuration. Therefore, to avoid redundant explanation, only an outline of the operation and function of each component is briefly described herein.

[0076] An image processing device (90) includes a memory (10) that stores software code for processing images for object analysis, and a processor (30) that executes the software code. At this time, the software code includes a command for receiving at least one image for a space, determining whether preprocessing is necessary by considering the image characteristics of a road surface included in the input image, improving image deterioration of a road surface included in a set target image according to a result of determining whether preprocessing is necessary, and outputting one or more images including the improved target image as an artificial intelligence model for object analysis. Here, the image characteristics of the road surface can be determined from a correlation between the variables and the detection performance when the detection performance of the artificial intelligence model for the road surface itself, a hazard object, or damage on the road surface changes due to at least one variable among a moving speed of the image processing device, a shutter speed of an image sensor provided in the image processing device, illuminance, and whether it is day or night.

[0077] Meanwhile, the image sensor (50) does not have to be included in the image processing device (90), but it is natural that the image acquired through the image sensor (50) should be provided as input. In addition, the image sensor (50) may be configured as an integral assembly with the image processing device (90) depending on implementation needs.

[0078] In addition, the software code may include a command to generate a histogram representing the distribution of each pixel value for the input image, count the number of pixels having an illuminance value smaller than an illuminance threshold value for a road surface image on the histogram, and set the image as a target image for preprocessing when the ratio of the number of counted pixels to the total number of pixels of the image is greater than the ratio threshold value.

[0079] Furthermore, the software code may include instructions for reducing noise of a road surface included in a target image by referring to surrounding pixel values ​​for a target image set to require preprocessing, and for non-linearly adjusting a brightness level of the filtered target image to enhance contrast of the road surface included in the target image.

[0080] Embodiments according to the present specification may be implemented by various means, for example, hardware, firmware, software, or a combination thereof. In the case of hardware implementation, an embodiment of the present specification may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, etc. In the case of firmware or software implementation, an embodiment of the present specification may be implemented in the form of a module, procedure, function, etc. that performs the capabilities or operations described above. Software code may be stored in a memory and executed by a processor. The memory may be located inside or outside the processor and may exchange data with the processor by various means already known in the art.

[0081] Meanwhile, the embodiments of the present specification can be implemented as computer-readable codes on a computer-readable recording medium. Computer-readable recording media include all types of recording devices that store data that can be read by a computer system. Examples of computer-readable recording media include ROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, etc. In addition, the computer-readable recording media can be distributed across network-connected computer systems, so that the computer-readable codes can be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the embodiments can be easily inferred by programmers in the technical field to which the present specification pertains.

[0082] One or more non-transitory computer-readable media storing one or more instructions according to one embodiment, the one or more instructions being executable by one or more processors, wherein the one or more instructions process an image for object analysis, wherein at least one image for a space is input, and whether pre-processing is necessary in consideration of image characteristics of a road surface included in the input image is determined, and according to a result of the determination of whether pre-processing is necessary, image deterioration of a road surface included in a set target image is improved, and one or more images including the improved target image are output as an artificial intelligence model for object analysis.

[0083] The present disclosure has been described above, focusing on various embodiments thereof. Those skilled in the art will appreciate that various embodiments may be modified without departing from the essential characteristics of the present disclosure. Therefore, the disclosed embodiments should be considered illustrative rather than restrictive. The scope of the present disclosure is set forth in the claims, not the foregoing description, and all differences within the scope equivalent thereto should be construed as being encompassed by the present disclosure.

[0084] According to the embodiments of the present specification described above, by considering the image characteristics of the road surface included in the image, and comprehensively considering the illuminance value of each pixel and the distribution ratio of pixels unsuitable for analysis within the image area, it is possible to determine whether preprocessing is necessary, thereby reducing the system burden and improving the performance of road surface image analysis by performing preprocessing on only some images among a plurality of images continuously input in real time. In addition, in improving image deterioration to be suitable for artificial intelligence analysis, noise is reduced by preserving the boundaries of the image by referring to the surrounding pixel values, and contrast is improved by non-linearly adjusting the brightness level of the image, thereby significantly improving the analysis performance for road surface areas where the environment and weather conditions change.

Claims

1. A method for processing an image for object analysis by an image processing device having at least one processor, A step in which an image processing device receives at least one image for a space; A step of determining whether pre-processing is necessary by considering the image characteristics of the road surface included in the input image by the image processing device; A step of improving image deterioration of a road surface included in a target image set according to the result of the judgment of whether preprocessing is necessary by the image processing device; and An image processing method comprising a step of outputting one or more images including an improved target image to an artificial intelligence model for object analysis by the image processing device.

2. In paragraph 1, The image characteristics of the above road surface are: An image processing method, wherein the detection performance of the artificial intelligence model for the road surface itself, hazardous materials or damage on the road surface changes depending on at least one variable among the moving speed of the image processing device, the shutter speed of the image sensor provided in the image processing device, the illuminance, and whether it is day or night, and the detection performance is determined from the correlation between the variable and the detection performance.

3. In paragraph 1, The step of determining whether the above preprocessing is necessary is: A step of generating a histogram representing the distribution of each pixel value for the input image; A step of counting the number of pixels having an illuminance value less than an illuminance threshold for a road surface image on the histogram; and An image processing method comprising the step of setting the image as a target image for preprocessing when the ratio of the number of counted pixels to the total number of pixels of the image is greater than a ratio threshold value.

4. In paragraph 3, An image processing method, wherein the above illuminance threshold value and the above ratio threshold value are each set so that the detection performance of the artificial intelligence model for a hazardous material object included in a road surface image is equal to or higher than a predetermined performance standard value.

5. In paragraph 1, The steps for improving the above image degradation are: A step of reducing noise of a road surface included in a target image by referring to surrounding pixel values ​​for a target image set to require preprocessing; and An image processing method comprising a step of non-linearly adjusting a brightness level of a target image to enhance the contrast of a road surface included in the target image.

6. In paragraph 5, The steps for reducing the noise of the above road surface are: An image processing method for performing filtering by aligning pixel values ​​of surrounding pixels by a predetermined filter size for each pixel in the target image, selecting a value located at the center of the aligned pixel values, and replacing the corresponding pixel value.

7. In paragraph 5, The step of improving the contrast of the above road surface is: An image processing method that calculates the ratio of each pixel value to the maximum range of pixel values ​​for each pixel of the target image, exponentiates the calculated ratio by a gamma correction constant, and then outputs the value obtained by multiplying the maximum range of pixel values ​​as a pixel value.

8. In paragraph 5, An image processing method, wherein the size of a filter for reducing noise on a road surface and the gamma correction constant for enhancing contrast on a road surface are each set so that the detection performance of the artificial intelligence model for a hazardous material object included in a road surface image is higher than a predetermined performance standard value.

9. In one or more non-transitory computer-readable media storing one or more instructions, The one or more instructions executable by one or more processors process an image for object analysis, At least one image for the space is input, Considering the image characteristics of the road surface included in the input image, determine whether pre-processing is necessary, Improves the image deterioration of the road surface included in the target image set according to the result of determining whether preprocessing is necessary. A computer-readable medium that outputs one or more images, including an improved target image, to an artificial intelligence model for object analysis.

10. Memory for storing software code for processing images for object analysis; and comprising a processor for executing the above software code, The above software code, An image processing device comprising a command for receiving at least one image for a space, determining whether preprocessing is necessary by considering the image characteristics of a road surface included in the input image, improving image deterioration of a road surface included in a set target image based on the result of determining whether preprocessing is necessary, and outputting one or more images including the improved target image as an artificial intelligence model for object analysis.

11. In paragraph 10, The image characteristics of the above road surface are: An image processing device, wherein the detection performance of the artificial intelligence model for the road surface itself, hazardous materials or damage on the road surface changes depending on at least one variable among the moving speed of the image processing device, the shutter speed of the image sensor provided in the image processing device, the illuminance, and whether it is day or night, and is determined from the correlation between the variable and the detection performance.

12. In paragraph 10, The above software code, Generate a histogram representing the distribution of each pixel value for the input image, Counting the number of pixels having an illuminance value less than an illuminance threshold for a road surface image on the above histogram, An image processing device comprising a command for setting the image as a target image for preprocessing when the ratio of the number of counted pixels to the total number of pixels of the image is greater than a ratio threshold value.

13. In paragraph 10, The above software code, For a target image that is set to require preprocessing, noise of the road surface included in the target image is reduced by referring to the surrounding pixel values, An image processing device comprising a command for non-linearly adjusting a brightness level of a target image to enhance the contrast of a road surface included in the target image.

Images