Image enhancement method and computer program product

Abstract

The application provides an image enhancement method and a computer program product, and relates to the technical field of image processing. The method comprises the following steps: acquiring at least one target gradient image; the target gradient image is an image used for representing the depth variation rate of a target object in a set direction; constructing an image pyramid about the target object based on the at least one target gradient image; setting a corresponding scale weight for each pyramid image layer according to a set method; and performing image pyramid restoration based on each size gradient image and the corresponding scale weight to obtain a reconstructed gradient image. The application solves the problem that the depth information is ignored in the image processing process of a three-dimensional image in the related art.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically, to an image enhancement method and a computer program product. Background Technology

[0002] With the rapid development of modern industrial manufacturing technology, especially in mold design, precision machining, and industrial measurement, the requirements for the dimensionality and accuracy of product inspection are increasing. Traditional two-dimensional image sensors are limited by factors such as lighting conditions, object surface texture, and shooting posture, often making it difficult to acquire complete spatial geometric information of objects, and are gradually failing to meet the inspection needs in complex production scenarios. Therefore, three-dimensional and 2.5D measurement technologies, with their advantages of directly acquiring object surface depth information and being insensitive to changes in ambient lighting, have been widely used in industrial automation inspection in recent years.

[0003] In 3D measurement, the raw surface information of an object is typically acquired in the form of point clouds or depth images. While point cloud data can fully represent the 3D surface structure of an object, its unstructured nature leads to a huge amount of preprocessing computation, making it difficult to adapt to all operating environments with high real-time requirements. In contrast, depth images map 3D depth information into a 2D pixel matrix, greatly reducing computational resource consumption, and therefore have become a more common data carrier in industrial inspection lines.

[0004] However, most depth image preprocessing techniques directly adopt traditional two-dimensional image processing algorithms, treating pixel values ​​in depth images merely as ordinary grayscale and brightness values. This approach ignores the three-dimensional geometric properties inherent in the pixels of a depth image and fails to fully utilize the physically meaningful information of depth gradients. Summary of the Invention

[0005] This application provides an image enhancement method, apparatus, electronic device, and storage medium, which can solve the problem of ignoring depth information during the image processing of three-dimensional images in related technologies. The technical solutions are as follows:

[0006] According to one aspect of this application, an image enhancement method includes: acquiring at least one target gradient image; the target gradient image is an image used to characterize the depth change rate of a target object in a predetermined direction; constructing an image pyramid about the target object based on the at least one target gradient image; the image pyramid includes multiple pyramid image layers of various scales, each pyramid image layer including the same number of size gradient images as the target gradient images, the size gradient images of the pyramid image layers being used to characterize the depth change rate of the target object in the predetermined direction at a corresponding scale, the size gradient images of different pyramid image layers corresponding to different scales; assigning a corresponding scale weight to each pyramid image layer according to a setting method; the scale weight being used to indicate the proximity of the scale corresponding to the size gradient image of the pyramid image layer to the target pyramid image layer where the target object is located; and performing image pyramid reconstruction based on each size gradient image and its corresponding scale weight to obtain a reconstructed gradient image.

[0007] According to one aspect of this application, an electronic device includes at least one processor and at least one memory, wherein the memory stores a computer program that, when executed by the processor, implements the image enhancement method as described above.

[0008] According to one aspect of this application, a storage medium having a computer program stored thereon, which, when executed by one or more processors, implements the image enhancement method as described above.

[0009] According to one aspect of this application, a computer program product includes a computer program that, when executed by one or more processors, implements the image enhancement method as described above.

[0010] The image enhancement method according to the above embodiments constructs an image pyramid containing multiple scales based on the target gradient image and assigns a scale weight to each pyramid image layer that reflects its proximity to the target pyramid image layer where the target object is located, thereby achieving multi-scale adaptive enhancement of the depth change features of the target object. Specifically, during the image pyramid restoration process, the scale weights are used to weight the size gradient images of each pyramid image layer, which enables the final reconstructed gradient image to focus on image information that matches the true scale of the target object and effectively suppress interference signals from unrelated pyramid image layers that are far from the true scale of the target object. This ensures that the reconstructed gradient image can accurately represent the depth change rate of the target object in a set direction and that matches the target pyramid image layer where the target object is located with the optimal signal-to-noise ratio. Attached Figure Description

[0011] Figure 1This is a schematic diagram based on the implementation environment involved in this application;

[0012] Figure 2 This is a hardware structure diagram of an electronic device according to an exemplary embodiment;

[0013] Figure 3 This is a flowchart illustrating an image enhancement method according to an exemplary embodiment;

[0014] Figure 4 yes Figure 3 A flowchart of step 330 in one embodiment corresponds to the following example;

[0015] Figure 5 yes Figure 3 A flowchart of step 370 in one embodiment corresponds to the following example;

[0016] Figure 6 yes Figure 3 A flowchart of the steps preceding step 310 in the corresponding embodiment;

[0017] Figure 7 yes Figure 3 A flowchart of step 350 in one embodiment corresponds to the following example;

[0018] Figure 8 yes Figure 7 A schematic diagram illustrating the specific implementation of a detection box for a target object in the corresponding embodiment;

[0019] Figures 9 to 11 This is a schematic diagram illustrating the specific implementation of an image enhancement method in an application scenario. Detailed Implementation

[0020] The present invention will now be described in further detail with reference to specific embodiments and accompanying drawings. Similar elements in different embodiments are referred to by associated similar element reference numerals. In the following embodiments, many details are described to facilitate a better understanding of this application. However, those skilled in the art will readily recognize that some features may be omitted in different situations, or may be replaced by other elements, materials, or methods. In some cases, certain operations related to this application are not shown or described in the specification. This is to avoid obscuring the core parts of this application with excessive description. For those skilled in the art, detailed description of these related operations is not necessary; they can fully understand the related operations based on the description in the specification and general technical knowledge in the art.

[0021] Furthermore, the features, operations, or characteristics described in the specification can be combined in any suitable manner to form various embodiments. At the same time, the steps or actions in the method description can be rearranged or adjusted in a manner obvious to those skilled in the art. Therefore, the various orders in the specification and drawings are only for the clear description of a particular embodiment and do not imply a necessary order, unless otherwise stated that a particular order must be followed.

[0022] The serial numbers assigned to components in this document, such as "first" and "second," are used only to distinguish the described objects and have no sequential or technical meaning. The terms "connection" and "linkage" used in this application, unless otherwise specified, include both direct and indirect connections (linkages).

[0023] As mentioned earlier, in 3D detection or measurement technology, the preprocessing of depth images usually directly adopts traditional 2D image processing algorithms.

[0024] The above processing method ignores the depth information in the 3D image, cannot effectively enhance the geometric features of the target object in a specific 3D direction, and cannot perform adaptive feature extraction based on the actual physical scale of the target object. As a result, the subsequent detection task is easily affected by background texture noise or non-target scale features, which reduces the accuracy and robustness of 3D detection.

[0025] As can be seen from the above, the related technologies still have the defect of ignoring depth information during the image processing of 3D images.

[0026] Therefore, the image enhancement method provided in this application can effectively emphasize the depth information of three-dimensional graphics during image processing. Accordingly, the image enhancement method is applicable to an image enhancement device, which can be deployed in an electronic device. The electronic device can be a computer device configured with a von Neumann architecture, such as a desktop computer, a laptop computer, a server, etc.

[0027] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0028] Figure 1 This is a schematic diagram of an implementation environment involved in an image enhancement method. It should be noted that this implementation environment is merely an example adapted to the present invention and should not be considered as providing any limitation on the scope of the invention.

[0029] The implementation environment includes a data acquisition terminal 110 and a server terminal 130.

[0030] Specifically, the acquisition terminal 110 can also be considered an image acquisition device, including but not limited to two-dimensional image sensors and three-dimensional vision sensors. It can capture multiple original images of the same object by illuminating it with light sources from different directions. Furthermore, it can use algorithms such as photometric stereo and phase deflection to fuse multiple original images to obtain the object's shape information, such as reflectance images, diffuse reflection images, and normal vector images. For example, the acquisition terminal 110 is a photometric stereo acquisition system.

[0031] Server 130 can be an electronic device such as a desktop computer, laptop computer, or server, or it can be a computer cluster consisting of multiple servers, or even a cloud computing center consisting of multiple servers. Server 130 is used to provide backend services, such as, but not limited to, image enhancement services.

[0032] The server 130 and the acquisition terminal 110 establish a network communication connection in advance via wired or wireless means, and data transmission between the server 130 and the acquisition terminal 110 is realized through this network communication connection. The transmitted data includes, but is not limited to, raw images, etc.

[0033] In one application scenario, through the interaction between the acquisition terminal 110 and the server 130, the acquisition terminal 110 captures and obtains the original image of the target object, and uploads the original image to the server 130 to request the server 130 to provide image enhancement services.

[0034] For server 130, after receiving the original image of the target object uploaded by acquisition terminal 110, it calls the image enhancement service to enhance the original image, generate at least one target gradient image, construct an image pyramid about the target object based on the at least one target gradient image, and set corresponding scale weights for each pyramid image layer according to a set method; and perform image pyramid reconstruction based on each size gradient image and the corresponding scale weights to obtain the reconstructed gradient image. The reconstructed gradient image is the image with image enhancement completed, thereby solving the problem of ignoring depth information in the image processing of three-dimensional images in related technologies.

[0035] Please see Figure 2 , Figure 2 This is a hardware structure diagram of an electronic device according to an exemplary embodiment. This electronic device is suitable for... Figure 1 The server shown is 130.

[0036] It should be noted that this electronic device is merely an example adapted to this application and should not be construed as providing any limitation on the scope of use of this application. Furthermore, this electronic device should not be interpreted as requiring or depending on any specific feature. Figure 2One or more components of the exemplary electronic device 200 shown.

[0037] The hardware structure of electronic device 200 can vary significantly due to differences in configuration or performance, such as... Figure 2 As shown, the electronic device 200 includes: a power supply 210, an interface 230, at least one memory 250, and at least one central processing unit (CPU) 270.

[0038] Specifically, power supply 210 is used to provide operating voltage for various hardware devices on electronic device 200.

[0039] Interface 230 includes at least one wired or wireless network interface 231 for interacting with external devices. For example, to perform... Figure 1 The diagram shows the interaction between the acquisition terminal 110 and the server terminal 130 in the implementation environment.

[0040] Of course, in other examples adapted in this application, interface 230 may further include at least one serial-to-parallel conversion interface 233, at least one input / output interface 235, and at least one USB interface 237, etc. Figure 2 As shown, this does not constitute a specific limitation.

[0041] The memory 250 serves as a carrier for resource storage and can be a read-only memory, random access memory, disk, or optical disk, etc. The resources stored on it include the operating system 251, application programs 253, and data 255, etc., and the storage method can be temporary storage or permanent storage.

[0042] The operating system 251 is used to manage and control the various hardware devices and application programs 253 on the electronic device 200, so as to enable the central processing unit 270 to perform calculations and processing on the massive data 255 in the memory 250. It can be Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0043] Application 253 is a computer program formed by computer-readable instructions based on operating system 251 to perform at least one specific task, and may include at least one module ( Figure 2 (Not shown), each module can contain corresponding computer-readable instructions. For example, the image enhancement device can be considered as an application 253 deployed on electronic device 200.

[0044] Data 255 can be photos, pictures, etc. stored on a disk, or it can be original images, target gradient images, image pyramids, reconstructed gradient images, etc., stored in memory 250.

[0045] The central processing unit 270 may include one or more processors and is configured to communicate with the memory 250 via at least one communication bus to read computer programs stored in the memory 250, thereby performing operations and processing on massive amounts of data 255 stored in the memory 250. For example, an image enhancement method may be implemented by the central processing unit 270 reading an application program 253 stored in the memory 250.

[0046] Furthermore, this application can also be implemented through hardware circuits or a combination of hardware circuits and software. Therefore, the implementation of this application is not limited to any specific hardware circuit, software, or combination thereof.

[0047] Please see Figure 3 This application provides an image enhancement method applicable to electronic devices, such as electronic devices that can be... Figure 1 The server 130 in the implementation environment is shown. The hardware structure of this electronic device can be as follows: Figure 2 As shown.

[0048] In the following method embodiments, for ease of description, the execution subject of each step of the method is an electronic device, but this does not constitute a specific limitation.

[0049] like Figure 3 As shown, the method may include the following steps:

[0050] Step 310: Obtain at least one target gradient image.

[0051] The target gradient image is an image used to characterize the rate of change of depth of the target object in a set direction; the target object refers to any object, such as an object to be inspected on an industrial production line, a smooth surface object with certain unevenness, a mobile phone, etc.

[0052] The depth change rate refers to the degree to which the depth value of a target object in three-dimensional space changes as the pixel coordinate position on the image plane changes. It can be understood that because the target object has a specific geometric shape (such as surface unevenness) and spatial orientation (such as tilt angle) in three-dimensional space, the depth of each point on its surface exhibits a non-uniform distribution. Therefore, when the target object is mapped onto a two-dimensional image plane, the depth values ​​corresponding to different pixel coordinate positions differ.

[0053] For example, in areas where the surface of the target object is flat and parallel to the image plane, the difference in depth values ​​between adjacent pixels is small, and the rate of depth change is low; while in areas where the surface of the target object is steep, the difference in depth values ​​between adjacent pixels is significant, and the rate of depth change is high.

[0054] Furthermore, the setting direction can refer to the gradient projection direction selected when quantifying the drastic change in the depth of the target object's surface. Specifically, the depth change rate characterizes how quickly the depth value changes in the image plane, while the setting direction determines on which reference direction (such as the X-axis) the change is measured.

[0055] For example, when the direction is set to the X-axis, the depth change rate means how quickly the depth value of a target object changes along the X-axis in three-dimensional space.

[0056] It is understandable that the setting direction can be determined based on the degree of change in the surface depth of the target object. In other words, the direction with the greatest change in surface depth of the target object can be selected as the setting direction, so as to map the geometric structure information in that setting direction.

[0057] Therefore, the target gradient image can reflect the rate of depth change of the target object in a set direction, thus highlighting the area where the depth change of the target object surface is most dramatic.

[0058] Step 330: Construct an image pyramid about the target object based on at least one target gradient image.

[0059] The image pyramid consists of multiple pyramid image layers at various scales. Each pyramid image layer includes the same number of size gradient images as the target gradient images. The size gradient images of the pyramid image layers are used to characterize the rate of depth change of the target object in a set direction at the corresponding scale. The scales corresponding to the size gradient images of different pyramid image layers are different.

[0060] First, it should be noted that an image pyramid is a collection of target gradient images at different scales. By constructing an image pyramid, it is possible to capture the macroscopic geometric structure (such as the overall outline) of the target object at a high level (low scale) while preserving the microscopic texture details (such as edge sharpness) of the target object at a low level (high scale).

[0061] Therefore, by using gradient images of various sizes in the image pyramid, we can emphasize the information of the target object at different scales, that is, the target object at different scales represents the rate of change of depth of the target object in a set direction.

[0062] Regarding the construction of the target pyramid, in one embodiment, such as Figure 4 As shown, step 330 may further include the following steps: using the target gradient image as the size gradient image corresponding to the lowest layer of the image pyramid.

[0063] Starting from the bottom layer and ending at the top layer of the image pyramid, for any pyramid image layer in the image pyramid:

[0064] Step 331: Filter and downsample the size gradient image of the current layer to obtain the size gradient image of the previous layer.

[0065] First of all, it should be noted that the target pyramid can be constructed in various ways, such as a Gaussian pyramid, a mean pyramid, or a Laplace pyramid, etc., and no specific method is specified here.

[0066] The number of pyramid layers in the image pyramid conforms to a set number of layers. The set number of layers can be related to the original size and scale of the target gradient image. For example, for a target gradient image with a size of 4096×3072, a set number of layers of 10 can cover most of the scale of the target gradient image.

[0067] Further, in one embodiment, step 331 may also include the following steps: filtering the size gradient image corresponding to the current layer using a target filtering kernel to obtain a filtered size gradient image corresponding to the current layer. Then, downsampling the filtered size gradient image corresponding to the current layer according to a set ratio to obtain the size gradient image corresponding to the layer above the current layer.

[0068] Regarding filtering, noise contained in the size gradient image of the current layer can be filtered out to prevent aliasing during subsequent downsampling.

[0069] In one embodiment, a Gaussian filter kernel can be used to filter the size gradient image of the current layer, or a mean filter kernel can be used to filter the size gradient image of the current layer; no specific limitation is made here.

[0070] Downsampling refers to the process of reducing the image size by a set ratio. This set ratio can refer to the scaling factor of the size gradient images between two adjacent pyramid image layers in the image pyramid. The set ratio can be set according to accuracy requirements. Generally speaking, the larger the set ratio (closer to 1), the larger the multi-scale coverage of the image pyramid and the higher the accuracy.

[0071] For example, if the ratio can be set to 1 / 2, then downsampling can start from the original size of the current layer's size gradient image and reduce it. Each reduction makes the length and width of the size gradient image corresponding to the previous layer become 1 / 2 of the size gradient image of the current layer, and the number of image pixels is reduced to 1 / 4 of the size gradient image of the current layer.

[0072] It should be noted that the number of layers is affected by the set ratio. For target gradient images of the same size, a larger set ratio (e.g., 0.9, meaning 90% of the current layer's gradient image size is retained in each downsampling) results in slower gradient image size decay, and thus a larger number of layers. Conversely, a smaller set ratio (e.g., 0.1, meaning 10% of the current layer's gradient image size is retained in each downsampling) results in faster gradient image size decay, and thus a smaller number of layers.

[0073] In addition, the setting ratio can be adjusted according to actual needs, sacrificing the number of scales in exchange for a faster computing speed.

[0074] The target filter kernel can be a filter kernel used to implement filtering and downsampling. The target filter kernel can be set according to the construction method of the target pyramid.

[0075] For example, if the target pyramid is constructed as a Gaussian pyramid, then a Gaussian filter kernel can be used as the target filter kernel; if the target pyramid is constructed as a mean pyramid, then a mean filter kernel can be used as the target filter kernel.

[0076] Step 333: Based on the size gradient image corresponding to each pyramid image layer, construct an image pyramid about the target object.

[0077] Therefore, by filtering and downsampling the size gradient image of the current layer, we can obtain the size gradient image of the previous layer, thereby obtaining the size gradient images of the pyramid image layers with a set number of layers, and thus constructing the image pyramid of the target object.

[0078] Under the above embodiments, through iterative filtering and downsampling processing, an image pyramid with a set number of levels can be constructed step by step based on the large-size target gradient image at the bottom layer. This not only effectively suppresses high-frequency noise and spectral aliasing by using filtering processing, ensuring the image noise anomaly suppression capability of each pyramid image layer, but also realizes the layered expression of the depth variation features of the target object in multi-scale space, thereby taking into account both macroscopic geometric contours and microscopic structural details.

[0079] Step 350: Assign corresponding scale weights to each pyramid image layer according to the setting method.

[0080] The scale weight is used to indicate the proximity of the scale corresponding to the size gradient image of the pyramid image layer to the target pyramid image layer where the target object is located.

[0081] The target pyramid image layer where the target object is located refers to the pyramid image layer whose scale best matches the size of the target object among all pyramid image layers in the image pyramid; proximity can refer to the proximity between the pyramid image layer and the target pyramid image layer. The higher the proximity, the more suitable the scale of the pyramid image layer is for representing the geometric structure of the target object.

[0082] The setting method can refer to the mapping rule between proximity and weight. Specifically, the scale weight is positively correlated with the proximity. The closer the proximity between the current pyramid image layer and the target pyramid image layer, the greater the scale weight assigned to it; conversely, the farther the proximity, the smaller the scale weight.

[0083] It should be understood that by using scale weights, the contribution of each pyramid image layer to image pyramid reconstruction can be dynamically adjusted according to the actual size of the target object. The pyramid image layer that best matches the size of the target object will provide the main image information (such as shape, contour, and structure) in image pyramid reconstruction.

[0084] Step 370: Perform image pyramid reconstruction based on the gradient image of each size and the corresponding scale weight to obtain the reconstructed gradient image.

[0085] First, it should be noted that image pyramid reconstruction and image pyramid construction are the opposite of each other. Image pyramid construction is based on the target gradient image, and through step-by-step filtering and downsampling, it generates a size gradient image with progressively decreasing size; while image pyramid reconstruction starts from the top of the pyramid and gradually restores the smallest size gradient image to the original size reconstructed gradient image through step-by-step upsampling.

[0086] It is understandable that, since the target gradient image used as input is an image used to characterize the depth change rate of the target object in a set direction, and each size gradient image is assigned a corresponding scale weight during the reconstruction process, the final reconstructed gradient image can emphasize the image information of the target object in the set direction and at the target pyramid image level.

[0087] In one embodiment, such as Figure 5 As shown, starting from the top layer and ending at the bottom layer of the image pyramid, for any layer of the image pyramid, the following steps can be included:

[0088] Step 371: Perform image processing operations on the size gradient image corresponding to the current layer to obtain the intermediate image corresponding to the current layer.

[0089] Among them, image processing operation refers to performing targeted feature enhancement or noise suppression processing on the size gradient image based on the geometric structure features and noise distribution characteristics of the target object in the size gradient image corresponding to the current layer. Intermediate image refers to the image obtained after completing the image processing operation on the size gradient image corresponding to the current layer.

[0090] Step 373: Perform weighted processing on the intermediate image based on the scale weight corresponding to the current layer to obtain the weighted image corresponding to the current layer.

[0091] It is understandable that through step 373, larger scale weights can be used to enhance geometric structural features (such as contour edges) that match the actual size of the target object, while smaller scale weights can be used to effectively attenuate or suppress background textures and high-frequency noise that do not match the actual size of the target object. This results in a weighted image with a higher signal-to-noise ratio and more accurate feature representation, providing a high-quality data foundation for subsequent pyramid reconstruction and fusion.

[0092] Step 375: If the current layer is not the top layer in the image pyramid, obtain the output image of the previous layer, enlarge it according to the set ratio, and fuse the weighted image corresponding to the current layer with the enlarged output image corresponding to the previous layer to obtain the output image of the current layer.

[0093] Step 377: If the current layer is the top layer in the image pyramid, then the weighted image corresponding to the current layer is used as the output image of the current layer.

[0094] It is understandable that, since image pyramid reconstruction is a top-down recursive reconstruction mechanism, the top layer serves as the baseline for image pyramid reconstruction, and there is no layer above it. Therefore, in the initial stage of image pyramid reconstruction, the weighted image corresponding to the top layer is used as the output image of the current layer. Furthermore, the output image of the bottom layer of the image pyramid is used as the reconstruction gradient image.

[0095] Furthermore, the output image of the previous layer, after being upsampled and magnified according to a set ratio, has the same size as the current layer, but the details are relatively blurry. Therefore, it mainly provides the main outline and general shape of the target object; while the weighted image of the current layer supplements the microscopic texture details unique to this size.

[0096] It should be noted that in image pyramid restoration, the set ratio is consistent with the size used in the image pyramid construction stage. For example, if the image pyramid construction stage uses a 1 / 2 set ratio for downsampling, then the image pyramid restoration stage will perform a corresponding upsampling (2x magnification) based on the same set ratio.

[0097] Image fusion can be achieved through overlay or channel stitching; no specific limitation is made here.

[0098] Under the above embodiments, image pyramid reconstruction can be performed based on each size gradient image and its corresponding scale weight to obtain a reconstructed gradient image. This allows the reconstructed gradient image to highlight features that match the actual size of the target object according to the guidance of the scale weight, while effectively suppressing interference from irrelevant scales, thereby more accurately reflecting the depth change information of the target object at the target pyramid image level.

[0099] Through the above process, by constructing an image pyramid containing multiple scales based on the target gradient image, and assigning a scale weight to each pyramid image layer that reflects its proximity to the target pyramid image layer where the target object is located, multi-scale adaptive enhancement of the depth variation features of the target object is achieved. Specifically, in the process of image pyramid reconstruction, the scale weights are used to weight the size gradient images of each pyramid image layer, so that the final reconstructed gradient image focuses on image information that matches the true scale of the target object, and effectively suppresses interference signals from unrelated pyramid image layers that are far from the true scale of the target object. This ensures that the reconstructed gradient image can accurately represent the depth variation rate of the target object in a set direction and that matches the target pyramid image layer where the target object is located with the optimal signal-to-noise ratio.

[0100] Please see Figure 6 In an exemplary embodiment, prior to step 310, the method may further include the following steps:

[0101] Step 410: Obtain at least one original gradient image.

[0102] In the case of acquiring multiple original gradient images, each original gradient image corresponds to a different gradient direction. The gradient direction can refer to the spatial vector direction used to calculate the rate of change of depth in the two-dimensional image plane coordinate system.

[0103] For example, when the gradient direction is horizontal (X-axis), it calculates the difference between pixel values ​​at horizontally adjacent positions, thus mainly reflecting the vertical edge information of the target object; when the gradient direction is vertical (Y-axis), it measures the difference between pixel values ​​at vertically adjacent positions, thus mainly reflecting the horizontal edge information of the target object.

[0104] Based on this, the X-axis and / or Y-axis can be used as gradient directions to completely cover most of the geometric structure information of the target object on the two-dimensional plane, without making specific limitations here.

[0105] Regarding the acquisition of the original gradient image, in one embodiment, before step 331, the method may further include the following steps: acquiring the original image; performing gradient extraction processing on the original image through a target direction filter kernel to obtain the original gradient image.

[0106] The original image refers to the image acquired by the image acquisition device of the target object. The image acquisition device can be an electronic device such as a two-dimensional image sensor or a three-dimensional vision sensor.

[0107] The reason for performing gradient extraction on the original image is understandable. Original images typically record the absolute distances of points on the surface of the target object relative to the camera. However, absolute distances are often significantly affected by the shooting distance or the object's placement and are difficult to directly reflect the object's geometric details. Gradient extraction transforms this absolute distance information into relative depth change rates, i.e., gradient information. A depth image (i.e., the original image) can express depth information qualitatively or quantitatively. Depth information can refer to data characterizing the spatial distances between points on the target object's surface and the image acquisition device.

[0108] Regarding target direction filtering kernels, they can include differential filtering kernels and Gaussian filtering kernels. Differential filtering kernels can be used to extract gradient information in the gradient direction from the original image, while Gaussian filtering kernels can be used to remove noise from the original image. Gaussian filtering kernels can be obtained by sampling the Gaussian distribution function or by approximating it using Pascal's triangle; no limitation is made here. In addition, other linear low-pass filtering kernels can be used to replace Gaussian filtering kernels to achieve the filtering effect; again, no limitation is made here.

[0109] In one embodiment, before step 410, the method may further include the following steps: obtaining a first directional filter kernel; obtaining a second directional filter kernel by transposing the first directional filter kernel; obtaining an original image; and performing gradient extraction processing on the original image using the first directional filter kernel and the second directional filter kernel respectively to obtain a first gradient image and a second gradient image.

[0110] The first directional filter kernel can be used to extract gradient information of the original image in the first direction, thereby obtaining a gradient image (i.e., the first gradient image) representing the target object in the first direction; the second directional filter kernel can be used to extract gradient information of the original image in the second direction, thereby obtaining a gradient image (i.e., the second gradient image) representing the target object in the second direction.

[0111] It should be noted that the first direction and the second direction are orthogonal in geometric space. Therefore, the second direction filter kernel can be obtained by transposing the first direction filter kernel.

[0112] In one embodiment, Used to characterize the first directional filter kernel , Represents convolution. Differential filter kernel , For one The size of the Gaussian filter kernel, Used to define the size of the Gaussian filter kernel; then, the second directional filter kernel... It can be ,in, This represents transposition.

[0113] For example, for designing a Gaussian filter kernel, selection As a Gaussian filter kernel. For designing a... Gaussian filter kernel, selection As a Gaussian filter kernel.

[0114] In one embodiment, at least one original gradient image includes a first gradient image and a second gradient image. The first gradient image is the gradient image of the target object along the X-axis. The second gradient image is the gradient image of the target object along the Y-axis.

[0115] Step 430: Calculate and determine the set direction based on the gradient information corresponding to each original gradient image.

[0116] The setting direction can refer to the gradient projection direction selected when quantifying the drastic changes in the surface depth of the target object.

[0117] It should be noted that the geometry and orientation of the target object in the actual scene are arbitrary. Fixed X / Y axes cannot represent the geometric features of the target object itself. Therefore, when the edges or textures of the target object are tilted (e.g., 45 degrees) or curved, using only the gradient direction of the X or Y axis to represent it will not accurately reflect the degree of depth change at that location.

[0118] To avoid the above situation, this embodiment adopts a set direction. Specifically, the set direction can be calculated based on the gradient information corresponding to the original gradient image (such as gradient information on the X-axis and Y-axis).

[0119] For example, if the gradient information of the first gradient image is x=10 and the gradient information of the second gradient image is y=20, then the set direction can be obtained according to θ=arctan(20,10), where the first input of arctan is the y value and the second input is the x value, and the returned angle θ∈[-π,π]. Since tanθ=y / x, we can get tanθ=20 / 10=2, and θ can be obtained through arctangent, where θ is the set direction.

[0120] Step 450: Based on the set direction, calculate the weights for each original gradient image to obtain the directional weights corresponding to each original gradient image.

[0121] Step 470: Based on the weight of each direction, the corresponding original gradient image is weighted and processed to obtain the corresponding target gradient image.

[0122] It should be understood that although the original gradient images completely cover the gradient directions in two-dimensional space (e.g., the X-axis and Y-axis directions), the image information carried in each original gradient image differs significantly due to the specific orientation of the geometric edges of the target object. For example, if the target edge is tilted by 30 degrees, the corresponding projection lengths in the X-axis and Y-axis directions will be drastically different.

[0123] Based on this, we can calculate the direction weights and use them to quantify the correlation between each original gradient image and the set direction. This ensures that when generating subsequent target gradient images, we can focus on strengthening the expression of gradient information in the set direction, while effectively suppressing redundant noise from irrelevant directions.

[0124] In one embodiment, the first direction weight is The weight in the second direction is .

[0125] in, Weights in the X-axis direction. Weights in the Y-axis direction. To define the direction. For example, multiply the first gradient image by a weight of 1 / 2, and multiply the second gradient image by a weight. At this point, the two original gradient images point to... The direction is determined to obtain the first target image and the second target image (i.e., the target gradient image), which will be highlighted in subsequent steps. Shape characteristics of direction.

[0126] Under the above embodiments, by introducing directional weights, the quality of the target gradient image is significantly improved. Since the geometric edges of the target object have a specific orientation, the contribution of each original gradient image to the image information in the set direction varies significantly. By using directional weights, the correlation between each original gradient image and the set direction can be quantified, ensuring that the original gradient image containing core geometric structure information is enhanced during the generation of the target gradient image, while redundant noise unrelated to the set direction is effectively suppressed. The final target gradient image can accurately represent the depth change of the target object in the set direction with extremely high clarity and fidelity.

[0127] In one embodiment, such as Figure 7 As shown, step 350 may also include the following steps:

[0128] Step 351: Determine the target pyramid image level where the target object is located based on the detection bounding box of the target object.

[0129] It is understandable that the size of the detection box (such as its area, perimeter, or dimensions) can reflect the size of the target object. Since each pyramid image layer carries image features at different sizes, it is possible to accurately determine the appropriate target pyramid image layer based on the detection box.

[0130] In one embodiment, step 351 can be achieved by... The implementation is as follows: where s is the side length of the smallest outer square of the target object (i.e., the detection box). It is the target pyramid image level.

[0131] Figure 8 This diagram illustrates the implementation of a detection bounding box for a target object. Figure 8 As shown on the left, edge enhancement processing can be performed based on the target gradient image to strengthen the edge contour lines of the target object; as shown on the left. Figure 8 As shown on the right, the image after edge enhancement is binarized to determine the corresponding detection box from the highlighted area of ​​the blank area corresponding to the target object.

[0132] Step 353: Based on the detection box of the target object, the target pyramid image level, and each pyramid image layer in the image pyramid, set the corresponding scale weight for each pyramid image layer.

[0133] Understandably, scale weights can accurately focus on and prioritize the size gradient information corresponding to different pyramid image layers based on the actual size of the target object. Since the detection box has determined the most suitable target pyramid image layer for the target object, it means that the image information carried by that target pyramid image layer best matches the actual size of the target object, containing the most effective image information. By setting corresponding scale weights for each pyramid image layer, the system can focus on enhancing the key image information of that target pyramid image layer, and gradually reduce the influence of other layers as the layer deviates. This scale-matching-based weight allocation mechanism ensures that subsequent processing can extract features that best match the actual size of the target object, thereby effectively avoiding redundant information interference from irrelevant layers and further improving the accuracy of image representation.

[0134] In one embodiment, step 353 may further include the following steps: obtaining the first parameter of the target pyramid image layer; calculating the baseline weight based on the first parameter, the detection box of the target object, and the target pyramid image layer; and setting corresponding scale weights for each pyramid image layer based on the baseline weights and the relationship between the target pyramid image layer and each pyramid image layer.

[0135] The first parameter indicates the importance of each image layer in the target pyramid. This first parameter defines the baseline weight of each image layer in the target pyramid at a global level, representing the importance of the image information contained within that layer.

[0136] By combining the size of the detection box with the target pyramid image level, the calculated baseline weights actually establish a reference baseline for the best-fitting scale layer.

[0137] Subsequently, the scale weights of each pyramid image layer are adaptively allocated based on the baseline weights, according to their relationship with the target pyramid image layers. This results in a continuous distribution of scale weights with a peak at the target pyramid image layer, ensuring that the image information that best matches the target object size is maximized while retaining auxiliary information from neighboring layers.

[0138] In one embodiment, the benchmark weight can be obtained through... To achieve, among which, It is the benchmark weight. It is the first parameter. It is a value in the range [0,1]. The greater the importance, the higher the value. The larger it is, the less important it is. The smaller, It is the target pyramid image level.

[0139] For example, It can be set to 0.8 to ensure that the image information of the target pyramid image layer occupies an absolutely dominant position, while leaving about 20% space for the adjacent layers of the target pyramid image layer to provide auxiliary information.

[0140] Regarding the layer weights corresponding to pyramid image layers other than the target pyramid image layer, in one embodiment, the following steps may be included: obtaining a second parameter; setting the scale weights corresponding to pyramid image layers larger than the target pyramid image layer to zero; setting the scale weights corresponding to pyramid image layers equal to the target pyramid image layer as baseline weights; for pyramid image layers smaller than the target pyramid image layer, based on the layer order corresponding to each pyramid image layer, using the second parameter to progressively attenuate the baseline weights to obtain the scale weights corresponding to pyramid image layers smaller than the target pyramid image layer.

[0141] The second parameter indicates the degree of attenuation of the scale weight when the scale of the pyramid image layer deviates from the target pyramid image level.

[0142] Specifically, setting the scale weight of pyramid image layers higher than the target pyramid image layer to zero is to eliminate blur interference. In an image pyramid, the higher the pyramid image layer, the lower the image resolution, and the smoother and blurrier the image information contained therein. Since the most suitable target pyramid image layer has been determined, the low-resolution information at higher levels no longer makes a significant contribution to representing the target object at the current size. Excluding it can effectively avoid distortion caused by ghosting or excessive background smoothing during image fusion.

[0143] Meanwhile, a layer-by-layer attenuation mechanism based on the second parameter is employed for pyramid image layers smaller than the target pyramid image layer level. The aim is to selectively introduce high-frequency detail compensation. The lower the pyramid image layer level, the higher the image resolution, providing finer texture and edge information than the target pyramid image layer level.

[0144] By setting a baseline weight and using a second parameter to attenuate layer by layer, local features from finer-scale pyramid image layers can be appropriately fused while ensuring that the target pyramid image layer dominates.

[0145] In one embodiment, the scale weights corresponding to each pyramid image layer can be represented by the following formula:

[0146]

[0147] in, It is the scale weight of the current layer. It is the current level. It is the target pyramid image level. It is the benchmark weight. It is the second parameter. The value of the second parameter determines the smooth transition effect of image information across different scales. If a proportional attenuation is desired, You can set it to 0.5. If you want to ignore the effects at lower scales, you can set it to... .

[0148] In addition, in specific application scenarios, besides following the aforementioned layer-by-layer attenuation, special scale weights can be directly set for specific pyramid image layers through Fourier transform and other methods. For example, if it is known that the target object has the most stable image information (such as a specific texture frequency) in a certain pyramid image layer, the scale weight of that specific pyramid image layer can be directly locked to the maximum value.

[0149] Through the above process, the target pyramid image level can be determined based on the detection box of the target object, and the baseline weight can be set based on this. The scale weight of each pyramid image level is set in conjunction with the baseline weight. This ensures that the reconstructed gradient image of the subsequent pyramid restoration can maintain the clear and stable main structure of the target object, while also supplementing the necessary micro details. This allows for the acquisition of the image expression with the highest contrast and the least interference when processing target objects of different sizes.

[0150] It should be noted that if the target gradient image includes a first target image and a second target image; in one embodiment, step 333 may further include the following steps: constructing an image pyramid based on the first target image and the second target image to obtain the image pyramid.

[0151] The first target image can be an image used to characterize the depth change rate of the target object in a set direction corresponding to the first direction, and the second target image can be an image used to characterize the depth change rate of the target object in a set direction corresponding to the second direction. The first direction can be the X-axis direction, and the second direction can be the Y-axis direction, which is not limited here.

[0152] Therefore, an image pyramid can be constructed based on the first target image and the second target image, thus obtaining the image pyramid. Each pyramid image layer in the image pyramid corresponds to a first image and a second image of the size gradient. The first image of the size gradient is used to characterize the depth change rate of the target object in a set direction corresponding to the first direction at the corresponding scale, and the second image of the size gradient is used to characterize the depth change rate of the target object in a set direction corresponding to the second direction at the corresponding scale.

[0153] Specifically, the first target image can be used as the first image of the size gradient corresponding to the lowest layer of the image pyramid, and the second target image can be used as the second image of the size gradient corresponding to the lowest layer of the image pyramid. Starting from the lowest layer to the highest layer in the image pyramid, for any pyramid image layer in the image pyramid: the first image of the size gradient and the second image of the size gradient of the current layer are filtered and downsampled to obtain the first image of the size gradient and the second image of the size gradient of the previous layer. The number of pyramid image layers in the image pyramid conforms to the set number of layers. Based on the first image of the size gradient and the second image of the size gradient corresponding to each pyramid image layer, an image pyramid about the target object is constructed.

[0154] In one embodiment, an image pyramid is constructed based on a first target image and a second target image. The process of obtaining the image pyramid is shown in the following formula:

[0155]

[0156]

[0157]

[0158] in, Represents the convolution operation. , These are the first and second images of the size gradient corresponding to the current layer, respectively. It is a pyramid image hierarchy. Corresponding to the previous layer of the current layer, when When =0, , These are the first target image and the second target image, respectively. , These are intermediate results obtained after filtering the first image and the second image of the size gradient of the current layer, respectively. This refers to the pixels corresponding to the first image and the second image of the current layer's size gradient. Indicates downsampling processing; Used to represent a Gaussian filter kernel.

[0159] In addition, a mean filter kernel can also be used. Alternative The expression for the mean filter kernel is as follows:

[0160]

[0161] Furthermore, since each pyramid image layer in the image pyramid corresponds to a first image of the size gradient and a second image of the size gradient, in one embodiment, the image pyramid reconstruction process further includes:

[0162] Image processing operations are performed on the first image and the second image of the size gradient corresponding to the current layer to obtain the first intermediate image and the second intermediate image corresponding to the current layer; based on the first intermediate image and the second intermediate image corresponding to the current layer, a first image fusion process is performed to obtain the fused image corresponding to the current layer;

[0163] The first intermediate image refers to the image obtained by performing image processing on the first image of the size gradient corresponding to the current layer, and the second intermediate image refers to the image obtained by performing image processing on the second image of the size gradient corresponding to the current layer.

[0164] It should be noted that the above image processing operations need to be image operations of the same principle; otherwise, they will interfere with each other in the subsequent image fusion process and fail to obtain a clear physical meaning.

[0165] For example, if the image processing operation is to extract the second derivative information of the gradient, then the first image of the size gradient needs to be filtered by the second derivative along the first direction (such as the X-axis direction), and the second image of the size gradient needs to be filtered by the second derivative along the second direction (such as the Y-axis direction).

[0166] Furthermore, the fused image is weighted based on the scale weight corresponding to the current layer to obtain the weighted fused image of the current layer; the weighted fused image of the current layer is then fused with the magnified output image of the previous layer to obtain the output image of the current layer.

[0167] It should be noted that the scale weights corresponding to each pyramid image layer can be applied to the size gradient image corresponding to that layer. Based on this, the fused image of the current layer can be weighted using a unified scale weight to obtain the weighted fused image of the current layer.

[0168] In one embodiment, the expression for the above image pyramid reconstruction process can be: ;in, This refers to the current layer. It refers to the layer above the current layer. It can refer to the output image of the current layer. This refers to the scale weight of the current layer; , This refers to the first image and the second image of the size gradient corresponding to the current layer. This refers to performing the same image processing operation on the first image and the second image of the size gradient, respectively. These are the first intermediate image and the second intermediate image, respectively. This refers to the first image fusion process; It could refer to the magnified output image of the previous layer. This refers to performing a second image fusion process, which combines the weighted fused image of the current layer with the magnified output image of the previous layer.

[0169] Under the above embodiments, by constructing a first target image and a second target image respectively, and using them as the bottom layer input of the image pyramid, the first image and the second image of the size gradient corresponding to each pyramid image layer are output. This allows the geometric projection information of the target object at different sizes in the set directions about the first and second directions to be preserved in each pyramid image layer. In addition, by using the scale weights corresponding to each pyramid image layer, the image pyramid is restored based on the first image and the second image of the size gradient of each pyramid image layer. According to the importance of the hierarchy determined by the scale weights, the proportion of the gradient information about the first direction and the gradient information about the second direction in each pyramid image layer in the final gradient reconstruction image can be adaptively adjusted. This allows the image pyramid restoration process to focus on preserving the gradient change rate in the set directions in the pyramid image layers that match the size of the target object, and effectively suppress background noise or blurring interference that may exist in the mismatched pyramid image layers. This ensures that the final generated gradient reconstruction image has better structural clarity and signal-to-noise ratio in the set directions.

[0170] Figures 9 to 11 This is a schematic diagram illustrating the specific implementation of an image enhancement method in an application scenario. In this scenario, the original image is a depth image, the original gradient image includes a first gradient image and a second gradient image, the target gradient image includes a first target image and a second target image, and the pyramid image layers of each image pyramid correspond to the first image and the second image of the size gradient.

[0171] Please see Figure 9 ,like Figure 9 As shown, after acquiring the depth image, gradient extraction can be performed on the depth image using a first directional filter kernel (filter kernel 1 in the figure) and a second directional filter kernel (filter kernel 2 in the figure) to obtain the first gradient image and the second gradient image.

[0172] Figure 10 The diagram illustrates a specific implementation of a first gradient image and a second gradient image, wherein... Figure 10 The first gradient image shown on the left is the gradient image of the target object on the X-axis. Figure 10 The second gradient image shown on the right is the gradient image of the target object on the Y-axis.

[0173] Therefore, the gradient information corresponding to the first gradient image and the second gradient image can be used to calculate and determine the set direction. Then, based on the set direction, the weights of the first gradient image and the second gradient image can be calculated respectively to obtain the first direction weight and the second direction weight corresponding to the first gradient image and the second gradient image.

[0174] Based on this, the first gradient image can be weighted using the first directional weight to obtain the first target image, and the second target image can be obtained by weighting the second directional weight using the second directional weight.

[0175] Regarding the construction of an image pyramid based on the first target image and the second target image, such as... Figure 11 As shown in the lower part, firstly, the first target image can be used as the first image of the size gradient corresponding to the lowest layer of the image pyramid, and the second target image can be used as the second image of the size gradient corresponding to the lowest layer of the image pyramid. Starting from the lowest layer to the highest layer in the image pyramid, for any pyramid image layer in the image pyramid: the first image of the size gradient and the second image of the size gradient of the current layer are filtered and downsampled by 1 / 2 to obtain the first image of the size gradient and the second image of the size gradient of the previous layer. The number of pyramid image layers in the image pyramid conforms to the set number of layers. Based on the first image of the size gradient and the second image of the size gradient corresponding to each pyramid image layer, an image pyramid about the target object is constructed.

[0176] Regarding the reconstruction of the image pyramid based on the first image and the second image corresponding to the size gradient of each pyramid image layer, such as... Figure 11 As shown in the upper part, image processing operations are performed on the first and second size gradient images corresponding to the current layer to obtain the first and second intermediate images corresponding to the current layer. A first image fusion process is then performed based on the first and second intermediate images corresponding to the current layer to obtain the fused image corresponding to the current layer. Further, the fused image is weighted based on the scale weight corresponding to the current layer to obtain the weighted fused image of the current layer. A second image fusion process is then performed between the weighted fused image of the current layer and the magnified output image of the previous layer to obtain the output image of the current layer. The output image of the lowest layer in the image pyramid is used as the reconstructed gradient image.

[0177] Reconstructing gradient images can emphasize the gradient change rate of a target object in a set direction and corresponding to a specific scale of the target object. In subsequent image tasks (such as multi-scale target detection or fine defect identification), this dual enhancement feature of direction focusing and scale filtering can be used to achieve better detection accuracy of targets of different sizes and the ability to suppress non-target scale noise.

[0178] In this application scenario, the three-dimensional geometric properties of depth images are utilized. By orthogonally decomposing the depth image into a first target image and a second target image, and fusing them with directional weights, accurate detection of the gradient change rate in a set direction is achieved, effectively avoiding the loss of directional information caused by single-dimensional processing. At the same time, this application scenario introduces a scale weight mechanism in the process of restoring the image pyramid. By weighting different pyramid image levels, adaptive enhancement and background noise suppression at a specific scale are achieved.

[0179] Furthermore, this hierarchical processing architecture based on image pyramids possesses high algorithmic versatility and scalability. Its internal image processing operators (such as the aforementioned filtering or downsampling operations) can be flexibly configured or replaced according to actual needs. This means that the architecture can be seamlessly integrated with various feature extraction algorithms. Only the core processing logic in the pyramid iteration process needs to be adjusted to achieve customized detection of different types of geometric features (such as edges, textures, or curvature), thereby meeting diverse visual task requirements.

[0180] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0181] This document describes various exemplary embodiments with reference to them. However, those skilled in the art will recognize that changes and modifications can be made to the exemplary embodiments without departing from the scope of this document. For example, various operational steps and components for performing operational steps can be implemented in different ways depending on the specific application or considering any number of cost functions associated with the operation of the system (e.g., one or more steps can be deleted, modified, or combined with other steps).

[0182] Those skilled in the art will understand that all or part of the functions of the various methods in the above embodiments can be implemented by hardware or by computer programs. When all or part of the functions in the above embodiments are implemented by computer programs, the program can be stored in a computer-readable storage medium, which may include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to achieve the above functions. For example, the program can be stored in the memory of a device, and when the program in the memory is executed by the processor, all or part of the above functions can be achieved. In addition, when all or part of the functions in the above embodiments are implemented by computer programs, the program can also be stored in a server, another computer, disk, optical disk, flash drive, or external hard drive, etc., and can be downloaded or copied to the memory of a local device, or the system of the local device can be updated. When the program in the memory is executed by the processor, all or part of the functions in the above embodiments can be achieved.

[0183] In the above embodiments, implementation can be achieved, in whole or in part, by software, hardware, firmware, or any combination thereof. Furthermore, as those skilled in the art will understand, the principles herein can be reflected in a computer program product on a computer-readable storage medium pre-loaded with computer-readable program code. Any tangible, non-transitory computer-readable storage medium may be used, including magnetic storage devices (hard disks, floppy disks, etc.), optical storage devices (CD-ROMs, DVDs, Blu-ray discs, etc.), flash memory, and / or the like. These computer program instructions can be loaded onto a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to form a machine, such that instructions executing on the computer or other programmable data processing apparatus can generate means for performing a specified function. These computer program instructions can also be stored in a computer-readable storage medium that can instruct the computer or other programmable data processing apparatus to operate in a particular manner, such that instructions stored in the computer-readable storage medium can form an article of manufacture including means for implementing the specified function. The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to perform a series of operational steps on the computer or other programmable apparatus to produce a computer-implemented process, such that instructions executing on the computer or other programmable apparatus can provide steps for implementing the specified function.

[0184] While the principles herein have been illustrated in various embodiments, numerous modifications to the structure, arrangement, proportions, elements, materials, and components, particularly suited to specific environmental and operational requirements, may be used without departing from the principles and scope of this disclosure. These modifications and other alterations or alterations will be included within the scope of this document.

[0185] The foregoing specific descriptions have been described with reference to various embodiments. However, those skilled in the art will recognize that various modifications and changes can be made without departing from the scope of this disclosure. Therefore, considerations for this disclosure are to be illustrative rather than restrictive, and all such modifications are to be included within its scope. Similarly, advantages, other advantages, and solutions to problems with respect to various embodiments have been described above. However, benefits, advantages, solutions to problems, and any elements that produce these, or make them more explicit, should not be construed as critical, essential, or necessary. The term “comprising” and any other variations thereof as used herein are non-exclusive inclusion, meaning that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not expressly listed or not part of the process, method, system, article, or apparatus. Furthermore, the term “coupled” and any other variations thereof as used herein refer to physical connections, electrical connections, magnetic connections, optical connections, communication connections, functional connections, and / or any other connections.

[0186] Those skilled in the art will recognize that many changes can be made to the details of the above embodiments without departing from the basic principles of the invention. Therefore, the scope of the invention should be determined only by the claims.

Claims

1. An image enhancement method, characterized in that, include: Obtain at least one target gradient image; The target gradient image is an image used to characterize the rate of depth change of the target object in a set direction; An image pyramid about the target object is constructed based on the at least one target gradient image; the image pyramid includes multiple pyramid image layers of multiple scales, each pyramid image layer includes the same number of size gradient images as the target gradient images, the size gradient images of the pyramid image layers are used to characterize the depth change rate of the target object in the set direction at the corresponding scale, and the scales corresponding to the size gradient images of different pyramid image layers are different; Each pyramid image layer is assigned a corresponding scale weight according to the setting method; The scale weight is used to indicate the proximity of the scale corresponding to the size gradient image of the pyramid image layer to the target pyramid image layer where the target object is located; Based on each size gradient image and its corresponding scale weight, an image pyramid is restored to obtain a reconstructed gradient image.

2. The method as described in claim 1, characterized in that, The acquisition of at least one target gradient image includes: Acquire at least one original gradient image; in the case of acquiring multiple original gradient images, each original gradient image corresponds to a different gradient direction; The set direction is determined by calculating the gradient information corresponding to each of the original gradient images; Based on the set direction, weights are calculated for each of the original gradient images to obtain the directional weights corresponding to each of the original gradient images. The original gradient image is weighted according to the weight of each direction to obtain the corresponding target gradient image.

3. The method as described in claim 2, characterized in that, Before acquiring at least one original gradient image, the method further includes: Obtain the original image; The original image is subjected to gradient extraction processing by a target direction filtering kernel to obtain an original gradient image; the target direction filtering kernel includes a differential filtering kernel and a Gaussian filtering kernel.

4. The method as described in claim 2, characterized in that, The at least one original gradient image includes a first gradient image and a second gradient image; wherein, the first gradient image is the gradient image of the target object in the X-axis direction; and the second gradient image is the gradient image of the target object in the Y-axis direction.

5. The method as described in claim 4, characterized in that, Before acquiring at least one original gradient image, the method further includes: Obtain the first directional filter kernel; The second directional filter kernel is obtained by transposing the first directional filter kernel; Obtain the original image; The original image is subjected to gradient extraction processing through the first directional filter kernel and the second directional filter kernel respectively to obtain the first gradient image and the second gradient image.

6. The method as described in claim 1, characterized in that, The construction of an image pyramid about the target object based on the at least one target gradient image includes: The target gradient image is used as the size gradient image corresponding to the lowest layer of the image pyramid; wherein, the number of pyramid image layers of the image pyramid conforms to a set number of layers; Starting from the bottom layer to the top layer in the image pyramid, for any pyramid image layer in the image pyramid: filter and downsample the size gradient image of the current layer to obtain the size gradient image of the previous layer; Based on the size gradient image corresponding to each pyramid image layer, the image pyramid about the target object is constructed.

7. The method as described in claim 6, characterized in that, The step of filtering and downsampling the size gradient image of the current layer to obtain the size gradient image of the previous layer includes: The size gradient image corresponding to the current layer is filtered by the target filtering kernel to obtain the filtered size gradient image corresponding to the current layer. The size gradient image corresponding to the current layer after filtering is downsampled according to a set ratio to obtain the size gradient image corresponding to the previous layer of the current layer.

8. The method as described in claim 1, characterized in that, The step of setting corresponding scale weights for each pyramid image layer according to the setting method includes: The target pyramid image level where the target object is located is determined based on the detection bounding box of the target object; Based on the detection bounding box of the target object, the target pyramid image level, and each pyramid image layer in the image pyramid, a corresponding scale weight is set for each pyramid image layer.

9. The method as described in claim 8, characterized in that, The method of setting corresponding scale weights for each pyramid image layer based on the detection bounding box of the target object, the target pyramid image layer, and each pyramid image layer in the image pyramid includes: Obtain the first parameter of the target pyramid image level; the first parameter is used to indicate the importance of the target pyramid image level. Based on the first parameter, the detection box of the target object, and the target pyramid image level, a baseline weight is calculated; Based on the baseline weights and the relationship between the target pyramid image layers and each pyramid image layer, corresponding scale weights are assigned to each pyramid image layer.

10. The method as described in claim 9, characterized in that, Based on the baseline weights and the relationship between the target pyramid image levels and each pyramid image layer, the following steps are taken to assign corresponding scale weights to each pyramid image layer: Obtain the second parameter; the second parameter is used to indicate the degree of attenuation of the scale weight when the scale of the pyramid image layer deviates from the target pyramid image level; Set the scale weight of the pyramid image layers that are larger than the target pyramid image layer to zero; The scale weights corresponding to the pyramid image layers that are equal to the target pyramid image layer level are set as the baseline weights. For each pyramid image layer smaller than the target pyramid image layer, based on the hierarchical order of each pyramid image layer, the reference weight is attenuated layer by layer using the second parameter to obtain the scale weight corresponding to each pyramid image layer smaller than the target pyramid image layer.

11. The method as described in claim 1, characterized in that, The step of performing image pyramid reconstruction based on each size gradient image and its corresponding scale weight to obtain a reconstructed gradient image includes: Starting from the top layer and ending at the bottom layer of the image pyramid, for any pyramid image layer in the image pyramid: Perform image processing operations on the size gradient image corresponding to the current layer to obtain the intermediate image corresponding to the current layer; The intermediate image is weighted based on the scale weights corresponding to the current layer to obtain the weighted image corresponding to the current layer. If the current layer is not the top layer in the image pyramid, then the output image of the layer above the current layer is obtained, enlarged according to a set ratio, and the weighted image corresponding to the current layer is fused with the enlarged output image corresponding to the layer above to obtain the output image of the current layer. If the current layer is the top layer in the image pyramid, then the weighted image corresponding to the current layer is used as the output image of the current layer; The output image of the lowest layer of the image pyramid is used as the reconstructed gradient image.

12. The method as described in claim 11, characterized in that, Each pyramid image layer in the image pyramid includes a corresponding size gradient first image and a size gradient second image; The step of performing image processing operations on the size gradient image corresponding to the current layer to obtain the intermediate image corresponding to the current layer includes: Image processing operations are performed on the first image and the second image of the size gradient corresponding to the current layer to obtain the first intermediate image and the second intermediate image corresponding to the current layer. The step of weighting the intermediate image based on the scale weights corresponding to the current layer to obtain the weighted image corresponding to the current layer includes: A first image fusion process is performed based on the first intermediate image and the second intermediate image corresponding to the current layer to obtain the fused image corresponding to the current layer. The fused image is weighted based on the scale weights corresponding to the current layer to obtain the weighted fused image of the current layer; The step of fusing the weighted image corresponding to the current layer with the magnified output image corresponding to the previous layer to obtain the output image of the current layer includes: The weighted fused image corresponding to the current layer is combined with the magnified output image corresponding to the previous layer for a second image fusion process to obtain the output image of the current layer.

13. The method according to any one of claims 1 to 12, characterized in that, The target gradient image includes a first target image and a second target image; The construction of an image pyramid about the target object based on the at least one target gradient image includes: An image pyramid is constructed based on the first target image and the second target image to obtain the image pyramid; each pyramid image layer in the image pyramid corresponds to a first image of size gradient and a second image of size gradient.

14. A computer-readable storage medium, characterized in that, The medium stores a computer program that can be executed by a processor to implement the method as described in any one of claims 1-13.

Images