Image processing method and device, electronic equipment and storage medium

Abstract

The present disclosure provides an image processing method and device, electronic equipment and storage medium, and relates to the technical field of vision. The image processing method comprises: performing visual recognition on a to-be-processed image to obtain a foreground image region of the to-be-processed image; determining a region other than the foreground image region in the to-be-processed image as a background image region of the to-be-processed image; performing dynamic window smoothing processing on the background image region to obtain a pre-processed image corresponding to the to-be-processed image; performing image coding processing on the pre-processed image to obtain a coded and decoded image; and processing the coded and decoded image based on a task network to obtain a task result of the to-be-processed image. The method can reduce the information quantity of a region of no interest, improve the compression ratio, enhance the contrast between the region of interest and the region of no interest, and has little effect on the accuracy of subsequent tasks.

Description

Technical Field

[0001] This disclosure relates to the field of visual technology, and in particular to an image processing method, an image processing apparatus, an electronic device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] Video compression coding for human vision and video compression coding for machine vision differ significantly in application scenarios, evaluation mechanisms, information processing, and information usage. The establishment of the Video Coding for Machines (VCM) standards group marks a shift in video compression coding technology towards addressing the needs of the machine vision field, researching compression coding techniques suitable for machine vision and human-machine hybrid vision scenarios to improve performance and efficiency in intelligent applications.

[0003] In related technologies, images for intelligent tasks are acquired, directly encoded and decoded to obtain encoded and decoded images, and then input into the intelligent task network to complete subsequent tasks. However, because the encoding and decoding of the original image is performed directly without considering the performance of the encoded and decoded image in subsequent tasks, important feature information for the task may be lost, affecting the accuracy and efficiency of the task.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] This disclosure provides an image processing method, an image processing apparatus, an electronic device, a computer-readable storage medium, and a computer program product, which at least to some extent overcome the problems existing in the related art.

[0006] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0007] According to one aspect of this disclosure, an image processing method is provided, comprising: performing visual recognition on an image to be processed to obtain a foreground image region of the image to be processed; determining a region in the image to be processed other than the foreground image region as a background image region of the image to be processed; performing dynamic window smoothing processing on the background image region to obtain a preprocessed image corresponding to the image to be processed; performing image encoding and decoding processing on the preprocessed image to obtain an encoded and decoded image; and processing the encoded and decoded image based on a task network to obtain a task result of the image to be processed.

[0008] In some embodiments of this disclosure, the step of performing dynamic window smoothing on the background image region to obtain a preprocessed image corresponding to the image to be processed includes: determining a target smoothing window corresponding to the background image region based on the distance from a pixel in the background image region to the region boundary of the foreground image region; performing window smoothing on the background image region based on the target smoothing window to obtain a smoothed background image region; and updating the background image region in the image to be processed using the smoothed background image region to obtain the preprocessed image.

[0009] In some embodiments of this disclosure, determining the target smoothing window corresponding to the background image region based on the distance from the pixels in the background image region to the region boundary of the foreground image region includes: dividing the background image region into a first sub-region and a second sub-region based on the distance from the pixels in the background image region to the region boundary of the foreground image region; wherein the distance between the pixels in the first sub-region and the region boundary of the foreground image region is less than or equal to a preset distance threshold, and the distance between the pixels in the second sub-region and the region boundary of the foreground image region is greater than the preset distance threshold; determining a first smoothing window corresponding to the first sub-region, and determining a second smoothing window corresponding to the second sub-region; wherein the first smoothing window is used to perform window smoothing processing on the first sub-region, and the second smoothing window is used to perform window smoothing processing on the second sub-region.

[0010] In some embodiments of this disclosure, the window size of the first smoothing window and the window size of the second smoothing window are preset, and the window size of the first smoothing window is smaller than the window size of the second smoothing window.

[0011] In some embodiments of this disclosure, the step of performing window smoothing processing on the background image region based on the target smoothing window to obtain a smoothed background image region includes: traversing the background image region based on the window size of the target smoothing window to obtain an image region to be smoothed; for the image region to be smoothed, calculating the pixel values ​​of the pixels in the image region to be smoothed using a preset calculation method to obtain a pixel smoothing value corresponding to the image region to be smoothed, and replacing the pixel values ​​of the pixels in the image region to be smoothed with the pixel smoothing value corresponding to the image region to be smoothed; after traversing the background image region, the smoothed background image region is obtained.

[0012] In some embodiments of this disclosure, the step of calculating the pixel values ​​of pixels within the image region to be smoothed using a preset calculation method to obtain a pixel smoothing value corresponding to the image region to be smoothed, and replacing the pixel values ​​of pixels within the image region to be smoothed with the pixel smoothing value corresponding to the image region to be smoothed, includes: obtaining the pixel values ​​of pixels within the image region to be smoothed in multiple channels; for each channel, calculating the pixel values ​​of pixels within the image region to be smoothed in that channel using the preset calculation method to obtain a pixel smoothing value corresponding to the image region to be smoothed in that channel; and replacing the pixel values ​​of pixels within the image region to be smoothed in that channel with the pixel smoothing value corresponding to the image region to be smoothed in that channel.

[0013] In some embodiments of this disclosure, the preset calculation method includes at least one of the following options: mean calculation, weighted mean calculation, median calculation, mode calculation, and Gaussian filtering.

[0014] According to another aspect of this disclosure, an image processing apparatus is also provided, comprising: a visual recognition module configured to perform visual recognition on an image to be processed to obtain a foreground image region of the image to be processed; a region determination module configured to determine a region in the image to be processed other than the foreground image region as a background image region of the image to be processed; a window smoothing module configured to perform dynamic window smoothing processing on the background image region to obtain a preprocessed image corresponding to the image to be processed; an encoding / decoding module configured to perform image encoding / decoding processing on the preprocessed image to obtain an encoded / decoded image; and a task processing module configured to process the encoded / decoded image based on a task network to obtain a task result for the image to be processed.

[0015] According to another aspect of this disclosure, an electronic device is also provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the image processing method described in any of the preceding claims by executing the executable instructions.

[0016] According to another aspect of this disclosure, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the image processing method described in any of the preceding claims.

[0017] According to another aspect of this disclosure, a computer program product is also provided, comprising: a computer program or instructions that, when executed by a processor, implement the image processing method of any one of the above.

[0018] The image processing method provided in the embodiments of this disclosure obtains foreground and background image regions by performing visual recognition on the image to be processed. That is, it distinguishes the region of interest (i.e., the foreground image region) and the region of no interest (i.e., the background image region) in the image to be processed. On this basis, dynamic window smoothing processing is performed on the background image region, which effectively reduces redundant information and noise in the background image region and reduces the amount of information in the region of no interest. This enables a higher compression ratio to be achieved during the encoding and decoding process, enhances the contrast between the region of interest and the region of no interest, and can better preserve the feature information that is important to subsequent tasks (i.e., the key features of the foreground image region) while ensuring encoding and decoding efficiency. It has little impact on the accuracy of subsequent tasks and helps to improve the accuracy and efficiency of the task.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0021] Figure 1 A schematic diagram of an exemplary application system architecture to which the image processing methods of the embodiments of this disclosure can be applied is shown;

[0022] Figure 2 This diagram illustrates a flowchart of an image processing method provided by an embodiment of the present disclosure;

[0023] Figure 3 A flowchart of a method for dynamic window smoothing of a background image region provided in an embodiment of this disclosure is shown;

[0024] Figure 4 This illustration shows a schematic diagram of dynamic window smoothing processing of a background image in an image to be processed, provided by an embodiment of this disclosure.

[0025] Figure 5 A flowchart is shown below illustrating a method for window smoothing of a background image region based on a target smoothing window, according to an embodiment of this disclosure.

[0026] Figure 6 This diagram illustrates an image processing apparatus according to an embodiment of the present disclosure.

[0027] Figure 7 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0028] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0029] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0030] Figure 1 A schematic diagram of an exemplary application system architecture to which the image processing methods of the embodiments of this disclosure can be applied is shown. For example... Figure 1 As shown, the system architecture may include terminal device 101, network 102 and server 103.

[0031] Network 102 is a medium used to provide a communication link between terminal device 101 and server 103, and can be a wired network or a wireless network.

[0032] Optionally, the aforementioned wireless or wired networks use standard communication technologies and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to Local Area Networks (LANs), Metropolitan Area Networks (MANs), Wide Area Networks (WANs), mobile, wired or wireless networks, private networks, or any combination of virtual private networks. In some embodiments, technologies and / or formats including Hyper Text Markup Language (HTML), Extensible Markup Language (XML), etc., are used to represent data exchanged over the network. Furthermore, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Networks (VPNs), and Internet Protocol Security (IPSec) can be used to encrypt all or some links. In other embodiments, custom and / or dedicated data communication technologies can be used to replace or supplement the aforementioned data communication technologies.

[0033] Terminal device 101 can be various electronic devices, including but not limited to smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, wearable devices, augmented reality devices, virtual reality devices, etc.

[0034] Optionally, the client of the application installed on different terminal devices 101 may be the same, or the client of the same type of application based on different operating systems. Depending on the terminal platform, the specific form of the application client may also be different; for example, the application client may be a mobile client, a PC client, etc.

[0035] Server 103 can be a server that provides various services, such as a backend management server that supports the device operated by the user using terminal device 101. The backend management server can analyze and process received requests and other data, and feed the processing results back to the terminal device.

[0036] Optionally, the server can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0037] Those skilled in the art will know that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative; any number of terminal devices, networks, and servers can be included depending on actual needs. This disclosure does not limit the scope of the embodiments.

[0038] Under the above system architecture, this disclosure provides an image processing method that can be executed by any electronic device with computing capabilities.

[0039] In some embodiments, the image processing method provided in this disclosure can be executed by a terminal device of the system architecture described above; in other embodiments, the image processing method provided in this disclosure can be executed by a server in the system architecture described above; in still other embodiments, the image processing method provided in this disclosure can be implemented by the terminal device and the server in the system architecture described above through interaction.

[0040] Figure 2 This diagram illustrates a flowchart of an image processing method provided by an embodiment of the present disclosure, such as... Figure 2 As shown, the image processing method provided in this embodiment includes the following steps.

[0041] Step S210: Perform visual recognition on the image to be processed to obtain the foreground image region of the image to be processed.

[0042] In this embodiment of the disclosure, the image to be processed can be understood as an image used for intelligent task processing. The image to be processed can be a static image, such as a photograph, or a dynamic image, such as a video frame.

[0043] In this embodiment of the disclosure, the foreground image region can be called the region of interest, which may include the main object or region of interest in the image; the background image region can be called the region of no interest, which may include content in the image other than the main object.

[0044] In one possible implementation, the image to be processed is input into a visual recognition module, which includes intelligent analysis algorithms for identifying key elements or features in the image. The visual recognition module outputs a visual recognition result, which includes information about the foreground object of interest, such as its location, shape, and size. Then, the foreground image region in the image to be processed is determined based on the visual recognition result.

[0045] In another possible implementation, the Faster R-CNN X101-FPN instance segmentation model is used for machine vision task pre-inference. Faster R-CNN is an advanced deep learning model for object detection and instance segmentation; X101-FPN is a variant of Faster R-CNN, where X101 indicates the use of ResNeXt-101 as the feature extraction network, and FPN indicates the use of a feature pyramid network to enhance the model's ability to detect objects at different scales. The segmentation results of the image to be processed, denoted as set S, are output by the Faster R-CNN X101-FPN instance segmentation model. Set S contains all pixels or regions in the image to be processed that are identified as foreground object regions. The output of this instance segmentation model can be used to determine the foreground image regions in the image to be processed.

[0046] Step S220: The region in the image to be processed, excluding the foreground image region, is determined as the background image region of the image to be processed.

[0047] In this embodiment of the disclosure, the foreground image region is regarded as a subset of the image to be processed, and it is inverted to obtain the region in the image to be processed other than the foreground image region, that is, the background image region.

[0048] Step S230: Perform dynamic window smoothing on the background image region to obtain the preprocessed image corresponding to the image to be processed.

[0049] In this embodiment of the disclosure, after obtaining the background image region, a dynamic window is used to smooth the background image region to complete the preprocessing operation of the image to be processed and obtain a preprocessed image. The dynamic window smoothing process can adaptively adjust according to the different characteristics of the background image region; that is, different windows can be used for different parts of the background image region, thus making the preprocessed image smoother and more uniform.

[0050] Step S240: Perform image encoding and decoding processing on the preprocessed image to obtain the encoded and decoded image.

[0051] In this embodiment of the disclosure, image encoding and decoding includes image encoding and image decompression, which is a compression and decompression technology that can compress image data into smaller file sizes for storage or transmission. By performing image encoding and decoding processing on pre-processed images, the storage and transmission requirements of image data can be reduced, while maintaining image quality as much as possible.

[0052] Step S250: Process the encoded and decoded image based on the task network to obtain the task result of the image to be processed.

[0053] In this embodiment of the disclosure, the task network can be a deep learning model, machine learning algorithm, etc. The task network can perform specific tasks based on the content of the image, such as object recognition, classification, detection, etc. After obtaining the encoded and decoded image, the encoded and decoded image is input into the task network, and the task result of the image to be processed is output, such as the name of the recognized object, classification label, location information, etc.

[0054] The image processing method provided in the embodiments of this disclosure obtains foreground and background image regions by performing visual recognition on the image to be processed. That is, it distinguishes the region of interest (i.e., the foreground image region) and the region of no interest (i.e., the background image region) in the image to be processed. On this basis, dynamic window smoothing processing is performed on the background image region, which effectively reduces redundant information and noise in the background image region and reduces the amount of information in the region of no interest. This enables a higher compression ratio to be achieved during the encoding and decoding process, enhances the contrast between the region of interest and the region of no interest, and can better preserve the feature information that is important to subsequent tasks (i.e., the key features of the foreground image region) while ensuring encoding and decoding efficiency. It has little impact on the accuracy of subsequent tasks and helps to improve the accuracy and efficiency of the task.

[0055] Figure 3 A flowchart illustrating a method for dynamic window smoothing of a background image region provided in an embodiment of this disclosure is shown. Figure 3 As shown, in some embodiments of this disclosure, step S230 above performs dynamic window smoothing processing on the background image region to obtain a preprocessed image corresponding to the image to be processed, including the following steps.

[0056] Step S310: Determine the target smoothing window corresponding to the background image region based on the distance between the pixels in the background image region and the region boundary of the foreground image region.

[0057] The target smoothing window refers to the window used to smooth the background image region. The size of the target smoothing window can be determined based on the positional relationship between the background image region and the foreground image region. For example, a smaller smoothing window is used for the background image region closer to the foreground image region, while a larger smoothing window is used for the background image region farther from the foreground image region. The shape of the target smoothing window can be set based on experience or requirements; for example, the target smoothing window can be rectangular, circular, or other shapes.

[0058] In this embodiment of the disclosure, after determining the background image region and the foreground image region in the image to be processed, the pixels in the background image region are analyzed, the distance from the pixel to the region boundary of the foreground image region is calculated, and the target smoothing window is determined based on the calculated distance.

[0059] In one implementation of this disclosure, the distance from a pixel in the background image region to the boundary of the foreground image region refers to the shortest distance from the pixel in the background image region to the boundary of the foreground image region. For example, if the foreground image region is rectangular with four boundaries, the distances from pixel P1 in the background image region to these four boundaries are calculated, and the minimum value is selected as the distance from pixel P1 to the boundary of the foreground image region.

[0060] In some embodiments of this disclosure, step S310, which determines the target smoothing window corresponding to the background image region based on the distance from the pixel point in the background image region to the region boundary of the foreground image region, includes: dividing the background image region into a first sub-region and a second sub-region based on the distance from the pixel point in the background image region to the region boundary of the foreground image region; determining a first smoothing window corresponding to the first sub-region; and determining a second smoothing window corresponding to the second sub-region.

[0061] Wherein, the distance between the pixels in the first sub-region and the region boundary of the foreground image region is less than or equal to a preset distance threshold, and the distance between the pixels in the second sub-region and the region boundary of the foreground image region is greater than the preset distance threshold; the first smoothing window is used to perform window smoothing processing on the first sub-region, and the second smoothing window is used to perform window smoothing processing on the second sub-region.

[0062] In some embodiments of this disclosure, the window size of the first smoothing window and the window size of the second smoothing window are preset, and the window size of the first smoothing window is smaller than the window size of the second smoothing window.

[0063] In this embodiment of the disclosure, the window size can be set to two levels, namely, a first smooth window and a second smooth window are preset, and the window size of the first smooth window is set to be smaller than the window size of the second smooth window.

[0064] In one possible implementation, the window size can be set as follows: the first smoothing window is n*n, and the second smoothing window is 2n*2n, where n is greater than or equal to 1 and can be set according to actual needs and specific application scenarios. For example, if subsequent tasks require the background image region to meet certain quality standards, n can be set smaller, such as n=2. If subsequent tasks are related to foreground image region detection and do not focus on the content of the background image region, n can be set larger, such as n=3.

[0065] In this embodiment of the disclosure, when determining the target smoothing window corresponding to the background image region, a first smoothing window can be used for window smoothing of the portion of the background image region that is close to the foreground image region, and a second smoothing window can be used for window smoothing of the portion of the background image region that is close to the foreground image region. Specifically, for the pixels in the background image region, if the distance from the pixel to the boundary of the foreground image region is less than or equal to a preset distance, the first smoothing window is used for window smoothing; if the distance from the pixel to the boundary of the foreground image region is greater than the preset distance, the second smoothing window is used for window smoothing.

[0066] Figure 4 This illustration shows a schematic diagram of dynamic window smoothing processing of a background image in an image to be processed, provided by an embodiment of this disclosure. For example... Figure 4 As shown, the two human figures in the image 400 to be processed are the foreground image region 410, and the other regions are the background image region 420.

[0067] The background image region 420 includes a first sub-region 421 and a second sub-region 422. From... Figure 4 As can be seen, the first sub-region 421 is close to the foreground image region 410, and the first smoothing window 430 is used to perform window smoothing on the first sub-region 421. The second sub-region 422 is far away from the foreground image region, and the second smoothing window 440 is used to perform window smoothing on the second sub-region 422. Furthermore, the window size of the first smoothing window 430 is smaller than the window size of the second smoothing window 440.

[0068] The image processing method provided in this disclosure divides the background image region into a first sub-region and a second sub-region based on the distance from the pixel point to the region boundary of the foreground image region, and applies a first smoothing window and a second smoothing window of different sizes to perform smoothing processing respectively. The first sub-region (i.e., closer to the foreground image region) uses a smaller smoothing window, which helps to reduce the blurring of the foreground edge caused by the smoothing processing. The second sub-region (i.e., farther from the foreground image region) uses a larger smoothing window, which helps to smooth the background region more effectively, reduce noise and texture changes, and make the background more uniform.

[0069] Thus, by using two different smoothing windows for window smoothing, effective background smoothing can be achieved while preserving foreground edges, which helps improve the overall image quality and makes it more suitable for subsequent image analysis, recognition, or detection tasks. Furthermore, the preset sizes of the first and second smoothing windows can be adjusted according to different images and application requirements, making this method applicable to various scenarios and achieving optimal processing results.

[0070] Step S320: Perform window smoothing processing on the background image region based on the target smoothing window to obtain the smoothed background image region.

[0071] After determining the target smoothing window corresponding to the background image region, window smoothing processing is performed on the background image region based on the target smoothing window to obtain the smoothed background image region.

[0072] In this embodiment of the disclosure, the background image region includes a first sub-region and a second sub-region. The first sub-region is close to the foreground image region, and the second sub-region is far from the foreground image region. A window smoothing process is performed on the first sub-region based on a first smoothing window to obtain a smoothed first sub-region, and a window smoothing process is performed on the second sub-region based on a second smoothing window to obtain a smoothed second sub-region.

[0073] Step S330: Update the background image region in the image to be processed using the smoothed background image region to obtain the preprocessed image.

[0074] The preprocessed image corresponding to the image to be processed includes the foreground image region and the smoothed background image region.

[0075] In this embodiment of the disclosure, the background image region includes a first sub-region and a second sub-region, and the preprocessed image corresponding to the image to be processed includes a foreground image region, a smoothed first sub-region, and a smoothed second sub-region.

[0076] Figure 5 A flowchart illustrating a method for window smoothing of a background image region based on a target smoothing window, provided by an embodiment of this disclosure, is shown. Figure 5 As shown, in some embodiments of this disclosure, step S320 above performs window smoothing processing on the background image region based on the target smoothing window to obtain the smoothed background image region, including:

[0077] Step S510: Based on the window size of the target smoothing window, traverse the background image region to obtain the image region to be smoothed.

[0078] In this embodiment of the disclosure, after determining the target smoothing window of the background image region, the window size of the target smoothing window is obtained. The background image region is traversed, and each time an image region with the same window size as the target smoothing window is taken out as the image region to be smoothed.

[0079] Step S520: For the image region to be smoothed, a preset calculation method is used to calculate the pixel value of the pixel points in the image region to be smoothed, obtain the pixel smoothing value corresponding to the image region to be smoothed, and replace the pixel value of the pixel points in the image region to be smoothed with the pixel smoothing value corresponding to the image region to be smoothed.

[0080] In this embodiment of the present disclosure, for an image region to be smoothed, a preset calculation method is used to calculate the pixel values ​​of all pixels in the image region to be smoothed in order to obtain the pixel smoothing value corresponding to the image region to be smoothed, and then the pixel smoothing value is used to replace the pixel values ​​of all pixels in the image region to be smoothed.

[0081] In some embodiments of this disclosure, the preset calculation method includes at least one of the following options: mean calculation, weighted mean calculation, median calculation, mode calculation, and Gaussian filtering.

[0082] The mean calculation refers to calculating the average value of all pixel values ​​within the area to be smoothed, and using this average value as the pixel smoothing value.

[0083] Weighted mean calculation is an extension of mean calculation, in which each pixel value in the region to be smoothed is assigned a weight, and a weighted average is calculated based on the pixel value weights and pixel values ​​of all pixels in the region to be smoothed. This weighted average is then used as the pixel smoothing value.

[0084] Median calculation is a non-linear smoothing method that sorts the pixel values ​​of all pixels within the region to be smoothed and takes the median value as the pixel smoothing value.

[0085] The mode calculation involves finding the pixel value that appears most frequently among all pixel values ​​in the region to be smoothed, and using it as the pixel smoothing value.

[0086] Gaussian filtering is a weighted average filtering method based on the Gaussian function. It assigns a weight to the pixel value of each pixel in the region to be smoothed. These weights are calculated based on the Gaussian function. Based on the assigned weights and pixel values, a Gaussian weighted average is calculated, and this Gaussian weighted average is used as the pixel smoothing value.

[0087] In this embodiment of the disclosure, any one or more of the above-described calculation methods can be selected to calculate the pixel values ​​of the pixels in the image region to be smoothed, thereby obtaining the pixel smoothing values ​​corresponding to the image region to be smoothed.

[0088] It should be noted that, in addition to the calculation methods such as mean calculation, weighted mean calculation, median calculation, mode calculation, and Gaussian filtering, other calculation methods can also be used, and this disclosure does not limit them.

[0089] The image processing method of this disclosure provides a variety of preset calculation methods that can be selected according to specific application scenarios and needs, so as to ensure that the best results can be achieved when processing different types of images.

[0090] In some embodiments of this disclosure, step S420 above uses a preset calculation method to calculate the pixel values ​​of pixels in the image region to be smoothed, obtain the pixel smoothing value corresponding to the image region to be smoothed, and replace the pixel values ​​of pixels in the image region to be smoothed with the pixel smoothing value corresponding to the image region to be smoothed, including: obtaining the pixel values ​​of pixels in the image region to be smoothed in multiple channels; for each channel, using a preset calculation method, calculating the pixel values ​​of pixels in the image region to be smoothed in the channel, obtaining the pixel smoothing value corresponding to the image region to be smoothed in the channel; and replacing the pixel values ​​of pixels in the image region to be smoothed in the channel with the pixel smoothing value corresponding to the image region to be smoothed in the channel.

[0091] In this embodiment of the present disclosure, for the image region to be smoothed, a preset calculation method is adopted to calculate the pixel smoothing value corresponding to the image region to be smoothed in multiple channels, that is, to calculate the pixel smoothing value corresponding to the image region to be smoothed in each channel, and then use the pixel smoothing value corresponding to the image region to be smoothed in each channel to replace the pixel value of the image region to be smoothed in that channel.

[0092] The image processing method provided in this disclosure analyzes each channel when processing the image region to be smoothed, calculates the pixel smoothing value corresponding to the image region to be smoothed in each channel, and then replaces the pixel value of the image region to be smoothed in each channel with the pixel smoothing value corresponding to the image region to be smoothed in each channel, so that each channel in the image can be processed independently, thereby preserving and enhancing the detail information in the image.

[0093] Step S530: After traversing the background image region, the smoothed background image region is obtained.

[0094] In this embodiment of the disclosure, after traversing the background image region, it is explained that a smoothing window process is performed on the background image region to obtain a smoothed background image region.

[0095] by Figure 4Taking an example, a first smoothing window 430 is used to perform window smoothing on the first sub-region 421. Starting from the upper left corner of the first sub-region 421, an image region with the same window size as the first smoothing window 430 is selected as the first image region to be smoothed. The pixel smoothing value corresponding to the first image region to be smoothed is calculated, and the pixel value of the pixels in the first image region to be smoothed is replaced with this pixel smoothing value. Then, an image region adjacent to the first image region to be smoothed and with the same window size as the first smoothing window 430 is selected as the second image region to be smoothed. The pixel smoothing value corresponding to the second image region to be smoothed is calculated, and the pixel value of the pixels in the second image region to be smoothed is replaced with this pixel smoothing value. This process is repeated to complete the window smoothing on the first sub-region 421.

[0096] by Figure 4 Taking this example, a second smoothing window 440 is used to perform window smoothing on the second sub-region 422. Starting from the upper left corner of the second sub-region 422, an image region with the same window size as the second smoothing window 440 is selected as the first image region to be smoothed. The pixel smoothing value corresponding to the first image region to be smoothed is calculated, and the pixel value of the pixels in the first image region to be smoothed is replaced with this pixel smoothing value. Then, an image region adjacent to the first image region to be smoothed and with the same window size as the second smoothing window 440 is selected as the second image region to be smoothed. The pixel smoothing value corresponding to the second image region to be smoothed is calculated, and the pixel value of the pixels in the second image region to be smoothed is replaced with this pixel smoothing value. This process is repeated to complete the window smoothing on the second sub-region 422.

[0097] The image processing method provided in this disclosure determines a target smoothing window based on the distance from pixels in the background image region to the boundary of the foreground image region, and smooths the background image region based on the target smoothing window. This significantly reduces noise and texture variations in the background, resulting in a smoother background. Furthermore, updating the background portion of the image to be processed using the smoothed background image region yields a preprocessed image. This preprocessed image achieves a better balance between background smoothness and foreground sharpness, improving the overall quality of the preprocessed image. This provides better input for subsequent image analysis, recognition, or detection tasks, with minimal impact on the accuracy of subsequent tasks, thus contributing to improved accuracy and efficiency.

[0098] It should be noted that the acquisition, storage, use, and processing of data in this disclosed technical solution comply with the relevant provisions of national laws and regulations. The various types of data, such as personal identity data, operational data, and behavioral data related to individuals, customers, and groups, obtained in the embodiments of this disclosure have all been authorized.

[0099] Based on the same inventive concept, this disclosure also provides an image processing apparatus, as described in the following embodiments. Since the principle by which this apparatus solves the problem is similar to that of the method embodiments described above, the implementation of this apparatus embodiment can refer to the implementation of the method embodiments described above, and repeated details will not be repeated.

[0100] Figure 6 This diagram illustrates an image processing apparatus according to an embodiment of the present disclosure, such as... Figure 6 As shown, the device 600 includes: a visual recognition module 610, a region determination module 620, a window smoothing module 630, an encoding / decoding module 640, and a task processing module 650.

[0101] The visual recognition module 610 is configured to perform visual recognition on the image to be processed to obtain the foreground image region of the image to be processed. The region determination module 620 is configured to determine the region in the image to be processed, excluding the foreground image region, as the background image region of the image to be processed. The window smoothing module 630 is configured to perform dynamic window smoothing processing on the background image region to obtain the preprocessed image corresponding to the image to be processed. The encoding and decoding module 640 is configured to perform image encoding and decoding processing on the preprocessed image to obtain the encoded and decoded image. The task processing module 650 is configured to process the encoded and decoded image based on the task network to obtain the task result of the image to be processed.

[0102] In some embodiments of this disclosure, the window smoothing module 630 is further configured to: determine a target smoothing window corresponding to the background image region based on the distance from the pixel point in the background image region to the region boundary of the foreground image region; perform window smoothing processing on the background image region based on the target smoothing window to obtain a smoothed background image region; and update the background image region in the image to be processed using the smoothed background image region to obtain a preprocessed image.

[0103] In some embodiments of this disclosure, the window smoothing module 630 is further configured to: divide the background image region into a first sub-region and a second sub-region based on the distance between pixels in the background image region and the region boundary of the foreground image region; the distance between pixels in the first sub-region and the region boundary of the foreground image region is less than or equal to a preset distance threshold, and the distance between pixels in the second sub-region and the region boundary of the foreground image region is greater than the preset distance threshold; determine a first smoothing window corresponding to the first sub-region, and determine a second smoothing window corresponding to the second sub-region; the first smoothing window is used to perform window smoothing processing on the first sub-region, and the second smoothing window is used to perform window smoothing processing on the second sub-region.

[0104] In some embodiments of this disclosure, the window size of the first smoothing window and the window size of the second smoothing window are preset, and the window size of the first smoothing window is smaller than the window size of the second smoothing window.

[0105] In some embodiments of this disclosure, the window smoothing module 630 is further configured to: traverse the background image region based on the window size of the target smoothing window to obtain the image region to be smoothed; for the image region to be smoothed, use a preset calculation method to calculate the pixel value of the pixel points in the image region to be smoothed to obtain the pixel smoothing value corresponding to the image region to be smoothed, and replace the pixel value of the pixel points in the image region to be smoothed with the pixel smoothing value corresponding to the image region to be smoothed; after traversing the background image region, obtain the smoothed background image region.

[0106] In some embodiments of this disclosure, the window smoothing module 630 is further configured to: obtain the pixel values ​​of pixels in the region to be smoothed in multiple channels; for each channel, calculate the pixel values ​​of pixels in the region to be smoothed in the channel using a preset calculation method to obtain the pixel smoothing value of the region to be smoothed in the channel; and replace the pixel values ​​of pixels in the region to be smoothed in the channel with the pixel smoothing value of the region to be smoothed in the channel.

[0107] In some embodiments of this disclosure, the preset calculation method includes at least one of the following options: mean calculation, weighted mean calculation, median calculation, mode calculation, and Gaussian filtering.

[0108] The image processing apparatus provided in this disclosure obtains a foreground image region and a background image region by performing visual recognition on the image to be processed. That is, it distinguishes the region of interest (i.e., the foreground image region) and the region of no interest (i.e., the background image region) in the image to be processed. Based on this, dynamic window smoothing processing is performed on the background image region, which effectively reduces redundant information and noise in the background image region and reduces the amount of information in the region of no interest. This enables a higher compression ratio to be achieved during the encoding and decoding process, enhances the contrast between the region of interest and the region of no interest, and can better preserve the feature information that is important to subsequent tasks (i.e., the key features of the foreground image region) while ensuring encoding and decoding efficiency. It has little impact on the accuracy of subsequent tasks and helps to improve the accuracy and efficiency of the task.

[0109] It should be noted that the examples and application scenarios implemented by the modules in the above device embodiments and the corresponding steps in the method embodiments are the same, but are not limited to the content disclosed in the above method embodiments. It should also be noted that the above modules, as part of the device, can be executed in a computer system such as a set of computer-executable instructions.

[0110] Those skilled in the art will understand that various aspects of the present invention can be implemented as systems, methods, or program products. Therefore, various aspects of the present invention can be specifically implemented in the following forms: entirely hardware implementations, entirely software implementations (including firmware, microcode, etc.), or implementations combining hardware and software aspects, collectively referred to herein as “circuits,” “modules,” or “systems.”

[0111] Based on the same inventive concept, this disclosure also provides an electronic device, which includes: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the image processing method described above by executing the executable instructions. Since the principle by which this electronic device solves the problem is similar to that of the above method embodiments, the implementation of this electronic device embodiment can refer to the implementation of the above method embodiments, and repeated details will not be described again.

[0112] The following reference Figure 7 To describe an electronic device 700 according to such an embodiment of the present disclosure. Figure 7 The electronic device 700 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0113] like Figure 7 As shown, the electronic device 700 is manifested in the form of a general-purpose computing device. The components of the electronic device 700 may include, but are not limited to: at least one processing unit 710, at least one storage unit 720, and a bus 730 connecting different system components (including storage unit 720 and processing unit 710).

[0114] The storage unit stores program code that can be executed by the processing unit 710, causing the processing unit 710 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure. For example, the processing unit 710 can perform the following steps of the above method embodiment: Step S210, performing visual recognition on the image to be processed to obtain the foreground image region of the image to be processed; Step S220, determining the region in the image to be processed other than the foreground image region as the background image region of the image to be processed; Step S230, performing dynamic window smoothing processing on the background image region to obtain a preprocessed image corresponding to the image to be processed; Step S240, performing image encoding and decoding processing on the preprocessed image to obtain an encoded and decoded image; Step S250, processing the encoded and decoded image based on the task network to obtain the task result of the image to be processed.

[0115] Storage unit 720 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 7201 and / or cache memory 7202, and may further include a read-only memory (ROM) 7203.

[0116] The storage unit 720 may also include a program / utility 7204 having a set (at least one) program module 7205, such program module 7205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0117] Bus 730 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0118] Electronic device 700 can also communicate with one or more external devices 740 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 700, and / or with any device that enables electronic device 700 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 750. Furthermore, electronic device 700 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 760. As shown, network adapter 760 communicates with other modules of electronic device 700 via bus 730. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0119] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0120] Based on the same inventive concept, this disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the image processing method described above. Since the principle by which this computer-readable storage medium embodiment solves the problem is similar to that of the above method embodiments, the implementation of this computer-readable storage medium embodiment can refer to the implementation of the above method embodiments, and repeated details will not be elaborated further.

[0121] More specific examples of computer-readable storage media in this disclosure may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0122] In this disclosure, a computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting a program for use by or in connection with an instruction execution system, apparatus, or device.

[0123] Optionally, the program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0124] In practical implementation, program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0125] Based on the same inventive concept, this disclosure also provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the image processing method of any one of the above method embodiments. Since the principle by which this computer program product embodiment solves the problem is similar to that of the above method embodiments, the implementation of this computer program product embodiment can refer to the implementation of the above method embodiments, and repeated details will not be elaborated further.

[0126] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0127] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0128] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0129] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. An image processing method, characterized in that, include: Visual recognition is performed on the image to be processed to obtain the foreground image region of the image to be processed; The region in the image to be processed, excluding the foreground image region, is defined as the background image region of the image to be processed. Dynamic window smoothing is performed on the background image region to obtain a preprocessed image corresponding to the image to be processed; wherein, the background image region includes a first sub-region and a second sub-region; the first sub-region is close to the foreground image region; a first smoothing window is used for the first sub-region; the second sub-region is far from the foreground image region; a second smoothing window is used for the second sub-region; the window size of the first smoothing window is smaller than the window size of the second smoothing window; The preprocessed image is subjected to image encoding and decoding processing to obtain the encoded and decoded image; The encoded and decoded image is processed based on the task network to obtain the task result of the image to be processed.

2. The method according to claim 1, characterized in that, The step of performing dynamic window smoothing on the background image region to obtain the preprocessed image corresponding to the image to be processed includes: The target smoothing window corresponding to the background image region is determined based on the distance from the pixel point in the background image region to the region boundary of the foreground image region; the target smoothing window includes the first smoothing window and the second smoothing window; Based on the target smoothing window, the background image region is subjected to window smoothing processing to obtain the smoothed background image region; The background image region in the image to be processed is updated using the smoothed background image region to obtain the preprocessed image.

3. The method according to claim 2, characterized in that, Determining the target smoothing window corresponding to the background image region based on the distance from a pixel in the background image region to the region boundary of the foreground image region includes: Based on the distance from the pixels in the background image region to the boundary of the foreground image region, the background image region is divided into a first sub-region and a second sub-region; the distance between the pixels in the first sub-region and the boundary of the foreground image region is less than or equal to a preset distance threshold, and the distance between the pixels in the second sub-region and the boundary of the foreground image region is greater than the preset distance threshold. Determine the first smoothing window corresponding to the first sub-region, and determine the second smoothing window corresponding to the second sub-region.

4. The method according to claim 3, characterized in that, The window sizes of the first smooth window and the second smooth window are preset.

5. The method according to claim 2 or 3, characterized in that, The step of performing window smoothing processing on the background image region based on the target smoothing window to obtain a smoothed background image region includes: Based on the window size of the target smoothing window, the background image region is traversed to obtain the image region to be smoothed; For the image region to be smoothed, a preset calculation method is used to calculate the pixel value of the pixel in the image region to be smoothed, and obtain the pixel smoothing value corresponding to the image region to be smoothed. The pixel value of the pixel in the image region to be smoothed is then replaced with the pixel smoothing value corresponding to the image region to be smoothed. After traversing the background image region, the smoothed background image region is obtained.

6. The method according to claim 5, characterized in that, The step of using a preset calculation method to calculate the pixel values ​​of pixels within the image region to be smoothed, obtaining the pixel smoothing value corresponding to the image region to be smoothed, and replacing the pixel values ​​of pixels within the image region to be smoothed with the pixel smoothing value corresponding to the image region to be smoothed includes: Obtain the pixel values ​​of the pixels in the image region to be smoothed in multiple channels; For each channel, the preset calculation method is used to calculate the pixel value of the pixel in the image region to be smoothed on the channel, and obtain the pixel smoothing value of the image region to be smoothed on the channel; the pixel value of the pixel in the image region to be smoothed on the channel is replaced by the pixel smoothing value of the image region to be smoothed on the channel.

7. The method according to claim 5, characterized in that, The preset calculation method includes at least one of the following options: mean calculation, weighted mean calculation, median calculation, mode calculation, and Gaussian filtering.

8. An image processing apparatus, characterized in that, include: The visual recognition module is configured to perform visual recognition on the image to be processed to obtain the foreground image region of the image to be processed; The region determination module is configured to determine the region in the image to be processed, excluding the foreground image region, as the background image region of the image to be processed. A window smoothing module is configured to perform dynamic window smoothing processing on the background image region to obtain a preprocessed image corresponding to the image to be processed; wherein, the background image region includes a first sub-region and a second sub-region; the first sub-region is close to the foreground image region; a first smoothing window is used for the first sub-region; the second sub-region is far from the foreground image region; a second smoothing window is used for the second sub-region; the window size of the first smoothing window is smaller than the window size of the second smoothing window; The encoding / decoding module is configured to perform image encoding / decoding processing on the preprocessed image to obtain the encoded / decoded image; The task processing module is configured to process the encoded and decoded image based on the task network to obtain the task result of the image to be processed.

9. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the image processing method of any one of claims 1 to 7 by executing the executable instructions.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the image processing method according to any one of claims 1 to 7.

11. A computer program product, comprising: A computer program or instruction, characterized in that, when executed by a processor, the computer program or instruction implements the image processing method according to any one of claims 1 to 7.

Images