Image processing method and device, electronic equipment and storage medium

Abstract

The invention provides an image processing method and device, electronic equipment and a storage medium, and relates to the technical field of vision. The image processing method comprises the following steps: performing visual identification on a to-be-processed image to obtain a foreground image area of the to-be-processed image; determining an area except the foreground image area in the to-be-processed image as a background image area of the to-be-processed image; performing dynamic window smoothing processing on the background image area to obtain a preprocessed image corresponding to the to-be-processed image; carrying out image coding and decoding processing on the preprocessed image to obtain a coded and decoded image; and processing the encoded and decoded image based on the task network to obtain a task result of the to-be-processed image. According to the method, the information amount of the non-interested area can be reduced, the contrast ratio between the interested area and the non-interested area is enhanced while the compression ratio is increased, and the influence on the precision of a subsequent task is small.

Description

Background Art

[0002] There are significant differences between video compression coding for human vision and machine vision in application scenarios, evaluation mechanisms, information processing and information use. The establishment of the Video Coding for Machines (VCM) standard group marks that video compression coding technology has begun to pay attention to the needs of the machine vision field and study compression coding technology suitable for machine vision and human-machine mixed vision scenarios to improve performance and efficiency in intelligent applications.

[0003] In the related art, an image for an intelligent task is obtained, and the image is directly encoded and decoded to obtain the encoded and decoded image, and then the encoded and decoded image is input into the intelligent task network to complete the subsequent task. However, since the original image is directly encoded and decoded, the performance of the encoded and decoded image in the subsequent task is not considered, which may result in the loss of feature information important to the task, affecting the accuracy and efficiency of the task.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The present disclosure provides an image processing method, an image processing device, an electronic device, a computer-readable storage medium, and a computer program product, which at least to some extent overcome the problems existing in the related art.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] According to one aspect of the present disclosure, there is provided an image processing method, including: performing visual recognition on an image to be processed to obtain a foreground image area of ​​the image to be processed; determining an area in the image to be processed other than the foreground image area as a background image area of ​​the image to be processed; performing dynamic window smoothing on the background image area to obtain a preprocessed image corresponding to the image to be processed; performing image encoding and decoding on the preprocessed image to obtain an image after encoding and decoding; and processing the image after encoding and decoding based on a task network to obtain a task result of the image to be processed.

[0008] In some embodiments of the present disclosure, the dynamic window smoothing processing is performed on the background image area to obtain a preprocessed image corresponding to the image to be processed, including: determining a target smoothing window corresponding to the background image area based on the distance from the pixel points in the background image area to the area boundary of the foreground image area; performing window smoothing processing on the background image area based on the target smoothing window to obtain a smoothed background image area; and using the smoothed background image area to update the background image area in the image to be processed to obtain the preprocessed image.

[0009] In some embodiments of the present disclosure, determining the target smoothing window corresponding to the background image area according to the distance from the pixel points in the background image area to the area boundary of the foreground image area includes: dividing the background image area into a first sub-area and a second sub-area according to the distance from the pixel points in the background image area to the area boundary of the foreground image area; the distance between the pixel points in the first sub-area and the area boundary of the foreground image area is less than or equal to a preset distance threshold, and the distance between the pixel points in the second sub-area and the area boundary of the foreground image area is greater than the preset distance threshold; determining a first smoothing window corresponding to the first sub-area, and determining a second smoothing window corresponding to the second sub-area; the first smoothing window is used to perform window smoothing processing on the first sub-area, and the second smoothing window is used to perform window smoothing processing on the second sub-area.

[0010] In some embodiments of the present disclosure, the window size of the first smoothing window and the window size of the second smoothing window are preset, and the window size of the first smoothing window is smaller than the window size of the second smoothing window.

[0011] In some embodiments of the present disclosure, the window smoothing processing is performed on the background image area based on the target smoothing window to obtain the background image area after the smoothing processing, including: based on the window size of the target smoothing window, traversing the background image area to obtain the image area to be smoothed; for the image area to be smoothed, using a preset calculation method, calculating the pixel values ​​of the pixel points in the image area to be smoothed to obtain the pixel smoothing values ​​corresponding to the image area to be smoothed, and replacing the pixel values ​​of the pixel points in the image area to be smoothed with the pixel smoothing values ​​corresponding to the image area to be smoothed; after completing the traversal of the background image area, the background image area after the smoothing processing is obtained.

[0012] In some embodiments of the present disclosure, the pixel values ​​of the pixels in the image area to be smoothed are calculated using a preset calculation method to obtain the pixel smoothing value corresponding to the image area to be smoothed, and the pixel smoothing value corresponding to the image area to be smoothed is used to replace the pixel values ​​of the pixels in the image area to be smoothed, including: obtaining the pixel values ​​of the pixels in the image area to be smoothed on multiple channels; for each channel, using the preset calculation method to calculate the pixel values ​​of the pixels in the image area to be smoothed on the channel to obtain the pixel smoothing value corresponding to the image area to be smoothed on the channel; replacing the pixel values ​​of the pixels in the image area to be smoothed on the channel with the pixel smoothing value corresponding to the image area to be smoothed on the channel.

[0013] In some embodiments of the present disclosure, the preset calculation method includes at least one of the following options: mean calculation, weighted mean calculation, median calculation, mode calculation and Gaussian filtering.

[0014] According to another aspect of the present disclosure, an image processing device is also provided, including: a visual recognition module, configured to perform visual recognition on an image to be processed to obtain a foreground image area of ​​the image to be processed; a region determination module, configured to determine an area other than the foreground image area in the image to be processed as a background image area of ​​the image to be processed; a window smoothing module, configured to perform dynamic window smoothing on the background image area to obtain a preprocessed image corresponding to the image to be processed; a codec module, configured to perform image codec processing on the preprocessed image to obtain an image after codec processing; and a task processing module, configured to process the image after codec processing based on a task network to obtain a task result of the image to be processed.

[0015] According to another aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the above-mentioned image processing methods by executing the executable instructions.

[0016] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the image processing method described in any one of the above is implemented.

[0017] According to another aspect of the present disclosure, a computer program product is provided, including: a computer program or instructions, wherein when the computer program or instructions are executed by a processor, any one of the above-mentioned image processing methods is implemented.

[0018] The image processing method provided in the embodiments of the present disclosure obtains a foreground image area and a background image area by performing visual recognition on the image to be processed, that is, distinguishes the area of ​​interest (that is, the foreground image area) and the area of ​​no interest (that is, the background image area) in the image to be processed, and on this basis, performs dynamic window smoothing on the background image area, effectively reducing redundant information and noise in the background image area, and reducing the amount of information in the area of ​​no interest, so that a higher compression ratio can be achieved during the encoding and decoding process, and the contrast between the area of ​​interest and the area of ​​no interest is enhanced. While ensuring the encoding and decoding efficiency, the feature information that is important to the subsequent tasks (that is, the key features of the foreground image area) can be better retained, and the accuracy of the subsequent tasks has little impact, which helps to improve the accuracy and efficiency of the tasks.

[0019] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0021] Figure 1 A schematic diagram of an exemplary application system architecture to which the image processing method in the embodiment of the present disclosure can be applied is shown;

[0022] Figure 2 A flow chart of an image processing method provided by an embodiment of the present disclosure is shown;

[0023] Figure 3 A flow chart of a method for performing dynamic window smoothing processing on a background image area provided by an embodiment of the present disclosure is shown;

[0024] Figure 4 A schematic diagram of performing dynamic window smoothing processing on a background image in an image to be processed provided by an embodiment of the present disclosure is shown;

[0025] Figure 5 A flow chart of a method for performing window smoothing processing on a background image area based on a target smoothing window provided by an embodiment of the present disclosure is shown;

[0026] Figure 6 A schematic diagram of an image processing device according to an embodiment of the present disclosure is shown;

[0027] Figure 7 A structural block diagram of an electronic device in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0028] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0029] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0030] Figure 1 FIG. 2 shows an exemplary application system architecture diagram to which the image processing method in the embodiment of the present disclosure can be applied. Figure 1 As shown, the system architecture may include a terminal device 101 , a network 102 and a server 103 .

[0031] The network 102 is a medium for providing a communication link between the terminal device 101 and the server 103, and can be a wired network or a wireless network.

[0032] Optionally, the wireless network or wired network described above uses standard communication technology and / or protocol. The network is typically the Internet, but may also be any network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a dedicated network or any combination of a virtual private network). In some embodiments, technologies and / or formats including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. are used to represent data exchanged over the network. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPSec) and the like may be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies may also be used to replace or supplement the above data communication technologies.

[0033] The terminal device 101 can be various electronic devices, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, smart speakers, smart watches, wearable devices, augmented reality devices, virtual reality devices, etc.

[0034] Optionally, the client of the application installed in different terminal devices 101 is the same, or the client of the same type of application based on different operating systems. Based on the different terminal platforms, the specific form of the client of the application can also be different, for example, the application client can be a mobile client, a PC client, etc.

[0035] The server 103 may be a server that provides various services, such as a background management server that provides support for the device operated by the user using the terminal device 101. The background management server may analyze and process the received request and other data, and feed back the processing results to the terminal device.

[0036] Optionally, the server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), as well as big data and artificial intelligence platforms.

[0037] Those skilled in the art will know that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration, and any number of terminal devices, networks and servers may be provided according to actual needs, and the embodiments of the present disclosure do not limit this.

[0038] Under the above system architecture, an image processing method is provided in an embodiment of the present disclosure. The method can be executed by any electronic device with computing and processing capabilities.

[0039] In some embodiments, the image processing method provided in the embodiments of the present disclosure can be executed by a terminal device of the above-mentioned system architecture; in other embodiments, the image processing method provided in the embodiments of the present disclosure can be executed by a server in the above-mentioned system architecture; in other embodiments, the image processing method provided in the embodiments of the present disclosure can be implemented by the terminal device and the server in the above-mentioned system architecture through interaction.

[0040] Figure 2 A flow chart of an image processing method provided by an embodiment of the present disclosure is shown as follows: Figure 2 As shown, the image processing method provided in the embodiment of the present disclosure includes the following steps.

[0041] Step S210: performing visual recognition on the image to be processed to obtain a foreground image region of the image to be processed.

[0042] In the embodiments of the present disclosure, the image to be processed can be understood as an image used for intelligent task processing, wherein the image to be processed can be a static image, such as a photo, or a dynamic image, such as a video frame.

[0043] In the disclosed embodiment, the foreground image area may be referred to as an area of ​​interest, which may include the main object or area of ​​interest in the image; the background image area may be referred to as an area of ​​no interest, which may include content other than the main object in the image.

[0044] In a possible implementation, the image to be processed is input into a visual recognition module, which includes an intelligent analysis algorithm for identifying key elements or features in the image. The visual recognition module outputs a visual recognition result, which includes the foreground target information of interest, such as the location, shape, size and other attribute information of the foreground target. Then, the foreground image area in the image to be processed is determined according to the visual recognition result.

[0045] In another possible implementation, the Faster RCNN X101-FPN instance segmentation model is used for pre-reasoning of machine vision tasks. Among them, Faster RCNN is an advanced deep learning model for target detection and instance segmentation; X101-FPN is a variant of Faster RCNN, X101 means using ResNeXt-101 as a feature extraction network, and FPN means using a feature pyramid network to enhance the model's detection capabilities for targets of different scales. The Faster RCNNX101-FPN instance segmentation model outputs the segmentation results of the image to be processed, which is recorded as a set S, where the set S contains all pixels or regions in the image to be processed that are identified as foreground target areas. Through the output of the instance segmentation model, the foreground image area in the image to be processed can be determined.

[0046] Step S220: determining the area other than the foreground image area in the image to be processed as the background image area of ​​the image to be processed.

[0047] In the disclosed embodiment, the foreground image area is regarded as a subset of the image to be processed, and the foreground image area is inverted to obtain the area of ​​the image to be processed except the foreground image area, that is, the background image area.

[0048] Step S230, performing dynamic window smoothing processing on the background image area to obtain a pre-processed image corresponding to the image to be processed.

[0049] In the embodiment of the present disclosure, after obtaining the background image area, the background image area is smoothed using a dynamic window to complete the preprocessing operation of the image to be processed and obtain a preprocessed image. The dynamic window smoothing process can be adaptively adjusted according to the different characteristics of the background image area, that is, the window used for different parts of the background image area can be different, so that the preprocessed image is smoother and more uniform.

[0050] Step S240, performing image encoding and decoding processing on the pre-processed image to obtain an image after the encoding and decoding processing.

[0051] In the disclosed embodiment, image coding includes image encoding and image decompression, which is a compression and decompression technology that can compress image data into a smaller file size for storage or transmission. By performing image coding and decoding on the pre-processed image, the storage and transmission requirements of the image data can be reduced, and the image quality can be maintained as much as possible.

[0052] Step S250, processing the image after encoding and decoding based on the task network to obtain the task result of the image to be processed.

[0053] In the disclosed embodiment, the task network may be a deep learning model, a machine learning algorithm, etc. The task network may perform specific tasks according to the content of the image, such as object recognition, classification, detection, etc. After obtaining the image after encoding and decoding, the image after encoding and decoding is input into the task network, and the task result of the image to be processed is output, such as the recognized object name, classification label, location information, etc.

[0054] The image processing method provided in the embodiments of the present disclosure obtains a foreground image area and a background image area by performing visual recognition on the image to be processed, that is, distinguishes the area of ​​interest (that is, the foreground image area) and the area of ​​no interest (that is, the background image area) in the image to be processed, and on this basis, performs dynamic window smoothing on the background image area, effectively reducing redundant information and noise in the background image area, and reducing the amount of information in the area of ​​no interest, so that a higher compression ratio can be achieved during the encoding and decoding process, and the contrast between the area of ​​interest and the area of ​​no interest is enhanced. While ensuring the encoding and decoding efficiency, the feature information that is important to the subsequent tasks (that is, the key features of the foreground image area) can be better retained, and the accuracy of the subsequent tasks has little impact, which helps to improve the accuracy and efficiency of the tasks.

[0055] Figure 3 FIG. 2 shows a flow chart of a method for performing dynamic window smoothing processing on a background image area provided by an embodiment of the present disclosure. Figure 3 As shown, in some embodiments of the present disclosure, the above step S230 performs dynamic window smoothing processing on the background image area to obtain a preprocessed image corresponding to the image to be processed, including the following steps.

[0056] Step S310, determining a target smoothing window corresponding to the background image area according to the distance from the pixel point in the background image area to the area boundary of the foreground image area.

[0057] The target smoothing window refers to the window used for smoothing the background image area. The size of the target smoothing window can be determined according to the positional relationship between the background image area and the foreground image area. For example, a small-sized smoothing window is used for the background image area close to the foreground image area, and a large-sized smoothing window is used for the background image area far from the foreground image area. The shape of the target smoothing window can be set according to experience or demand. For example, the target smoothing window is rectangular, circular or other shapes.

[0058] In the disclosed embodiment, after determining the background image area and the foreground image area in the image to be processed, the pixel points in the background image area are analyzed, the distance from the pixel points to the area boundary of the foreground image area is calculated, and the target smoothing window is determined based on the calculated distance.

[0059] In one implementation of the present disclosure, the distance from a pixel point in the background image area to the area boundary of the foreground image area refers to the shortest distance from a pixel point in the background image area to the area boundary of the foreground image area. For example, the foreground image area is a rectangle with four boundaries. The distances from the pixel point P1 in the background image area to the four boundaries are calculated respectively, and the minimum value is selected as the distance from the pixel point P1 to the area boundary of the foreground image area.

[0060] In some embodiments of the present disclosure, the above-mentioned step S310 determines the target smoothing window corresponding to the background image area according to the distance from the pixel points in the background image area to the regional boundary of the foreground image area, including: dividing the background image area into a first sub-area and a second sub-area according to the distance from the pixel points in the background image area to the regional boundary of the foreground image area; determining a first smoothing window corresponding to the first sub-area, and determining a second smoothing window corresponding to the second sub-area.

[0061] Among them, the distance between the pixel points in the first sub-area and the area boundary of the foreground image area is less than or equal to the preset distance threshold, and the distance between the pixel points in the second sub-area and the area boundary of the foreground image area is greater than the preset distance threshold; the first smoothing window is used to perform window smoothing processing on the first sub-area, and the second smoothing window is used to perform window smoothing processing on the second sub-area.

[0062] In some embodiments of the present disclosure, the window size of the first smoothing window and the window size of the second smoothing window are preset, and the window size of the first smoothing window is smaller than the window size of the second smoothing window.

[0063] In the embodiment of the present disclosure, the window size can be set to two levels, that is, a first smoothing window and a second smoothing window are pre-set, and the window size of the first smoothing window is set smaller than the window size of the second smoothing window.

[0064] In a possible implementation, the window size can be set to n*n for the first smoothing window and 2n*2n for the second smoothing window, where n is greater than or equal to 1 and can be set according to actual needs, specific application scenarios, etc. For example, if the image quality of the background image area is required to meet the standard in the subsequent task, n can be set to a smaller value, such as n is set to 2. If the subsequent task is related to the detection of the foreground image area and does not focus on the content of the background image area, n can be set to a larger value, such as n is set to 3.

[0065] In the disclosed embodiment, when determining the target smoothing window corresponding to the background image area, for the part of the background image area close to the foreground image area, the first smoothing window can be used to perform window smoothing processing, and for the part of the background image area that is not the foreground image area, the second smoothing window can be used to perform window smoothing processing. Specifically, the pixel points in the background image area are analyzed, and if the distance from the pixel point to the area boundary of the foreground image area is less than or equal to the preset distance, the first smoothing window is used to perform window smoothing processing, and if the distance from the pixel point to the area boundary of the foreground image area is greater than the preset distance, the second smoothing window is used to perform window smoothing processing.

[0066] Figure 4 FIG. 2 is a schematic diagram showing a dynamic window smoothing process for a background image in an image to be processed provided by an embodiment of the present disclosure. Figure 4 As shown, two character regions in the image to be processed 400 are foreground image regions 410 , and the other regions are background image regions 420 .

[0067] The background image area 420 includes a first sub-area 421 and a second sub-area 422. Figure 4 It can be seen that the first sub-region 421 is close to the foreground image region 410, and the first smoothing window 430 is used to perform window smoothing on the first sub-region 421. The second sub-region 422 is far away from the foreground image region, and the second smoothing window 440 is used to perform window smoothing on the second sub-region 422. In addition, the window size of the first smoothing window 430 is smaller than the window size of the second smoothing window 440.

[0068] The image processing method provided by the embodiment of the present disclosure divides the background image area into a first sub-area and a second sub-area according to the distance from the pixel point to the area boundary of the foreground image area, and applies a first smoothing window and a second smoothing window of different sizes for smoothing processing, respectively. The first sub-area (i.e., close to the foreground image area) uses a smaller smoothing window, which helps to reduce the foreground edge blur caused by the smoothing processing, and the second sub-area (i.e., far from the foreground image area) uses a larger smoothing window, which helps to more effectively smooth the background area, reduce noise and texture changes, and make the background more uniform.

[0069] In this way, by performing window smoothing processing using two different smoothing windows, effective background smoothing can be achieved while protecting the foreground edge, which helps to improve the quality of the overall image and make it more suitable for subsequent image analysis, recognition or detection tasks. In addition, the sizes of the preset first smoothing window and the second smoothing window can be adjusted according to different images and application requirements, so that the method can be applied to various scenarios and achieve the best processing effect.

[0070] Step S320 , performing window smoothing processing on the background image area based on the target smoothing window to obtain a background image area after the smoothing processing.

[0071] After determining the target smoothing window corresponding to the background image region, the background image region is subjected to window smoothing processing based on the target smoothing window to obtain the background image region after the smoothing processing.

[0072] In the embodiment of the present disclosure, the background image region includes a first sub-region and a second sub-region, the first sub-region is close to the foreground image region, and the second sub-region is far from the foreground image region. The first sub-region is subjected to window smoothing based on a first smoothing window to obtain the first sub-region after smoothing, and the second sub-region is subjected to window smoothing based on a second smoothing window to obtain the second sub-region after smoothing.

[0073] Step S330, using the smoothed background image area to update the background image area in the image to be processed to obtain a pre-processed image.

[0074] The pre-processed image corresponding to the image to be processed includes a foreground image area and a background image area after smoothing.

[0075] In the disclosed embodiment, the background image region includes a first sub-region and a second sub-region, and the pre-processed image corresponding to the image to be processed includes a foreground image region, a smoothed first sub-region, and a smoothed second sub-region.

[0076] Figure 5 FIG. 4 is a flowchart of a method for performing window smoothing processing on a background image area based on a target smoothing window provided by an embodiment of the present disclosure. Figure 5 As shown, in some embodiments of the present disclosure, the above step S320 performs window smoothing on the background image area based on the target smoothing window to obtain the background image area after smoothing, including:

[0077] Step S510: based on the window size of the target smoothing window, traverse the background image area to obtain the image area to be smoothed.

[0078] In the disclosed embodiment, after determining the target smoothing window of the background image area, the window size of the target smoothing window is obtained, and the background image area is traversed, and an image area with the same window size as the target smoothing window is taken out each time as the image area to be smoothed.

[0079] Step S520: For the image area to be smoothed, a preset calculation method is used to calculate the pixel values ​​of the pixel points in the image area to be smoothed, to obtain the pixel smoothing value corresponding to the image area to be smoothed, and the pixel smoothing value corresponding to the image area to be smoothed is used to replace the pixel values ​​of the pixel points in the image area to be smoothed.

[0080] In the embodiment of the present disclosure, for the image area to be smoothed, a preset calculation method is used to calculate the pixel values ​​of all pixel points in the image area to be smoothed to obtain the pixel smoothing value corresponding to the image area to be smoothed, and then the pixel smoothing value is used to replace the pixel values ​​of all pixel points in the image area to be smoothed.

[0081] In some embodiments of the present disclosure, the preset calculation method includes at least one of the following options: mean calculation, weighted mean calculation, median calculation, mode calculation and Gaussian filtering.

[0082] The mean calculation refers to calculating the average value of all pixel values ​​in the area to be smoothed, and using the average value as the pixel smoothing value.

[0083] Weighted mean calculation is an extension of mean calculation, in which each pixel value in the area to be smoothed is assigned a weight, and the weighted average value is calculated based on the pixel value weights and pixel values ​​of all pixels in the area to be smoothed, and this weighted average value is used as the pixel smoothing value.

[0084] Median calculation is a nonlinear smoothing method that sorts the pixel values ​​of all pixels in the smoothing area and takes the middle value as the pixel smoothing value after sorting.

[0085] The majority calculation is to find the pixel value that appears most frequently among the pixel values ​​of all the pixels in the area to be smoothed, and use it as the pixel smoothing value.

[0086] Gaussian filtering is a weighted mean filtering method based on the Gaussian function. A weight is assigned to the pixel value of each pixel in the area to be smoothed. These weights are calculated based on the Gaussian function. Based on the assigned weights and pixel values, the Gaussian weighted average is calculated and used as the pixel smoothing value.

[0087] In the disclosed embodiment, any one or more of the above combined calculation methods can be selected to calculate the pixel values ​​of the pixel points in the image area to be smoothed to obtain the pixel smoothing value corresponding to the image area to be smoothed.

[0088] It should be noted that in addition to the calculation methods such as mean calculation, weighted mean calculation, median calculation, mode calculation and Gaussian filtering, other calculation methods can also be used, and the embodiments of the present disclosure are not limited to this.

[0089] The image processing method of the embodiment of the present disclosure provides a variety of optional preset calculation methods, and can select a suitable calculation method according to specific application scenarios and needs, ensuring that the best effect can be achieved when processing different types of images.

[0090] In some embodiments of the present disclosure, the above-mentioned step S420 adopts a preset calculation method to calculate the pixel values ​​of the pixel points in the image area to be smoothed, obtain the pixel smoothing value corresponding to the image area to be smoothed, and replace the pixel values ​​of the pixel points in the image area to be smoothed with the pixel smoothing value corresponding to the image area to be smoothed, including: obtaining the pixel values ​​of the pixel points in the area to be smoothed on multiple channels; for each channel, using a preset calculation method, calculating the pixel values ​​of the pixel points in the image area to be smoothed on the channel, and obtaining the pixel smoothing value corresponding to the image area to be smoothed on the channel; replacing the pixel values ​​of the pixel points in the image area to be smoothed on the channel with the pixel smoothing value corresponding to the image area to be smoothed on the channel.

[0091] In the embodiment of the present disclosure, for the image area to be smoothed, a preset calculation method is used to calculate the pixel smoothing value corresponding to the image area to be smoothed in multiple channels, that is, the pixel smoothing value corresponding to the image area to be smoothed under each channel is calculated, and then the pixel smoothing value corresponding to the image area to be smoothed under each channel is used to replace the pixel value of the image area to be smoothed under the channel.

[0092] The image processing method provided by the embodiment of the present disclosure analyzes each channel when processing the image area to be smoothed, calculates the pixel smoothing value corresponding to the image area to be smoothed under each channel, and then uses the pixel smoothing value corresponding to the image area to be smoothed under each channel to replace the pixel value of the image area to be smoothed under the channel, so that each channel in the image can be processed independently, thereby retaining and enhancing the detail information in the image.

[0093] Step S530: After the traversal of the background image area is completed, a smoothed background image area is obtained.

[0094] In the embodiment of the present disclosure, after the traversal of the background image region is completed, it is described that a smoothing window process is completed on the background image region to obtain the background image region after the smoothing process.

[0095] by Figure 4For example, the first smoothing window 430 is used to perform window smoothing on the first sub-region 421. Starting from the upper left corner of the first sub-region 421, an image region with the same window size as the first smoothing window 430 is selected as the first image region to be smoothed, and the pixel smoothing value corresponding to the first image region to be smoothed is calculated, and the pixel smoothing value is used to replace the pixel value of the pixel point in the first image region to be smoothed. Then, an image region adjacent to the first image region to be smoothed and with the same window size as the first smoothing window 430 is selected as the second image region to be smoothed, and the pixel smoothing value corresponding to the second image region to be smoothed is calculated, and the pixel smoothing value is used to replace the pixel value of the pixel point in the second image region to be smoothed. By analogy, the window smoothing process is completed for the first sub-region 421.

[0096] by Figure 4 For example, the second smoothing window 440 is used to perform window smoothing on the second sub-region 422. Starting from the upper left corner of the second sub-region 422, an image region with the same window size as the second smoothing window 440 is selected as the first image region to be smoothed, and the pixel smoothing value corresponding to the first image region to be smoothed is calculated, and the pixel smoothing value is used to replace the pixel value of the pixel point in the first image region to be smoothed. Then, an image region adjacent to the first image region to be smoothed and with the same window size as the second smoothing window 440 is selected as the second image region to be smoothed, and the pixel smoothing value corresponding to the second image region to be smoothed is calculated, and the pixel smoothing value is used to replace the pixel value of the pixel point in the second image region to be smoothed. By analogy, the window smoothing process is completed for the second sub-region 422.

[0097] The image processing method provided by the embodiment of the present disclosure determines the target smoothing window according to the distance from the pixel point in the background image area to the boundary of the foreground image area, and performs smoothing processing on the background image area based on the target smoothing window, which can significantly reduce the noise and texture changes in the background, making the background smoother. In addition, by using the background image area after smoothing processing to update the background part in the image to be processed, a pre-processed image can be obtained, which achieves a better balance between background smoothness and foreground clarity, improves the overall quality of the pre-processed image, provides better input for subsequent image analysis, recognition or detection tasks, has less impact on the accuracy of subsequent tasks, and helps to improve the accuracy and efficiency of tasks.

[0098] It should be noted that the acquisition, storage, use, and processing of data in the technical solution disclosed in this disclosure are in compliance with the relevant provisions of national laws and regulations, and various types of data such as personal identity data, operation data, behavioral data, etc. related to individuals, customers, and groups obtained in the embodiments of this disclosure have all been authorized.

[0099] Based on the same inventive concept, an image processing device is also provided in the embodiment of the present disclosure, as described in the following embodiment. Since the principle of solving the problem in the device embodiment is similar to that in the above method embodiment, the implementation of the device embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be repeated.

[0100] Figure 6 A schematic diagram of an image processing device in an embodiment of the present disclosure is shown. Figure 6 As shown, the device 600 includes: a visual recognition module 610, a region determination module 620, a window smoothing module 630, a coding module 640 and a task processing module 650.

[0101] Among them, the visual recognition module 610 is configured to: perform visual recognition on the image to be processed to obtain the foreground image area of ​​the image to be processed. The area determination module 620 is configured to: determine the area other than the foreground image area in the image to be processed as the background image area of ​​the image to be processed. The window smoothing module 630 is configured to: perform dynamic window smoothing processing on the background image area to obtain a preprocessed image corresponding to the image to be processed. The codec module 640 is configured to: perform image codec processing on the preprocessed image to obtain the image after codec processing. The task processing module 650 is configured to: process the image after codec processing based on the task network to obtain the task result of the image to be processed.

[0102] In some embodiments of the present disclosure, the window smoothing module 630 is further configured to: determine a target smoothing window corresponding to the background image area based on the distance from the pixel points in the background image area to the area boundary of the foreground image area; perform window smoothing on the background image area based on the target smoothing window to obtain a smoothed background image area; and use the smoothed background image area to update the background image area in the image to be processed to obtain a preprocessed image.

[0103] In some embodiments of the present disclosure, the window smoothing module 630 is further configured to: divide the background image area into a first sub-area and a second sub-area according to the distance from the pixel points in the background image area to the area boundary of the foreground image area; the distance between the pixel points in the first sub-area and the area boundary of the foreground image area is less than or equal to a preset distance threshold, and the distance between the pixel points in the second sub-area and the area boundary of the foreground image area is greater than the preset distance threshold; determine a first smoothing window corresponding to the first sub-area, and determine a second smoothing window corresponding to the second sub-area; the first smoothing window is used to perform window smoothing processing on the first sub-area, and the second smoothing window is used to perform window smoothing processing on the second sub-area.

[0104] In some embodiments of the present disclosure, the window size of the first smoothing window and the window size of the second smoothing window are preset, and the window size of the first smoothing window is smaller than the window size of the second smoothing window.

[0105] In some embodiments of the present disclosure, the window smoothing module 630 is further configured to: traverse the background image area based on the window size of the target smoothing window to obtain the image area to be smoothed; for the image area to be smoothed, use a preset calculation method to calculate the pixel values ​​of the pixel points in the image area to be smoothed to obtain the pixel smoothing values ​​corresponding to the image area to be smoothed, and replace the pixel values ​​of the pixel points in the image area to be smoothed with the pixel smoothing values ​​corresponding to the image area to be smoothed; after the traversal of the background image area is completed, obtain the background image area after smoothing.

[0106] In some embodiments of the present disclosure, the window smoothing module 630 is further configured to: obtain pixel values ​​of pixel points in the area to be smoothed on multiple channels; for each channel, use a preset calculation method to calculate the pixel values ​​of pixel points in the image area to be smoothed on the channel to obtain the pixel smoothing value corresponding to the image area to be smoothed on the channel; and replace the pixel values ​​of pixel points in the image area to be smoothed on the channel with the pixel smoothing value corresponding to the image area to be smoothed on the channel.

[0107] In some embodiments of the present disclosure, the preset calculation method includes at least one of the following options: mean calculation, weighted mean calculation, median calculation, mode calculation and Gaussian filtering.

[0108] The image processing device provided by the embodiment of the present disclosure obtains a foreground image area and a background image area by performing visual recognition on the image to be processed, that is, distinguishes the area of ​​interest (that is, the foreground image area) and the area of ​​no interest (that is, the background image area) in the image to be processed, and on this basis, performs dynamic window smoothing processing on the background image area, which effectively reduces redundant information and noise in the background image area, and reduces the amount of information in the area of ​​no interest, so that a higher compression ratio can be achieved in the encoding and decoding process, and the contrast between the area of ​​interest and the area of ​​no interest is enhanced. While ensuring the encoding and decoding efficiency, it can better retain the feature information that is important for subsequent tasks (that is, the key features of the foreground image area), has little impact on the accuracy of subsequent tasks, and helps to improve the accuracy and efficiency of tasks.

[0109] It should be noted that the examples and application scenarios implemented by the modules in the above-mentioned device embodiment are the same as those of the corresponding steps in the method embodiment, but are not limited to the contents disclosed in the above-mentioned method embodiment. It should be noted that the above-mentioned modules as part of the device can be executed in a computer system such as a set of computer executable instructions.

[0110] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as a system, method or program product. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as a "circuit", "module" or "system".

[0111] Based on the same inventive concept, an electronic device is also provided in an embodiment of the present disclosure, the electronic device comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the above-mentioned image processing methods by executing the executable instructions. Since the principle of solving the problem in the electronic device embodiment is similar to that in the above-mentioned method embodiment, the implementation of the electronic device embodiment can refer to the implementation of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0112] Refer to the following Figure 7 An electronic device 700 according to this embodiment of the present disclosure is described. Figure 7 The electronic device 700 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0113] like Figure 7 As shown, the electronic device 700 is in the form of a general computing device. The components of the electronic device 700 may include but are not limited to: at least one processing unit 710, at least one storage unit 720, and a bus 730 connecting different system components (including the storage unit 720 and the processing unit 710).

[0114] The storage unit stores a program code, and the program code can be executed by the processing unit 710, so that the processing unit 710 executes the steps described in the above "Exemplary Method" section of this specification according to various exemplary embodiments of the present disclosure. For example, the processing unit 710 can execute the following steps of the above method embodiment: step S210, visually identifying the image to be processed to obtain the foreground image area of ​​the image to be processed; step S220, determining the area other than the foreground image area in the image to be processed as the background image area of ​​the image to be processed; step S230, performing dynamic window smoothing on the background image area to obtain a pre-processed image corresponding to the image to be processed; step S240, performing image encoding and decoding processing on the pre-processed image to obtain an image after encoding and decoding processing; step S250, processing the image after encoding and decoding processing based on the task network to obtain the task result of the image to be processed.

[0115] The storage unit 720 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 7201 and / or a cache storage unit 7202 , and may further include a read-only storage unit (ROM) 7203 .

[0116] The storage unit 720 may also include a program / utility 7204 having a set (at least one) of program modules 7205, such program modules 7205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0117] Bus 730 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0118] The electronic device 700 may also communicate with one or more external devices 740 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 700, and / or communicate with any device that enables the electronic device 700 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 750. Furthermore, the electronic device 700 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 760. As shown, the network adapter 760 communicates with other modules of the electronic device 700 via a bus 730. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0119] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0120] Based on the same inventive concept, the embodiment of the present disclosure also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, any of the above-mentioned image processing methods is implemented. Since the principle of solving the problem in the embodiment of the computer-readable storage medium is similar to that in the above-mentioned method embodiment, the implementation of the embodiment of the computer-readable storage medium can refer to the implementation of the above-mentioned method embodiment, and the repeated parts will not be repeated.

[0121] More specific examples of computer-readable storage media in the present disclosure may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0122] In the present disclosure, a computer readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0123] Alternatively, the program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0124] In a specific implementation, the program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0125] Based on the same inventive concept, a computer program product is also provided in an embodiment of the present disclosure, including a computer program product, including: a computer program or an instruction, wherein when the computer program or the instruction is executed by a processor, the image processing method of any one of the above method embodiments is implemented. Since the principle of solving the problem in the computer program product embodiment is similar to that in the above method embodiment, the implementation of the computer program product embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be repeated.

[0126] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0127] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0128] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0129] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

Claims

1. An image processing method, characterized in that: include: Performing visual recognition on the image to be processed to obtain a foreground image region of the image to be processed; Determine an area in the image to be processed except the foreground image area as a background image area of ​​the image to be processed; Performing dynamic window smoothing processing on the background image area to obtain a preprocessed image corresponding to the image to be processed; Performing image encoding and decoding processing on the pre-processed image to obtain an image after encoding and decoding processing; The image after encoding and decoding is processed based on the task network to obtain the task result of the image to be processed.

2. The method according to claim 1, characterized in that The step of performing dynamic window smoothing on the background image region to obtain a pre-processed image corresponding to the image to be processed includes: Determining a target smoothing window corresponding to the background image area according to a distance from a pixel point in the background image area to a region boundary of the foreground image area; Performing window smoothing processing on the background image area based on the target smoothing window to obtain a background image area after smoothing processing; The background image region in the image to be processed is updated by using the smoothed background image region to obtain the pre-processed image.

3. The method according to claim 2, characterized in that The step of determining a target smoothing window corresponding to the background image area according to a distance from a pixel point in the background image area to a region boundary of the foreground image area comprises: The background image area is divided into a first sub-area and a second sub-area according to the distance between the pixel points in the background image area and the area boundary of the foreground image area; the distance between the pixel points in the first sub-area and the area boundary of the foreground image area is less than or equal to a preset distance threshold, and the distance between the pixel points in the second sub-area and the area boundary of the foreground image area is greater than the preset distance threshold; A first smoothing window corresponding to the first sub-region is determined, and a second smoothing window corresponding to the second sub-region is determined; the first smoothing window is used to perform window smoothing processing on the first sub-region, and the second smoothing window is used to perform window smoothing processing on the second sub-region.

4. The method according to claim 3, characterized in that The window size of the first smoothing window and the window size of the second smoothing window are preset, and the window size of the first smoothing window is smaller than the window size of the second smoothing window.

5. The method according to claim 2 or 3, characterized in that: The performing window smoothing on the background image region based on the target smoothing window to obtain the background image region after the smoothing process comprises: Based on the window size of the target smoothing window, traverse the background image area to obtain the image area to be smoothed; For the image area to be smoothed, a preset calculation method is used to calculate the pixel values ​​of the pixel points in the image area to be smoothed, to obtain the pixel smoothing value corresponding to the image area to be smoothed, and the pixel smoothing value corresponding to the image area to be smoothed is used to replace the pixel values ​​of the pixel points in the image area to be smoothed; After the traversal of the background image area is completed, the background image area after the smoothing process is obtained.

6. The method according to claim 5, characterized in that The method of using a preset calculation method to calculate the pixel values ​​of the pixel points in the image area to be smoothed, obtaining the pixel smoothing value corresponding to the image area to be smoothed, and replacing the pixel values ​​of the pixel points in the image area to be smoothed with the pixel smoothing value corresponding to the image area to be smoothed includes: Obtaining pixel values ​​of pixel points in the area to be smoothed on multiple channels; For each channel, the preset calculation method is used to calculate the pixel values ​​of the pixel points in the image area to be smoothed on the channel to obtain the pixel smoothing value corresponding to the image area to be smoothed on the channel; and the pixel smoothing value corresponding to the image area to be smoothed on the channel is used to replace the pixel values ​​of the pixel points in the image area to be smoothed on the channel.

7. The method according to claim 5, characterized in that The preset calculation method includes at least one of the following options: mean calculation, weighted mean calculation, median calculation, mode calculation and Gaussian filtering.

8. An image processing device, characterized in that: include: A visual recognition module is configured to perform visual recognition on the image to be processed to obtain a foreground image area of ​​the image to be processed; A region determination module, configured to determine a region other than the foreground image region in the image to be processed as a background image region of the image to be processed; A window smoothing module is configured to perform dynamic window smoothing processing on the background image area to obtain a pre-processed image corresponding to the image to be processed; A coding and decoding module is configured to perform image coding and decoding processing on the pre-processed image to obtain an image after coding and decoding processing; The task processing module is configured to process the image after the encoding and decoding processing based on the task network to obtain the task result of the image to be processed.

9. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the image processing method according to any one of claims 1 to 7 by executing the executable instructions.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image processing method according to any one of claims 1 to 7 is implemented.

11. A computer program product comprising: A computer program or instruction, characterized in that when the computer program or instruction is executed by a processor, it implements the image processing method described in any one of claims 1 to 7.

Images