Image processing method, computing device, storage medium and computer program product

Abstract

The embodiment of the invention relates to the technical field of computers, in particular to an image processing method, computing equipment, a storage medium and a computer program product.The image processing method comprises the steps that a target object contained in a to-be-processed image is determined, and target edge information of the target object is determined; determining a stretching area in the to-be-processed image according to the target edge information; and according to coordinate information and pixel information corresponding to the target edge information, carrying out pixel stretching processing on the stretching area to obtain a target image. According to the image processing method and device, the stretching region part in the to-be-processed image is selected and processed according to the coordinate information and the pixel information corresponding to the target edge information, so that the computing resource consumption in the image processing process is reduced, and the efficiency and the real-time performance of image processing are ensured.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of computer technology, and in particular, to image processing methods, computing devices, storage media, and computer program products. Background Art

[0002] With the development of computer technology, image processing technology has become more and more mature. For example, it is possible to make basic adjustments to images or perform complex visual effects synthesis. Specifically, it is possible to adjust image properties such as brightness, contrast, saturation and hue, add filters to images, and perform special effects synthesis processing on images and perform image restoration, image resolution enhancement, etc. However, in the process of image processing, it is usually necessary to consume more computing resources to process the image, which affects the efficiency and real-time performance of image processing. Therefore, an effective technical solution is urgently needed to solve the above problems. Summary of the invention

[0003] In view of this, an embodiment of the present specification provides an image processing method. One or more embodiments of the present specification also relate to an image processing apparatus, a computing device, a computer-readable storage medium and a computer program product to solve the technical defects existing in the prior art.

[0004] According to a first aspect of an embodiment of this specification, there is provided an image processing method, including: Determine a target object contained in the image to be processed, and determine target edge information of the target object; Determining a stretching area in the image to be processed according to the target edge information; According to the coordinate information and pixel information corresponding to the target edge information, pixel stretching processing is performed on the stretching area to obtain a target image.

[0005] According to a second aspect of the embodiments of this specification, there is provided an image processing apparatus, including: A first determination module is configured to determine a target object contained in the image to be processed, and determine target edge information of the target object; A second determining module is configured to determine a stretching area in the image to be processed according to the target edge information; The processing module is configured to perform pixel stretching processing on the stretching area according to the coordinate information and pixel information corresponding to the target edge information to obtain a target image.

[0006] According to a third aspect of an embodiment of this specification, a computing device is provided, including: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor, the steps of the above method are implemented.

[0007] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores a computer program / instruction, and the steps of the above method are implemented when the computer program / instruction is executed by a processor.

[0008] According to a fifth aspect of the embodiments of this specification, a computer program product is provided, comprising a computer program / instruction, which implements the steps of the above method when executed by a processor.

[0009] An embodiment of the present specification provides an image processing method, including: determining a target object contained in an image to be processed, and determining target edge information of the target object; determining a stretching area in the image to be processed based on the target edge information; performing pixel stretching processing on the stretching area based on coordinate information and pixel information corresponding to the target edge information to obtain a target image.

[0010] In the above method, by determining the target object contained in the image to be processed and determining the target edge information of the target object, the stretched area portion of the image to be processed that needs to be processed is determined according to the target edge information, and the stretched area is subjected to pixel stretching processing according to the coordinate information and pixel information corresponding to the target edge information, so that the stretched area portion in the obtained target image has a pixel stretching effect, realizing the special effects synthesis of the image to be processed, ensuring the richness of the image processing, and by selecting the stretched area portion in the image to be processed and processing it according to the coordinate information and pixel information corresponding to the target edge information, the computing resource consumption in the image processing process is reduced, and the efficiency and real-time performance of the image processing are ensured. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is an application scenario diagram of an image processing method provided by an embodiment of this specification; Figure 2 is a flow chart of an image processing method provided by an embodiment of this specification; Figure 3 is a mapping relationship diagram in an image processing method provided by an embodiment of this specification; Figure 4 is a processing flow chart of an image processing method provided by an embodiment of this specification; Figure 5 is a structural schematic diagram of an image processing device provided by an embodiment of this specification; Figure 6It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0012] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0013] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0014] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0015] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0016] In one or more embodiments of this specification, a large model refers to a deep learning model with large-scale model parameters, which usually contains hundreds of millions, tens of billions, hundreds of billions, trillions, or even more than 10 trillion model parameters. A large model can also be called a foundation model / foundation model. The large model is pre-trained with large-scale unlabeled corpus to produce a pre-trained model with more than 100 million parameters. This model can adapt to a wide range of downstream tasks, and the model has good generalization ability, such as a large-scale language model (LLM), a multi-modal pre-training model, etc.

[0017] When the big model is used in practice, only a small number of samples are needed to fine-tune the pre-trained model and it can be applied to different tasks. The big model can be widely used in natural language processing (NLP), computer vision and other fields. Specifically, it can be applied to computer vision tasks such as visual question answering (VQA), image caption (IC), image generation, as well as natural language processing tasks such as text-based sentiment classification, text summary generation, and machine translation. The main application scenarios of the big model include digital assistants, intelligent robots, search, online education, office software, e-commerce, intelligent design, etc.

[0018] First, the terms involved in one or more embodiments of this specification are explained.

[0019] png: portable network graphics, a lossless compressed bitmap image format.

[0020] Color channels: In digital image processing, images are usually composed of multiple color channels, which represent different color components of the image. For common RGB (red, green, blue) images, there are three color channels; for RGBA images, there are four color channels. Among them, the r channel represents the red component in the image. Each pixel has an intensity value in this channel, which determines the brightness of the pixel in the red spectrum. The intensity value is usually an integer between 0 and 255 (in 8-bit images), where 0 represents no red component and 255 represents the brightest red. The g channel represents the green component in the image. Similar to the r channel, each pixel also has an intensity value in this channel, which determines the brightness of the pixel in the green spectrum. Similarly, the intensity value is also an integer between 0 and 255. The b channel represents the blue component in the image. The intensity value of each pixel in this channel determines the brightness of the pixel in the blue spectrum, and the intensity value is also an integer between 0 and 255. The a channel is not part of the RGB image, but exists in the RGBA image. The a channel represents the transparency (or opacity) in the image. The value of each pixel in this channel determines the degree of transparency of the pixel. The intensity value is also an integer between 0 and 255, where 0 means completely transparent and 255 means completely opaque.

[0021] In this specification, an image processing method is provided. This specification also relates to an image processing apparatus, a computing device, a computer-readable storage medium and a computer program product, which are described in detail one by one in the following embodiments.

[0022] See also Figure 1 , Figure 1 An application scenario diagram of an image processing method provided according to an embodiment of the present specification is shown.

[0023] like Figure 1 As shown, Figure 1 It includes a terminal side device 102 and a cloud side device 104.

[0024] In one embodiment of the present specification, the image processing method provided in the embodiment of the present specification can be applied to a social sharing platform, which can be set in the terminal device 102 in the form of an application or a webpage, and the user can log in to the social sharing platform through the terminal device 102 to share dynamics on the social sharing platform, such as picture dynamics, text dynamics, and video dynamics. When the user wants to share picture dynamics, the image processing method can be used to perform pixel stretching processing on the picture that the user wants to share, and realize special effects synthesis of the picture, so that the picture dynamics shared by the user are more interesting.

[0025] In specific implementation, the user can select the image to be processed that he wants to share in the terminal device 102, and the terminal device 102 can send the image to be processed to the cloud device 104. The cloud device 104 can determine the target object contained in the image to be processed and the target edge information of the target object, determine the stretching area in the image to be processed according to the target edge information, and perform pixel stretching processing on the stretching area according to the coordinate information and pixel information corresponding to the target edge information to obtain the target image, and send the target image to the terminal device 102. The terminal device 102 can display the target image on the display interface, and the user decides whether to upload the target image to the social sharing platform.

[0026] In addition, the image processing method provided in the embodiments of this specification can also be applied to image processing software, which is not limited in the embodiments of this specification.

[0027] The end-side device 102 may include a browser, an APP (Application), or a web application such as an H5 (Hyper Text Markup Language 5, version 5 of Hypertext Markup Language) application, or a light application (also known as a mini-program, a lightweight application) or a cloud application, etc. The end-side device may be based on a software development kit (SDK) of the corresponding service provided by the server, such as based on the real-time communication (RTC, Real Time Communication) SDK development and acquisition. The end-side device may be deployed in an electronic device and needs to rely on the device to run or some APPs in the device to run. The electronic device may have a display screen and support information browsing, such as a personal mobile terminal such as a mobile phone, a tablet computer, a personal computer, etc. Various other types of applications may also be configured in the electronic device, such as human-computer dialogue applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0028] The cloud-side device 104 can be understood as a server that provides various services, including physical servers and cloud servers, such as a server that provides communication services for multiple clients, a server for background training that supports the model used on the client, and a server that processes the data sent by the client. It should be noted that the cloud-side device 104 can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. The cloud-side device 104 can also be a server for a distributed system, or a server combined with a blockchain. The cloud-side device 104 can also be a cloud server for basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDNs), and big data and artificial intelligence platforms, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0029] It is worth noting that the image processing method provided in the embodiments of this specification can be executed by the cloud-side device 104 , can also be executed by the terminal-side device 102 , or can also be executed jointly by the terminal-side device 102 and the cloud-side device 104 .

[0030] See also Figure 2 , Figure 2 A flowchart of an image processing method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0031] Step 202: Determine a target object contained in the image to be processed, and determine target edge information of the target object.

[0032] Among them, the image to be processed can be understood as the image that needs to be processed, the target object contained in the image to be processed can be understood as the object displayed in the image to be processed, and the target edge information of the target object can be understood as the target edge line (i.e., contour line) of the target object that meets the preset attribute conditions. For example, the image to be processed can be a landscape picture, then the target object contained in the image to be processed can be the trees, buildings, rivers, lakes, blue sky, etc. displayed in the landscape picture, and the target edge information of the target object can be, for example, the contour lines of trees, the contour lines of buildings, etc.

[0033] In a specific implementation, the step of determining the target object contained in the image to be processed includes: According to the object segmentation algorithm, object segmentation processing is performed on the image to be processed to obtain the target object contained in the image to be processed.

[0034] The object segmentation algorithm may be understood as an algorithm for identifying and detecting objects contained in an image to be processed.

[0035] In practical applications, the object segmentation algorithm may be, for example, a deep learning algorithm, or may be a large model obtained through pre-training, which is not limited in the embodiments of this specification.

[0036] In summary, by using the object segmentation algorithm, the target object contained in the image to be processed can be identified and detected, which is convenient for the subsequent determination of the stretching area and further realizes the pixel stretching processing of the image to be processed.

[0037] Further, the determining the target edge information of the target object includes: Determining candidate edge information of the target object; The target edge information is determined from the candidate edge information according to a preset determination rule.

[0038] Among them, the candidate edge information of the target object can be understood as all the candidate edge lines of the target object. For example, if the target object is a building, then the target object can have 4 candidate edge lines in the image to be processed, and the 4 candidate edge lines are respectively located above, below, on the left and on the right of the target object. The preset determination rule can be understood as a rule for determining the target edge information from the candidate edge information. Then, the target edge information can be understood as the candidate edge information determined from the candidate edge information and meeting the preset attribute conditions.

[0039] Specifically, all candidate edge lines of the target object may be determined, and the target edge line may be determined from all candidate edge lines of the target object according to a preset determination rule.

[0040] In summary, by determining the target edge information from the candidate edge information, it is convenient to subsequently select the stretching area to be stretched from the image to be processed according to the target edge information, without having to perform image processing on the entire image to be processed, thereby reducing the consumption of computing resources.

[0041] In practical applications, the image to be processed contains multiple target objects; The determining the candidate edge information of the target object includes: Determining candidate edge information of each target object among the multiple target objects; The step of determining the target edge information from the candidate edge information according to a preset determination rule includes: According to a preset determination rule, the target edge information is determined from the candidate edge information of each target object.

[0042] Specifically, the image to be processed may contain multiple target objects, then all candidate edge lines of each target object in the multiple target objects may be determined, and the target edge line may be determined from all candidate edge lines of each target object according to a preset determination rule.

[0043] For example, if the image to be processed contains target object A, target object B and target object C, then candidate edge lines A1, A2, A3 and A4 of target object A can be determined, candidate edge lines B1, B2, B3 and B4 of target object B can be determined, candidate edge lines C1, C2, C3 and C4 of target object C can be determined, and the target edge line can be determined from these candidate edge lines A1, A2, A3 and A4, B1, B2, B3 and B4, C1, C2, C3 and C4.

[0044] In practical applications, the number of determined target edge information may be fixed. For example, 4 pieces of target edge information may be determined from the candidate edge information. Alternatively, other numbers of target edge information may be determined. This specification does not limit this.

[0045] In summary, by determining the target edge information from all the candidate edge information of multiple target objects, the stretching area is determined to ensure the image processing effect.

[0046] In a specific implementation, determining the target edge information from the candidate edge information of each target object according to a preset determination rule includes: Determine attribute information of candidate edge information of each target object; The candidate edge information whose attribute information meets the preset attribute condition is determined as the target edge information.

[0047] The attribute information of the candidate edge information may include length information of the candidate edge information and / or distance information of the candidate edge information from the edge of the image to be processed. The attribute information satisfies the preset attribute condition, which can be understood as the length information satisfies the preset length threshold and / or the distance information of the candidate edge information from the edge of the image to be processed satisfies the preset distance threshold.

[0048] Specifically, the length information of each candidate edge information may be determined, and the candidate edge information whose length information meets a preset length threshold may be determined as the target edge information. Alternatively, the length information of each candidate edge information may be determined, and the length information of each candidate edge information may be compared, and the candidate edge information with the longest length may be determined as the target edge information according to the comparison result. Alternatively, the distance information of each candidate edge information from the edge (upper boundary or lower boundary) of the image to be processed may be determined, and the candidate edge information whose distance information meets a preset distance threshold may be determined as the target edge information. Alternatively, the distance information of each candidate edge information from the edge (upper boundary or lower boundary) of the image to be processed may be determined, and the distance information may be compared, and the candidate edge information with the shortest distance may be determined as the target edge information according to the comparison result.

[0049] In summary, by determining the candidate edge information whose attribute information meets the preset attribute conditions as the target edge information, it is convenient to subsequently determine the stretching area.

[0050] Step 204: Determine a stretching area in the image to be processed according to the target edge information.

[0051] The stretching area can be understood as an image area in the image to be processed where pixel stretching is required.

[0052] In a specific implementation, determining the stretching area in the image to be processed according to the target edge information includes: Determine an image area of ​​the target edge information in a preset stretching direction; The image region is determined as a stretching region in the image to be processed.

[0053] Among them, the preset stretching direction can be understood as the direction of stretching pixels in the stretching area. The preset stretching direction can be, for example, an upward stretching direction, a downward stretching direction, a leftward stretching direction or a rightward stretching direction according to the location of the target edge information.

[0054] Specifically, the preset stretching direction may be determined based on the distance information between the target edge information and the edge of the image to be processed. For example, when the distance information between the target edge information and the upper boundary of the image to be processed is greater than the distance information between the target edge information and the lower boundary of the image to be processed, the preset stretching direction may be an upward stretching direction. This embodiment of the specification does not limit this.

[0055] In practical applications, after determining all candidate edge lines of the target object contained in the image to be processed, the longest edge line can be selected from all candidate edge lines as the target edge line. According to the position of the target edge line, the part of the target edge line that is upward or downward is determined as the stretching area, which can be used for subsequent pixel stretching.

[0056] In summary, by determining the stretching area, it is convenient to achieve the synthesis effect of the image later, thereby increasing the viewing and interest of the image.

[0057] Step 206: Perform pixel stretching processing on the stretching area according to the coordinate information and pixel information corresponding to the target edge information to obtain a target image.

[0058] The coordinate information corresponding to the target edge information can be understood as the position coordinates of the pixel point where the target edge information is located in the image to be processed, including the horizontal coordinate (i.e., the X coordinate) and the vertical coordinate (i.e., the Y coordinate). The pixel information corresponding to the target edge information can be understood as the filling color of the pixel point where the target edge information is located in the image to be processed.

[0059] In a specific implementation, performing pixel stretching processing on the stretching area according to the coordinate information and pixel information corresponding to the target edge information to obtain the target image includes: Creating a mapping relationship between the coordinate information and the pixel information according to the coordinate information and the pixel information corresponding to the target edge information; According to the mapping relationship, pixel stretching processing is performed on the stretching area to obtain a target image.

[0060] The mapping relationship between the coordinate information and the pixel information can be understood as a mapping relationship diagram between the position coordinates of the pixel point where the target edge information is located in the image to be processed and the filling color, such as Figure 3 As shown, Figure 3 A mapping relationship diagram in an image processing method provided according to an embodiment of the present specification is shown.

[0061] Specifically, a mapping relationship diagram between position coordinates and fill colors can be created based on the position coordinates and fill colors of the pixel points where the target edge information is located in the image to be processed, and pixel stretching processing is performed on the stretching area based on the mapping relationship diagram to obtain the target image after pixel stretching.

[0062] In practical applications, a mapping relationship diagram of the filling color that needs to be filled in the stretched area can be obtained, and the target edge line can be stretched and filled downward or upward. In other words, the position coordinates of the filling color of the target edge line are filled downward or upward to each position that needs to be filled with color in the stretched area. At this time, the pixel points in the stretched area that need to change color store the position coordinates of the target edge line.

[0063] In summary, by creating a mapping relationship graph, it is convenient to subsequently perform inverse analysis according to the mapping relationship graph to obtain the actual pixel values ​​of the stretched area, thereby obtaining a target image.

[0064] In practical applications, the step of creating a mapping relationship between the coordinate information and the pixel information according to the coordinate information and the pixel information corresponding to the target edge information includes: Filling the pixel information of the target edge information according to the preset pixel information to obtain the filled pixel information; According to the coordinate information corresponding to the target edge information, the coordinate information in the preset stretching direction is replaced to obtain the replaced coordinate information; A mapping relationship between the coordinate information and the pixel information is created according to the filled pixel information and the replaced coordinate information.

[0065] The preset pixel information may be understood as a preset fixed pixel color value that needs to be filled with the pixel information of the target edge information.

[0066] Based on this, the pixel information of the target edge information can be filled according to the pre-set fixed pixel color value that needs to be filled in the pixel information of the target edge information to obtain the filled pixel information, and the coordinate information in the stretching area and in the preset stretching direction can be replaced according to the coordinate information corresponding to the target edge information to obtain the replaced coordinate information, and the mapping relationship between the coordinate information and the pixel information can be obtained according to the filled pixel information and the replaced coordinate information.

[0067] In practical applications, the color of the target edge line can be filled with a color value that is randomly offset up and down by 5 pixels according to the preset stretching direction of the target edge line (such as the upward or downward stretching direction), and the coordinate information in the preset stretching direction is replaced with the vertical coordinate information of the current location of the target edge line according to the preset stretching direction of the target edge line. The mapped pixel coordinate values ​​are obtained and stored in a png picture to obtain a mapping relationship diagram between the coordinate information and the pixel information (i.e., a coordinate mapping texture diagram).

[0068] In summary, by performing pixel coordinate filling and mapping, the mapping between coordinate information and pixel information is achieved, which facilitates subsequent pixel stretching.

[0069] Specifically, performing pixel stretching processing on the stretching area according to the mapping relationship to obtain a target image includes: Performing an inverse analysis on the mapping relationship to obtain a target pixel value of the stretching area; The stretched area is filled according to the target pixel value to obtain a target image.

[0070] In practical applications, the actual pixel value (ie, target pixel value) of the portion to be stretched and filled (ie, the stretched area) can be inversely analyzed based on the mapping relationship diagram, and the stretched area can be filled based on the actual pixel value to obtain the target image.

[0071] Furthermore, after performing inverse analysis on the mapping relationship to obtain the target pixel value of the stretching area, the method further includes: Restoring the target pixel information to target coordinate information; The step of filling the stretched area according to the target pixel value to obtain a target image includes: According to the target coordinate information, sampling the stretched area to obtain a stretched image; The stretched image and the image to be processed are mixed to obtain a target image.

[0072] In practical applications, the sampled target pixel information (i.e., color value) can be restored to target coordinate information (i.e., actual coordinate value) according to the mapping relationship diagram, and the target coordinate information can be normalized and then sampled in the stretched area of ​​the image to be processed to obtain a stretched image, and the stretched image and the image to be processed (i.e., original image) can be mixed to obtain the target image.

[0073] In summary, the stretched image is obtained by restoring the target pixel information, and the target image with the pixel stretching effect is obtained by mixing the stretched image and the image to be processed.

[0074] In a specific implementation, the step of mixing the stretched image and the image to be processed to obtain a target image includes: Performing segmentation mask processing on the target object in the image to be processed to obtain a segmentation mask image corresponding to the image to be processed; The segmentation mask image is used as a mixing factor to mix the stretched image and the image to be processed to obtain a target image.

[0075] In practical applications, the target object in the image to be processed can be segmented and masked according to the image segmentation model to obtain a segmentation mask image corresponding to the image to be processed (i.e., the main segmentation mask), and the segmentation mask image can be used as a mixing factor to mix the stretched image and the image to be processed to obtain the target image (i.e., the final semi-stretched effect image).

[0076] In summary, pixel stretching of the image to be processed is achieved through image mixing.

[0077] The creating a mapping relationship between the coordinate information and the pixel information includes: According to a preset storage rule, the coordinate information is stored in a color channel in the pixel information to create a mapping relationship between the coordinate information and the pixel information.

[0078] The preset storage rule can be understood as a storage rule for coordinate information. The color channels can include an r channel, a g channel, an a channel, and a b channel.

[0079] In practical applications, since the pixel information (i.e., color value) is 0-255 and the coordinate information (i.e., position coordinate) is the actual image size, when storing the mapping relationship (i.e., mapping table), the r channel and the g channel can be used to store the X coordinate value, and the b channel and the a channel can be used to store the Y coordinate value. When reading the mapping relationship, the r channel stores the quotient and the g channel stores the remainder. When using the mapping table, it is necessary to reversely map (i.e., reversely parse) it back to achieve the specified area (i.e., stretching area) for pixel stretching.

[0080] In summary, by storing coordinate information in the color channel, the inverse mapping of the mapping table reading is achieved.

[0081] In addition, during the processing of the image to be processed, the target object in the image to be processed can be removed, the background in the image to be processed can be pixel-stretched to obtain a stretched image, and then the target object can be filled into the stretched image to obtain a target image.

[0082] In practical applications, before determining the target object contained in the image to be processed, the method further includes: Receive the image to be processed sent by the client; After obtaining the target image, the method further includes: The target image is sent to the client and displayed through the display interface of the client.

[0083] Specifically, the user can send the image to be processed through the client, the client sends the image to be processed to the server, the server executes the image processing method to process the image to be processed, obtains the target image, and sends the target image to the client, and displays the target image through the client's display interface.

[0084] Then, in practical applications, before determining the target object contained in the image to be processed, or after sending the target image to the client, the following steps are also included: In response to the image processing instruction sent by the client, determining a preset stretching direction corresponding to the image processing instruction; The image processing instruction is determined by the client according to a selection instruction of an image processing control on the display interface.

[0085] Specifically, an image processing control may be displayed on the display interface of the client. The user clicks on the image processing control. The client generates an image processing instruction in response to the user's clicking operation. The image processing instruction can be used to customize the preset stretching direction of the pixel stretching of the image to be processed. For example, if the user selects the preset stretching direction as upward stretching, the client can send the image processing instruction carrying the preset stretching direction to the server. The server responds to the image processing instruction and determines that the preset stretching direction is upward stretching.

[0086] In summary, the preset stretching direction can be customized through user instructions to meet the user's personalized needs for image processing.

[0087] In practical applications, the determining of target edge information of the target object further includes: In response to the edge information selection instruction sent by the client, target edge information corresponding to the edge information selection instruction is determined, and the target edge information is determined as the target edge information of the target object.

[0088] The edge information selection instruction is determined by the client according to a selection instruction of an edge information selection control on the display interface.

[0089] Specifically, the target edge information of the target object can be customized by the user. The image edge information selection control and the image to be processed can be displayed on the display interface of the client. The user clicks on the image edge information selection control. The client generates an image edge information selection instruction in response to the user's clicking operation. The image edge information selection instruction can be used to customize the target edge information of the target object. For example, the user can click on the candidate edge information of the target object on the image to be processed, and click the image edge information selection control after the clicking is completed. The client can send the image edge information selection instruction carrying the target edge information selected by the user to the server. The server determines the target edge information in response to the image edge information selection instruction.

[0090] In addition, after the server sends the target image to the client, the user can adjust the target image displayed on the client's display interface, such as reselecting the target edge information for pixel stretching. The specific process is similar to the above and will not be repeated here.

[0091] In summary, the target edge information can be customized through user instructions to meet the user's personalized needs for image processing.

[0092] In summary, in the above method, by determining the target object contained in the image to be processed and determining the target edge information of the target object, the stretched area portion of the image to be processed that needs to be processed is determined according to the target edge information, and the stretched area is subjected to pixel stretching processing according to the coordinate information and pixel information corresponding to the target edge information, so that the stretched area portion in the obtained target image has a pixel stretching effect, and the special effects synthesis of the image to be processed is realized, and the richness of the image processing is ensured. Moreover, by selecting the stretched area portion in the image to be processed and processing it according to the coordinate information and pixel information corresponding to the target edge information, the computing resource consumption in the image processing process is reduced, and the efficiency and real-time performance of the image processing are ensured.

[0093] The following combination Figure 4 , taking the application of the image processing method provided in this specification in image pixel stretching as an example, the image processing method is further described. Figure 4A processing flow chart of an image processing method provided by an embodiment of the present specification is shown, which specifically includes the following steps.

[0094] Step 402: Determine the target object contained in the image to be processed.

[0095] Specifically, all objects (i.e., target objects) in the image to be processed can be detected through object segmentation.

[0096] Step 404: Determine candidate edge information of the target object.

[0097] Specifically, the image to be processed can be traversed, and the actual segmentation line (ie, candidate edge information) of each target object can be found and calculated according to the segmentation algorithm.

[0098] Step 406: Determine target edge information from the candidate edge information according to a preset determination rule.

[0099] Specifically, the distances between all segmentation lines and the upper and lower boundaries of the image to be processed may be calculated, and the segmentation line with the shortest distance may be used as the target segmentation line (ie, target edge information).

[0100] Step 408: Fill the pixel information of the target edge information according to the preset pixel information to obtain the filled pixel information.

[0101] Specifically, the color of the target segmentation line may be filled with a color value that is randomly offset up or down by 5 pixels according to the upward or downward stretching direction of the target segmentation line.

[0102] Step 410: According to the coordinate information corresponding to the target edge information, the coordinate information in the preset stretching direction is replaced to obtain the replaced coordinate information.

[0103] Specifically, according to the stretching direction of the target segmentation line, the coordinate values ​​of the pixel points in the stretching direction may be replaced with the vertical coordinates of the current location of the target segmentation line.

[0104] Step 412: Create a mapping relationship between the coordinate information and the pixel information according to the filled pixel information and the replaced coordinate information.

[0105] Specifically, the mapped pixel coordinate values ​​may be stored in a png image to obtain a mapping relationship diagram between the coordinate information and the pixel information.

[0106] Step 414: The algorithm sends the segmented body and the mapped coordinate map.

[0107] The segmented subject can be understood as the target object in the image to be processed, and the mapped coordinate diagram can be understood as the mapping relationship diagram between coordinate information and pixel information.

[0108] Step 416: The engine parses the configuration file to obtain the segmentation mask image and mapping relationship diagram corresponding to the image to be processed.

[0109] Step 418: Perform inverse analysis on the mapping relationship to obtain the target pixel value of the stretched area, and restore the target pixel information to the target coordinate information.

[0110] Specifically, the sampled color value (ie, target pixel value) can be restored to the actual coordinate value (ie, target coordinate information) according to the mapping relationship map (ie, coordinate mapping texture map).

[0111] Step 420: sampling the stretched area according to the target coordinate information to obtain a stretched image.

[0112] Specifically, the image to be processed may be normalized according to the actual coordinate values ​​and then sampled to obtain a stretched image.

[0113] Step 422: Using the segmentation mask image as a mixing factor, the stretched image and the image to be processed are mixed to obtain a target image.

[0114] Among them, the target image is the final semi-stretched effect image.

[0115] In summary, in the above method, by determining the target object contained in the image to be processed and determining the target edge information of the target object, the stretched area portion of the image to be processed that needs to be processed is determined according to the target edge information, and the stretched area is subjected to pixel stretching processing according to the coordinate information and pixel information corresponding to the target edge information, so that the stretched area portion in the obtained target image has a pixel stretching effect, and the special effects synthesis of the image to be processed is realized, and the richness of the image processing is ensured. Moreover, by selecting the stretched area portion in the image to be processed and processing it according to the coordinate information and pixel information corresponding to the target edge information, the computing resource consumption in the image processing process is reduced, and the efficiency and real-time performance of the image processing are ensured.

[0116] Corresponding to the above method embodiment, this specification also provides an image processing device embodiment, Figure 5 FIG. 2 shows a schematic diagram of the structure of an image processing device provided by an embodiment of the present specification. Figure 5 As shown, the device comprises: A first determination module 502 is configured to determine a target object contained in the image to be processed, and determine target edge information of the target object; A second determination module 504 is configured to determine a stretching area in the image to be processed according to the target edge information; The processing module 506 is configured to perform pixel stretching processing on the stretching area according to the coordinate information and pixel information corresponding to the target edge information to obtain a target image.

[0117] In an optional embodiment, the processing module 506 is further configured to: Creating a mapping relationship between the coordinate information and the pixel information according to the coordinate information and the pixel information corresponding to the target edge information; According to the mapping relationship, pixel stretching processing is performed on the stretching area to obtain a target image.

[0118] In an optional embodiment, the processing module 506 is further configured to: Filling the pixel information of the target edge information according to the preset pixel information to obtain the filled pixel information; According to the coordinate information corresponding to the target edge information, the coordinate information in the preset stretching direction is replaced to obtain the replaced coordinate information; A mapping relationship between the coordinate information and the pixel information is created according to the filled pixel information and the replaced coordinate information.

[0119] In an optional embodiment, the processing module 506 is further configured to: Performing an inverse analysis on the mapping relationship to obtain a target pixel value of the stretching area; The stretched area is filled according to the target pixel value to obtain a target image.

[0120] In an optional embodiment, the processing module 506 is further configured to: Restoring the target pixel information to target coordinate information; According to the target coordinate information, sampling the stretched area to obtain a stretched image; The stretched image and the image to be processed are mixed to obtain a target image.

[0121] In an optional embodiment, the processing module 506 is further configured to: Performing segmentation mask processing on the target object in the image to be processed to obtain a segmentation mask image corresponding to the image to be processed; The segmentation mask image is used as a mixing factor to mix the stretched image and the image to be processed to obtain a target image.

[0122] In an optional embodiment, the processing module 506 is further configured to: According to a preset storage rule, the coordinate information is stored in a color channel in the pixel information to create a mapping relationship table between the coordinate information and the pixel information.

[0123] In an optional embodiment, the second determining module 504 is further configured to: Determine an image area of ​​the target edge information in a preset stretching direction; The image region is determined as a stretching region in the image to be processed.

[0124] In an optional embodiment, the first determining module 502 is further configured to: Determining candidate edge information of the target object; The target edge information is determined from the candidate edge information according to a preset determination rule.

[0125] In an optional embodiment, the image to be processed contains multiple target objects; The first determining module 502 is further configured to: Determining candidate edge information of each target object among the multiple target objects; According to a preset determination rule, the target edge information is determined from the candidate edge information of each target object.

[0126] In an optional embodiment, the first determining module 502 is further configured to: Determine attribute information of candidate edge information of each target object; The candidate edge information whose attribute information meets the preset attribute condition is determined as the target edge information.

[0127] In an optional embodiment, the first determining module 502 is further configured to: According to the object segmentation algorithm, object segmentation processing is performed on the image to be processed to obtain the target object contained in the image to be processed.

[0128] In an optional embodiment, the device further includes a communication module configured to: Receive the image to be processed sent by the client; The target image is sent to the client and displayed through the display interface of the client.

[0129] In an optional embodiment, the first determining module 502 is further configured to: In response to the image processing instruction sent by the client, determining a preset stretching direction corresponding to the image processing instruction; The image processing instruction is determined by the client according to a selection instruction of an image processing control on the display interface.

[0130] In an optional embodiment, the first determining module 502 is further configured to: In response to the edge information selection instruction sent by the client, target edge information corresponding to the edge information selection instruction is determined, and the target edge information is determined as the target edge information of the target object.

[0131] The edge information selection instruction is determined by the client according to a selection instruction of an edge information selection control on the display interface.

[0132] In the above-mentioned device, by determining the target object contained in the image to be processed and determining the target edge information of the target object, the stretched area portion of the image to be processed that needs to be processed is determined according to the target edge information, and the stretched area is subjected to pixel stretching processing according to the coordinate information and pixel information corresponding to the target edge information, so that the stretched area portion in the obtained target image has a pixel stretching effect, realizing the special effects synthesis of the image to be processed, ensuring the richness of the image processing, and by selecting the stretched area portion in the image to be processed and processing it according to the coordinate information and pixel information corresponding to the target edge information, the computing resource consumption in the image processing process is reduced, and the efficiency and real-time performance of the image processing are ensured.

[0133] The above is a schematic scheme of an image processing device of this embodiment. It should be noted that the technical scheme of the image processing device and the technical scheme of the above-mentioned image processing method belong to the same concept, and the details not described in detail in the technical scheme of the image processing device can be referred to the description of the technical scheme of the above-mentioned image processing method.

[0134] Figure 6 The block diagram of a computing device 600 according to an embodiment of the present specification is shown. The components of the computing device 600 include but are not limited to a memory 610 and a processor 620. The processor 620 is connected to the memory 610 via a bus 630, and the database 650 is used to store data.

[0135] The computing device 600 also includes an access device 640 that enables the computing device 600 to communicate via one or more networks 660. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 640 may include one or more of any type of network interface (e.g., a network interface card (NIC)) that is wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0136] In one embodiment of the present application, the above components of the computing device 600 and Figure 6 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 6 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of the present application. Those skilled in the art may add or replace other components as needed.

[0137] The computing device 600 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 600 may also be a mobile or stationary server.

[0138] The processor 620 is used to execute the following computer program / instructions, which implement the steps of the above-mentioned image processing method when executed by the processor.

[0139] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the computing device embodiment, since it is basically similar to the image processing method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the image processing method embodiment.

[0140] An embodiment of the present specification further provides a computer-readable storage medium storing a computer program / instruction, which implements the steps of the above-mentioned image processing method when executed by a processor.

[0141] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the computer-readable storage medium embodiment, since it is basically similar to the image processing method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the image processing method embodiment.

[0142] An embodiment of the present specification further provides a computer program product, including a computer program / instruction, which implements the steps of the above-mentioned image processing method when executed by a processor.

[0143] The above is a schematic solution of a computer program product of this embodiment. It should be noted that the technical solution of the computer program product and the technical solution of the above-mentioned image processing method belong to the same concept, and the details not described in detail in the technical solution of the computer program product can be referred to the description of the technical solution of the above-mentioned image processing method.

[0144] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0145] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0146] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0147] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0148] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. An image processing method, comprising: Determine a target object contained in the image to be processed, and determine target edge information of the target object; Determining a stretching area in the image to be processed according to the target edge information; According to the coordinate information and pixel information corresponding to the target edge information, pixel stretching processing is performed on the stretching area to obtain a target image.

2. The method according to claim 1, wherein the pixel stretching process is performed on the stretching area according to the coordinate information and pixel information corresponding to the target edge information to obtain the target image, comprising: Creating a mapping relationship between the coordinate information and the pixel information according to the coordinate information and the pixel information corresponding to the target edge information; According to the mapping relationship, pixel stretching processing is performed on the stretching area to obtain a target image.

3. The method according to claim 2, wherein the step of creating a mapping relationship between the coordinate information and the pixel information according to the coordinate information and the pixel information corresponding to the target edge information comprises: Filling the pixel information of the target edge information according to the preset pixel information to obtain the filled pixel information; According to the coordinate information corresponding to the target edge information, the coordinate information in the preset stretching direction is replaced to obtain the replaced coordinate information; A mapping relationship between the coordinate information and the pixel information is created according to the filled pixel information and the replaced coordinate information.

4. The method according to claim 2, wherein the step of performing pixel stretching processing on the stretching area according to the mapping relationship to obtain a target image comprises: Performing an inverse analysis on the mapping relationship to obtain a target pixel value of the stretching area; The stretched area is filled according to the target pixel value to obtain a target image.

5. The method according to claim 4, after performing inverse analysis on the mapping relationship to obtain the target pixel value of the stretching area, further comprises: Restoring the target pixel information to target coordinate information; The step of filling the stretched area according to the target pixel value to obtain a target image includes: According to the target coordinate information, sampling the stretched area to obtain a stretched image; The stretched image and the image to be processed are mixed to obtain a target image.

6. The method according to claim 5, wherein the step of mixing the stretched image and the image to be processed to obtain a target image comprises: Performing segmentation mask processing on the target object in the image to be processed to obtain a segmentation mask image corresponding to the image to be processed; The segmentation mask image is used as a mixing factor to mix the stretched image and the image to be processed to obtain a target image.

7. The method according to claim 3, wherein the creating a mapping relationship between the coordinate information and the pixel information comprises: According to a preset storage rule, the coordinate information is stored in a color channel in the pixel information to create a mapping relationship between the coordinate information and the pixel information.

8. The method according to claim 1, wherein determining the stretching area in the image to be processed according to the target edge information comprises: Determine an image area of ​​the target edge information in a preset stretching direction; The image region is determined as a stretching region in the image to be processed.

9. The method according to claim 1, wherein determining target edge information of the target object comprises: Determining candidate edge information of the target object; The target edge information is determined from the candidate edge information according to a preset determination rule.

10. The method according to claim 9, wherein the image to be processed contains a plurality of target objects; The determining the candidate edge information of the target object includes: Determining candidate edge information of each target object among the multiple target objects; The step of determining the target edge information from the candidate edge information according to a preset determination rule includes: According to a preset determination rule, the target edge information is determined from the candidate edge information of each target object.

11. The method according to claim 10, wherein determining the target edge information from the candidate edge information of each target object according to a preset determination rule comprises: Determine attribute information of candidate edge information of each target object; The candidate edge information whose attribute information meets the preset attribute condition is determined as the target edge information.

12. The method according to claim 1, wherein determining the target object contained in the image to be processed comprises: According to the object segmentation algorithm, object segmentation processing is performed on the image to be processed to obtain the target object contained in the image to be processed.

13. The method according to claim 1, before determining the target object contained in the image to be processed, further comprising: Receive the image to be processed sent by the client; After obtaining the target image, the method further includes: The target image is sent to the client and displayed through the display interface of the client.

14. The method according to claim 13, further comprising: In response to the image processing instruction sent by the client, determining a preset stretching direction corresponding to the image processing instruction; The image processing instruction is determined by the client according to a selection instruction of an image processing control on the display interface.

15. The method according to claim 13, wherein determining target edge information of the target object further comprises: In response to the edge information selection instruction sent by the client, determine the target edge information corresponding to the edge information selection instruction, and determine the target edge information as the target edge information of the target object; The edge information selection instruction is determined by the client according to a selection instruction of an edge information selection control on the display interface.

16. A computing device comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer program / instructions are executed by the processor, the steps of the method described in any one of claims 1 to 15 are implemented.

17. A computer-readable storage medium storing a computer program / instruction, wherein the computer program / instruction, when executed by a processor, implements the steps of the method according to any one of claims 1 to 15.

18. A computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 15.

Images