Image expansion method and device based on edge detection and physical simulation, equipment and medium

Abstract

The invention relates to a picture expansion method, device and equipment based on edge detection and physical simulation and a medium, which are applied to the technical field of picture processing, and the method comprises the following steps: acquiring picture edge segmentation parameters and a to-be-expanded picture; performing edge detection segmentation on the to-be-expanded picture based on the picture edge segmentation parameter to generate an edge segmentation result; obtaining an edge region histogram of the edge segmentation result; determining background color data of the edge segmentation result based on the edge region histogram; determining a background region type of the edge segmentation result based on the background color data and a preset model; and carrying out picture expansion processing on the to-be-expanded picture based on the background region type and a preset physical simulation technology to generate an expanded picture. The method has the effect of improving the accuracy of picture expansion.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of picture processing, in particular to a picture expansion method and device based on edge detection and physical simulation, equipment and medium. BACKGROUND

[0002] Due to the rapid development of deep learning in picture generation, generative large models are rapidly applied to the field of picture expansion. The generation logic of the existing large model is to input the picture to be expanded into a visual language model or other multi-modal large model to extract the keywords of the picture, and then input the picture and the picture keywords into the picture generation model to guide the picture expansion.

[0003] However, the existing picture expansion method is based on the training paradigm of the visual language model in the form of picture-text pairing. During the manual annotation process, only the main body and key elements in the picture are usually marked. The trained model focuses on the main body of the picture, so it cannot well give the keywords of the edge area to be expanded in the picture. After inputting these inaccurate keywords into the generation model, it is easy to cause misunderstanding of the generation model and produce incorrect results. Moreover, the current training method of the generation model is also based on the picture-language pairing method, which also causes the generation model to output the elements trained in the training process, such as pure color gradient commodity pictures used in the e-commerce field, which cannot be properly expanded along the pure color gradient background of the original picture. In addition, the background blurring capability of some cameras makes the background of the picture often show a continuous color block of the background object color tone, and the generation model cannot well fit the color tone change caused by background blurring. SUMMARY

[0004] In order to improve the accuracy of picture expansion, the present application provides a picture expansion method and device based on edge detection and physical simulation.

[0005] In a first aspect, the present application provides a picture expansion method based on edge detection and physical simulation, which adopts the following technical solution:

[0006] A picture expansion method based on edge detection and physical simulation, comprising:

[0007] Obtaining picture edge segmentation parameters and a picture to be expanded;

[0008] Performing edge detection segmentation on the picture to be expanded based on the picture edge segmentation parameters to generate an edge segmentation result;

[0009] Obtaining an edge region histogram of the edge segmentation result;

[0010] Determining background color data of the edge segmentation result based on the edge region histogram;

[0011] determine a background region type of the edge segmentation result based on the background color data and a preset model;

[0012] perform picture expansion processing on the picture to be expanded based on the background region type and a preset physical simulation technique, to generate an expanded picture.

[0013] By adopting the technical solution, the picture to be expanded is segmented based on the picture segmentation parameter, a plurality of edge segmentation regions after segmentation are obtained, the regions after segmentation are taken as the edge segmentation result, the background color data of each segmentation region in the edge segmentation result is calculated, the background region type is determined based on the background color data, different background types correspond to different expansion manners, the regions after segmentation are expanded based on the background region type, and after the expansion is completed, the expanded picture is obtained through aggregation processing. The overall expansion is processed based on the actual edge segmentation result, different processing manners are adopted for different types of background regions, the processing is more in line with the current background situation, and therefore the accuracy of picture expansion is improved.

[0014] Optionally, the edge detection and segmentation of the picture to be expanded based on the picture edge segmentation parameter to generate an edge segmentation result comprises:

[0015] obtaining a picture size, a segmentation fixed value and an edge intensity threshold value of the picture to be expanded;

[0016] determining a maximum initial value and a minimum initial value of the picture to be expanded based on the picture size and the picture edge segmentation parameter;

[0017] determining whether the maximum initial value and the minimum initial value are within a numerical range of the segmentation fixed value;

[0018] if the maximum initial value and the minimum initial value are within the numerical range of the segmentation fixed value, determining an edge background region of the picture to be expanded based on the maximum initial value and the minimum initial value;

[0019] if the maximum initial value and the minimum initial value are not within the numerical range of the segmentation fixed value, determining an edge background region of the picture to be expanded based on the segmentation fixed value;

[0020] segmenting the edge background region according to four regions of top, bottom, left and right based on the edge intensity threshold value, to generate an edge segmentation result.

[0021] Optionally, the determination of the background color data of the edge segmentation result based on the edge region histogram comprises:

[0022] obtaining an adaptive weight parameter;

[0023] annotating a main annotation region of the edge segmentation result based on the edge region histogram and the adaptive weight parameter;

[0024] calculating background color data of the easy region based on a preset algorithm and the main annotation region.

[0025] Optionally, the background region type of the edge segmentation result is determined based on the background color data and a preset model, including:

[0026] determining whether the background color of the edge segmentation result is a pure color gradient based on the background color data;

[0027] if the background color of the edge segmentation result is not a pure color gradient, determining that the background region type of the edge background region is a pure color gradient background;

[0028] if the background color of the edge segmentation result is not a pure color gradient, determining that the background region type of the edge background region is a natural image background.

[0029] Optionally, the picture expansion processing on the picture to be expanded based on the background region type and a preset physical simulation technology to generate an expanded picture includes:

[0030] extracting a keyword from the natural image background based on a preset visual language model to generate a region keyword;

[0031] optimizing and recombining the region keywords of the four regions of the edge segmentation result to generate an optimized prompt word;

[0032] performing picture expansion processing on the picture to be expanded based on the pure color gradient background, the optimized prompt word and the preset simulation technology to generate an expanded picture.

[0033] Optionally, the picture expansion processing on the picture to be expanded based on the pure color gradient background, the optimized prompt word and the preset simulation technology to generate an expanded picture includes:

[0034] performing color diffusion simulation on the pure color gradient background based on the preset simulation technology to generate a color diffusion result;

[0035] expanding the edge segmentation result based on the color diffusion result to generate a first expanded region;

[0036] expanding the edge segmentation result based on a preset picture generation large model, the optimized prompt word and the natural image background to generate a second expanded region;

[0037] performing expansion processing on the picture to be expanded based on the first expansion region and the second expansion region, to generate an expanded picture.

[0038] Optionally, after the picture expansion processing on the picture to be expanded based on the background region type and the preset physical simulation technique is performed to generate an expanded picture, the method further includes:

[0039] dividing the picture to be expanded and the expanded picture into expansion regions to generate a division result;

[0040] annotating the expansion regions of the expanded picture based on the division result.

[0041] In a second aspect, the present application provides a picture expansion device based on edge detection and physical simulation, which adopts the following technical scheme:

[0042] A picture expansion device based on edge detection and physical simulation, comprising:

[0043] a picture data acquisition module, configured to acquire picture edge segmentation parameters and a picture to be expanded;

[0044] a division result generation module, configured to perform edge detection segmentation on the picture to be expanded based on the picture edge segmentation parameters, to generate an edge segmentation result;

[0045] a histogram data acquisition module, configured to acquire an edge region histogram of the edge segmentation result;

[0046] a background data determination module, configured to determine background color data of the edge segmentation result based on the edge region histogram;

[0047] a background type determination module, configured to determine a background region type of the edge segmentation result based on the background color data and a preset model;

[0048] an expanded picture generation module, configured to perform picture expansion processing on the picture to be expanded based on the background region type and a preset physical simulation technique, to generate an expanded picture.

[0049] According to the picture segmentation parameter, the picture to be expanded is segmented to obtain a plurality of edge segmentation regions, the segmented regions are taken as edge segmentation results, background color data of each segmentation region in the edge segmentation results is calculated, the background region type is determined according to the background color data, different background types correspond to different expansion modes, the segmented regions are expanded according to the background region type, and after the expansion is completed, aggregation processing is performed to obtain the final expanded picture.

[0050] In a third aspect, the present application provides an electronic device, which adopts the technical scheme as follows:

[0051] An electronic device, comprising a processor coupled with a memory;

[0052] The processor is configured to execute a computer program stored in the memory, so that the electronic device executes the computer program of the picture expansion method based on edge detection and physical simulation according to any one of the first aspect.

[0053] In a fourth aspect, the present application provides a computer readable storage medium, which adopts the technical scheme as follows:

[0054] A computer readable storage medium stores a computer program capable of being loaded and executed by a processor to execute the picture expansion method based on edge detection and physical simulation according to any one of the first aspect.

[0055] In summary, the present application includes at least one of the following beneficial technical effects:

[0056] According to the picture segmentation parameter, the picture to be expanded is segmented to obtain a plurality of edge segmentation regions, the segmented regions are taken as edge segmentation results, background color data of each segmentation region in the edge segmentation results is calculated, the background region type is determined according to the background color data, different background types correspond to different expansion modes, the segmented regions are expanded according to the background region type, and after the expansion is completed, aggregation processing is performed to obtain the final expanded picture, the overall expansion is processed according to the actual edge segmentation result, different processing modes are adopted for different types of background regions, which is more suitable for the current background situation, thereby improving the accuracy of picture expansion. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 is a flowchart of a picture expansion method based on edge detection and physical simulation provided by an embodiment of the present application.

[0058] Figure 2 is a structural block diagram of a picture expansion device based on edge detection and physical simulation provided by an embodiment of the present application.

[0059] Figure 3 is a structural block diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0060] The present application will be further described in detail below with reference to the accompanying drawings.

[0061] An embodiment of the present application provides a picture expansion method based on edge detection and physical simulation, which can be executed by an electronic device. The electronic device can be a server or a terminal device. The server can be a standalone physical server, a server cluster composed of multiple physical servers or a distributed system, or a cloud server providing cloud computing services. The terminal device can be a smart phone, a tablet computer, a desktop computer, etc., but is not limited thereto.

[0062] Figure 1 is a flowchart of a picture expansion method based on edge detection and physical simulation provided by an embodiment of the present application.

[0063] As shown in Figure 1 , the main process of the method is described as follows (steps S101-S104):

[0064] In step S101, picture edge segmentation parameters and a picture to be expanded are obtained.

[0065] In this embodiment, the picture to be expanded is a picture that needs to be expanded, and the picture edge segmentation parameters are set segmentation percentages, including maximum segmentation parameters, minimum segmentation parameters and fixed parameters. The picture to be expanded is segmented according to the set picture edge segmentation parameters.

[0066] In step S102, edge detection segmentation is performed on the picture to be expanded based on the picture edge segmentation parameters, to generate an edge segmentation result.

[0067] For step S102, the picture size, the segmentation fixed value and the edge intensity threshold value of the picture to be expanded are obtained; the maximum initial value and the minimum initial value of the picture to be expanded are determined based on the picture size and the picture edge segmentation parameter; it is judged whether the maximum initial value and the minimum initial value are within the numerical range of the segmentation fixed value; if the maximum initial value and the minimum initial value are within the numerical range of the segmentation fixed value, the edge background region of the picture to be expanded is determined based on the maximum initial value and the minimum initial value; if the maximum initial value and the minimum initial value are not within the numerical range of the segmentation fixed value, the edge background region of the picture to be expanded is determined based on the segmentation fixed value; the edge background region is segmented into four regions of top, bottom, left and right based on the edge intensity threshold value, and an edge segmentation result is generated.

[0068] In the present embodiment, in the field of picture expansion, compared with the key subject of the picture, the region close to the edge of the picture is more important, which is often the background region of the picture, and the essence of picture expansion is to reasonably extend the background region. When processing, the edge background region needs to be accurately obtained from the picture. If the region is too small, the specific features of the edge background region cannot be extracted, and if the region is too large, too many irrelevant elements will be mixed in, thereby affecting the subsequent further expansion.

[0069] When dividing the background region, the limit maximum and minimum values of the region are set through the picture size ratio or the fixed size, the 5% and 20% of the picture are taken as the edge segmentation parameters, and two initial values, i.e. the maximum initial value and the minimum initial value, are calculated using the edge segmentation parameters. The edge segmentation parameter is obtained by inversely calculating the average size of the picture edge region after extracting the position of the picture subject from the real data. If the segmentation is too large, the picture subject elements are likely to be included, and if the segmentation is too small, the features of the edge region cannot be reasonably analyzed. At the same time, in order to avoid extreme values caused by the proportional value of the extreme size of the input picture, a segmentation fixed value is set, i.e. the two initial values of the input picture should satisfy the numerical range, and if not, the fixed value is applied for edge region segmentation. Then the region preliminarily divided is analyzed for the edge texture in the picture to obtain the edge feature intensity of the picture. If the edge feature intensity exceeds the edge intensity threshold value, it can be regarded as a strong region transformation feature in the picture, and the position is taken as the segmentation point of the edge region of the picture. If there is no point exceeding the edge intensity threshold value, the position with the maximum intensity is selected. The edge intensity threshold value is set to 100, and the edge intensity threshold value is obtained by statistical processing according to the real data and the subject edge features of the real data. Too high will cause the subject element region to be classified as the edge region, affecting the subsequent edge keyword extraction, and too small will cause the edge region to be too small to reasonably analyze the features of the edge region. According to the above method, the edge regions above, below, left and right of the picture are obtained in turn, and the pictures of the four regions are taken as the edge segmentation result.

[0070] The edge intensity calculation formula is:

[0071] ;

[0072] wherein x, y are the coordinate positions of the pixel points in the picture.

[0073] In step S103, an edge region histogram of the edge segmentation result is obtained.

[0074] In this embodiment, after obtaining the edge segmentation result, color extraction is performed on the edge segmentation result, and an edge region histogram belonging to the edge segmentation result is constructed. The image histogram can play a role similar to clustering the picture in the calculation process.

[0075] In step S104, the background color data of the edge segmentation result is determined based on the edge region histogram.

[0076] For step S104, an adaptive weight parameter is obtained; the main labeling region of the edge segmentation result is labeled based on the edge region histogram and the adaptive weight parameter; and the background color data of the easy region is calculated based on a preset algorithm and the main labeling region.

[0077] In this embodiment, in order to better detect the solid color gradient background, the irrelevant elements, i.e. the foreground, in the region need to be removed first. By adaptively adjusting the weight size through the variance between histogram wave peaks, the main region of the picture can be automatically marked. The formula is as follows:

[0078] ;

[0079] wherein, is the variance, which refers to the variance of the pixel value distribution of the valley region between two adjacent wave peaks in the histogram distribution. The variance of the valley region between the adjacent wave peaks can reflect the separation degree of the two main regions. ω is the adaptive weight parameter, which can adjust the value of the final threshold T according to the saliency of the wave peak.

[0080] Through the above method, the initial rough segmentation is completed. The edge segmentation result is directly applied to the subsequent watershed algorithm as the initial region seed, without manual intervention for parameter adjustment. The initial seed is applied to the watershed algorithm to more finely segment the foreground and background of the picture. The obtained foreground is extracted, and the background is reserved as the background color data. In the traditional watershed algorithm, water injection is simulated from the marked position, and a gradient remapping method is used to suppress noise interference. The specific formula is as follows:

[0081] ;

[0082] Wherein, G(x, y) is the gradient of the coordinates in the picture, η is the noise suppression threshold, the noise suppression threshold is 40 in the present application, the threshold is obtained by statistically processing the noise information characteristics of real data, the threshold is relatively large in actual processing, but too high or too low will affect the actual use performance of the watershed algorithm.

[0083] In step S105, the background region type of the edge segmentation result is determined based on the background color data and the preset model.

[0084] In step S105, it is determined whether the background color of the edge segmentation result is a pure color gradient based on the background color data; if the background color of the edge segmentation result is not a pure color gradient, it is determined that the background region type of the edge background region is a pure color gradient background; if the background color of the edge segmentation result is not a pure color gradient, it is determined that the background region type of the edge background region is a natural image background.

[0085] In the present embodiment, the variances of RGB channels and HSV channels are calculated for the extracted background color data, and the data whose variances are not within the threshold range will be directly determined as a natural picture, the intensity threshold is 50, the threshold is obtained by statistically processing the color gradient characteristics of real data, too high will cause the pure color gradient region to be processed as a natural picture, and too small will cause the subsequent pure color gradient picture to be unable to be normally processed. For the picture whose variance is within the interval, the clustering algorithm is used to analyze the distribution of the values, if the clustering result shows that there are multiple clusters and the distribution weights of different clusters are relatively uniform, it can be determined that the edge region is a natural image background, otherwise it is a pure color gradient background.

[0086] In step S106, the picture expansion processing is performed on the picture to be expanded based on the background region type and the preset physical simulation technology, and an expanded picture is generated.

[0087] In step S106, the keyword extraction is performed on the natural image background based on the preset visual language model, and the region keywords are generated; the region keywords of the four regions of the edge segmentation result are optimized and recombined, and the optimized prompt words are generated; the picture expansion processing is performed on the picture to be expanded based on the pure color gradient background, the optimized prompt words and the preset simulation technology, and the expanded picture is generated.

[0088] In this embodiment, the natural image background is input into a visual language model (vlm), and similar models include CogVLM2 and other multi-modal large models. The visual language model can output more targeted natural image background descriptions after excluding the subject interference, such as sky, earth, grass, etc. The region keywords extracted by the visual language model in the upper, lower, left, and right regions are uniformly input into a text generation type large language model (LLM), and all keywords are uniformly reorganized and optimized to obtain the final prompt words. The extraction instruction can be: These key words are partial descriptions of an image. Please help me reorganize and optimize them reasonably to make them more suitable for the prompt words of the generative model.

[0089] Further, based on the pure color gradient background, the optimized prompt words, and the preset simulation technology, the picture expansion processing is performed on the to-be-expanded picture to generate an expanded picture. The color diffusion simulation based on the preset simulation technology is performed on the pure color gradient background to generate a color diffusion result. The edge segmentation result is expanded based on the color diffusion result to generate a first expansion region. The edge segmentation result is expanded based on the preset picture generation type large model, the optimized prompt words, and the natural image background to generate a second expansion region. The to-be-expanded picture is expanded based on the first expansion region and the second expansion region to generate an expanded picture.

[0090] In this embodiment, the edge regions in the edge segmentation results of different background region types are processed by a double-branch method. For a segmentation region with a pure color gradient background, the color diffusion process of the segmentation region is physically simulated and estimated by using the N-S equation. The simulation process is a continuous physical process similar to fluid diffusion. The equation solving target is to generate a smooth color diffusion field The algorithm iteratively calculates the unique solution of the equation , that is, the pixel value distribution of the diffused image. The final result is a gradual color field that naturally transitions from the edge to the to-be-expanded region. The color gradient effect more consistent with the original image is calculated by simulation to complete the picture expansion. In this application, the N-S equation is implemented in the picture expansion field as follows:

[0091] ;

[0092] wherein, is the image contour direction, is the image second derivative direction, For the picture expansion solution, x, y are the coordinate positions in the picture. The specific formula is:

[0093] ;

[0094] ;

[0095] Then, the region with a natural image background type and the corresponding prompt word are sent into the picture generation model for picture expansion. The pure color expansion result and the natural image expansion result are combined according to the up, down, left and right directions in the edge segmentation result to obtain the final result.

[0096] In the embodiment, the to-be-expanded picture and the expanded picture are divided into expansion regions to generate a division result. The expansion regions of the expanded picture are labeled based on the division result.

[0097] After the to-be-expanded picture is expanded to obtain the expanded picture, the to-be-expanded picture and the expanded picture are compared to determine the expansion region. The region is used as the division result, and the division result is used to label the expansion region of the expanded picture. According to the division result, it can be quickly determined which region is obtained after expansion, which facilitates subsequent staff to analyze the expansion effect.

[0098] Figure 2 A structure block diagram of a picture expansion device 200 based on edge detection and physical simulation is provided for the application embodiment.

[0099] As shown in Figure 2 , the picture expansion device 200 based on edge detection and physical simulation mainly includes:

[0100] The picture data acquisition module 201 is configured to acquire picture edge segmentation parameters and a to-be-expanded picture.

[0101] The segmentation result generation module 202 is configured to perform edge detection segmentation on the to-be-expanded picture based on the picture edge segmentation parameters to generate an edge segmentation result.

[0102] The histogram data acquisition module 203 is configured to acquire an edge region histogram of the edge segmentation result.

[0103] The background data determination module 204 is configured to determine background color data of the edge segmentation result based on the edge region histogram.

[0104] The background type determination module 205 is configured to determine a background region type of the edge segmentation result based on the background color data and a preset model.

[0105] The expansion picture generation module 206 is configured to perform picture expansion processing on the to-be-expanded picture based on the background region type and a preset physical simulation technology, and generate an expanded picture.

[0106] As an optional implementation of the present embodiment, the segmentation result generation module 202 is specifically configured to obtain a picture size of the to-be-expanded picture, a segmentation fixed value, and an edge intensity threshold value; determine a maximum initial value and a minimum initial value of the to-be-expanded picture based on the picture size and picture edge segmentation parameters; determine whether the maximum initial value and the minimum initial value are within a numerical range of the segmentation fixed value; if the maximum initial value and the minimum initial value are within the numerical range of the segmentation fixed value, determine an edge background region of the to-be-expanded picture based on the maximum initial value and the minimum initial value; if the maximum initial value and the minimum initial value are not within the numerical range of the segmentation fixed value, determine the edge background region of the to-be-expanded picture based on the segmentation fixed value; and segment the edge background region into four regions of top, bottom, left and right based on the edge intensity threshold value, and generate an edge segmentation result.

[0107] As an optional implementation of the present embodiment, the background data determination module 204 is specifically configured to obtain an adaptive weight parameter; label a main labeling region of the edge segmentation result based on the edge region histogram and the adaptive weight parameter; and calculate background color data of the to-be-labeled region based on a preset algorithm and the main labeling region.

[0108] As an optional implementation of the present embodiment, the background type determination module 205 is specifically configured to determine whether the background color of the edge segmentation result is a pure color gradient based on the background color data; if the background color of the edge segmentation result is not a pure color gradient, determine that the background region type of the edge background region is a pure color gradient background; and if the background color of the edge segmentation result is not a pure color gradient, determine that the background region type of the edge background region is a natural image background.

[0109] As an optional implementation of the present embodiment, the expansion picture generation module 206 includes:

[0110] The keyword generation module is configured to perform keyword extraction on the natural image background based on a preset visual language model, and generate region keywords;

[0111] The keyword optimization module is configured to optimize and recombine the region keywords of the four regions of top, bottom, left and right in the edge segmentation result, and generate optimized prompt words.

[0112] The picture expansion processing module is configured to perform picture expansion processing on the to-be-expanded picture based on the pure color gradient background, the optimized prompt words, and a preset simulation technology, and generate an expanded picture.

[0113] In the optional embodiment, the picture expansion processing module is specifically configured to perform color diffusion simulation on the pure color gradient background based on a preset simulation technology to generate a color diffusion result; expand the edge segmentation result based on the color diffusion result to generate a first expansion region; expand the edge segmentation result based on a preset picture generation model, an optimization prompt word, and a natural image background to generate a second expansion region; and expand the edge segmentation result based on the first expansion region and the second expansion region to generate an expanded picture.

[0114] As an optional implementation of the embodiment, the picture expansion device 200 based on edge detection and physical simulation further includes:

[0115] The division result generation module is configured to divide the to-be-expanded picture and the expanded picture into expansion regions to generate a division result.

[0116] The expansion region labeling module is configured to label the expansion regions of the expanded picture based on the division result.

[0117] In one example, the modules in any of the above devices can be one or more integrated circuits configured to implement the above methods, for example, one or more application specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs), or a combination of at least two of these integrated circuit forms.

[0118] For another example, when the modules in the device can be implemented in the form of a processing element scheduler, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call programs. For another example, these modules can be integrated together to be implemented in the form of a system-on-a-chip (SOC).

[0119] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0120] Figure 3 The structural block diagram of the electronic device 300 provided by the embodiment of the present application is shown.

[0121] As Figure 3As shown, the electronic device 300 includes a processor 301 and a memory 302, and can further include one or more of an information input / output (I / O) interface 303, a communication component 304, and a communication bus 305.

[0122] The processor 301 is configured to control overall operations of the electronic device 300 to complete all or part of the steps of the picture expansion method based on edge detection and physical simulation described above. The memory 302 is configured to store various types of data to support operations of the electronic device 300, which can include, for example, instructions for any application or method operating on the electronic device 300, and application-related data. The memory 302 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as one or more of a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, or an optical disk.

[0123] The I / O interface 303 provides an interface between the processor 301 and other interface modules, which can be a keyboard, a mouse, a button, and the like. The buttons can be virtual buttons or physical buttons. The communication component 304 is configured to perform wired or wireless communication between the electronic device 300 and other devices. The wireless communication, such as Wi-Fi, Bluetooth, near field communication (NFC), 2G, 3G, or 4G, or a combination of one or more of them, so the corresponding communication component 304 can include a Wi-Fi component, a Bluetooth component, and an NFC component.

[0124] The electronic device 300 can be implemented with one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic elements for performing the picture expansion method based on edge detection and physical simulation given in the above-described embodiments.

[0125] The communication bus 305 can include a path for transmitting information between the above-described components. The communication bus 305 can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, or the like. The communication bus 305 can be divided into an address bus, a data bus, a control bus, and the like.

[0126] The electronic device 300 can include, but is not limited to, a mobile terminal of a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (Personal Digital Assistant), a PAD (a tablet computer), a PMP (a portable multimedia player), a car terminal (for example, a car navigation terminal), and the like, and a fixed terminal such as a digital TV, a desktop computer, and the like, and can be a server or the like.

[0127] The present application also provides a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the steps of the picture expansion method based on edge detection and physical simulation described above.

[0128] The computer-readable storage medium can include a U disk, a mobile hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and the like, various media that can store program codes.

[0129] The terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0130] The description above merely illustrates preferred embodiments of the present application and the principles of the technology employed. It should be understood by those skilled in the art that the scope of the application involved in the present application is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by the combinations of the above technical features or their equivalent features without departing from the above application concept. For example, the above technical features can be replaced with the technical features applied in the present application (but not limited to) having similar functions to form technical solutions.

Claims

1. An image augmentation method based on edge detection and physical simulation, characterized in that, include: Obtain image edge segmentation parameters and the image to be expanded; Based on the image edge segmentation parameters, perform edge detection and segmentation on the image to be expanded to generate an edge segmentation result; Obtain the edge region histogram of the edge segmentation result; The background color data of the edge segmentation result is determined based on the edge region histogram; The background region type of the edge segmentation result is determined based on the background color data and the preset model; Based on the background region type and preset physical simulation technology, the image to be expanded is processed to generate an expanded image.

2. The method according to claim 1, characterized in that, The step of performing edge detection and segmentation on the image to be expanded based on the image edge segmentation parameters, and generating edge segmentation results, includes: Obtain the image size, segmentation fixed value, and edge strength threshold of the image to be expanded; The maximum and minimum initial values ​​of the image to be expanded are determined based on the image size and the image edge segmentation parameters. Determine whether the maximum initial value and the minimum initial value are within the numerical range of the segmented fixed value; If the maximum initial value and the minimum initial value are within the numerical range of the segmentation fixed value, then the edge background region of the image to be expanded is determined based on the maximum initial value and the minimum initial value. If the maximum initial value and the minimum initial value are not within the numerical range of the segmentation fixed value, then the edge background region of the image to be expanded is determined based on the segmentation fixed value. The edge background region is segmented into four regions (top, bottom, left, and right) based on the edge intensity threshold to generate an edge segmentation result.

3. The method according to claim 1, characterized in that, The background color data used to determine the edge segmentation result based on the edge region histogram includes: Obtain adaptive weight parameters; The main labeled regions of the edge segmentation result are labeled based on the edge region histogram and the adaptive weight parameters; The background color data of the convenient area is calculated based on the preset algorithm and the main marked area.

4. The method according to claim 2, characterized in that, The background region type determined based on the background color data and the preset model for the edge segmentation result includes: Based on the background color data, determine whether the background color of the edge segmentation result is a solid color gradient; If the background color of the edge segmentation result is not a solid color gradient, then the background region type of the edge background region is determined to be a solid color gradient background. If the background color of the edge segmentation result is not a solid color gradient, then the background region type of the edge background region is determined to be a natural image background.

5. The method according to claim 4, characterized in that, The step of performing image expansion processing on the image to be expanded based on the background region type and a preset physical simulation technology to generate an expanded image includes: Based on a preset visual language model, keywords are extracted from the background of the natural image to generate region keywords; The keywords of the four regions (top, bottom, left, and right) in the edge segmentation result are optimized and recombined to generate optimized prompt words; Based on the solid color gradient background, the optimized prompt words, and the preset simulation technology, the image to be expanded is processed to generate an expanded image.

6. The method according to claim 5, characterized in that, The step of performing image expansion processing on the image to be expanded based on the solid color gradient background, the optimized prompt words, and the preset simulation technology to generate an expanded image includes: Based on the preset simulation technology, a color diffusion simulation is performed on the solid color gradient background to generate a color diffusion result; The edge segmentation result is expanded based on the color diffusion result shown to generate a first expanded region; The edge segmentation result is expanded based on the preset image generation model, the optimized prompt words, and the natural image background to generate a second expanded region; The image to be expanded is expanded based on the first expanded region and the second expanded region to generate an expanded image.

7. The method according to claim 1, characterized in that, After performing image expansion processing on the image to be expanded based on the background region type and a preset physical simulation technology to generate an expanded image, the process further includes: The image to be expanded and the expanded image are divided into expanded regions to generate a division result; The extended regions of the extended image are labeled based on the segmentation results.

8. An image augmentation device based on edge detection and physical simulation, characterized in that, include: The image data acquisition module is used to acquire image edge segmentation parameters and the image to be expanded. The segmentation result generation module is used to perform edge detection and segmentation on the image to be expanded based on the image edge segmentation parameters, and generate edge segmentation results. The histogram data acquisition module is used to acquire the edge region histogram of the edge segmentation result; The background data determination module is used to determine the background color data of the edge segmentation result based on the edge region histogram. The background type determination module is used to determine the background region type of the edge segmentation result based on the background color data and a preset model. An extended image generation module is used to perform image extension processing on the image to be extended based on the background region type and a preset physical simulation technology, and generate an extended image.

9. An electronic device, characterized in that, Includes a processor, which is coupled to a memory; The processor is configured to execute a computer program stored in the memory to cause the electronic device to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It includes a computer program or instructions that, when run on a computer, cause the computer to perform the method as described in any one of claims 1 to 7.

Images