Scene image processing method and device, electronic equipment and storage medium

Abstract

The invention relates to a scene image processing method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a to-be-processed image; inputting the to-be-processed image into a key point identification model to obtain a plurality of key points of the to-be-processed image and coordinate information corresponding to the key points; and adjusting the to-be-processed image according to the coordinate information of the key points to obtain a target scene image. Errors caused by human factors are effectively reduced, the efficiency and accuracy of scene image processing are greatly improved, and meanwhile the high consistency of the generated scene image and the original image content is ensured. Besides, a coordinate conversion principle is adopted in an image adjustment stage, coordinate information of a target key point is converted into corresponding scene coordinate information, accurate zooming, cutting or filling processing of an original image is realized, scene conversion of the image is ensured to meet professional requirements, key information of the image is reserved, and integrity of the image is enhanced.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a scene image processing method, device, electronic device and storage medium. Background Art

[0002] In the modern film and television production process, generating shots of specific scenes often requires manually extracting corresponding frames from massive amounts of film and television video materials and performing detailed labeling, which is not only time-consuming and labor-intensive, but also requires a lot of manpower investment.

[0003] However, even with a huge investment of manpower and material resources, the scene images generated by this method are still less consistent with the original image content due to inevitable errors in the manual extraction and labeling process, resulting in the final generated image being unable to fully and accurately reproduce the scenes and details in the original video.

[0004] Therefore, there is an urgent need to develop a scene image processing method, device, electronic device and storage medium to solve one or more of the above-mentioned problems. Summary of the invention

[0005] In view of this, in order to solve the above technical problems or part of the technical problems, the embodiments of the present invention provide a scene image processing method, device, electronic device and storage medium.

[0006] In a first aspect, the present application provides a scene image processing method, the method comprising:

[0007] Get the image to be processed;

[0008] Inputting the image to be processed into a key point recognition model to obtain a plurality of key points of the image to be processed and coordinate information corresponding to the key points;

[0009] According to the coordinate information of the key points, the image to be processed is adjusted to obtain a target scene image.

[0010] In a possible implementation, adjusting the image to be processed according to the coordinate information of the key point to obtain the target scene image includes:

[0011] According to the coordinate information of the key point, obtaining the scene coordinate information corresponding to the image to be processed;

[0012] According to the scene coordinate information, the image to be processed is adjusted to obtain an initial scene image;

[0013] The initial scene image is redrawn to obtain a target scene image.

[0014] In a possible implementation, obtaining the scene coordinate information corresponding to the image to be processed according to the coordinate information of the key point includes:

[0015] Acquiring scene conversion parameters of the image to be processed, wherein the scene conversion parameters include near-view conversion parameters, mid-view conversion parameters, or far-view conversion parameters;

[0016] Selecting a target key point corresponding to the scene conversion parameter from a plurality of key points according to the coordinate information of the key point;

[0017] Determining a coordinate transformation rule of the image to be processed according to the scene transformation parameters;

[0018] The coordinate conversion rule is used to convert the coordinate information of the target key point into the scene coordinate information corresponding to the image to be processed.

[0019] In a possible implementation, performing image adjustment on the image to be processed according to the scene coordinate information to obtain an initial scene image includes:

[0020] In the case where the scene conversion parameter of the image to be processed is a distant scene conversion parameter, the image to be processed is reduced according to the scene coordinate information to obtain a reduced scene image;

[0021] Performing blank filling processing on the reduced scene image to obtain an initial scene image;

[0022] The initial scene image has the same size as the image to be processed.

[0023] In a possible implementation, performing image adjustment on the image to be processed according to the scene coordinate information to obtain an initial scene image includes:

[0024] When the scene conversion parameter of the image to be processed is a near scene conversion parameter or a mid scene conversion parameter, the image to be processed is enlarged according to the scene coordinate information to obtain an enlarged scene image;

[0025] Performing cropping processing on the enlarged scene image to obtain an initial scene image;

[0026] The initial scene image has the same size as the image to be processed.

[0027] In a possible implementation manner, redrawing the initial scene image to obtain a target scene image includes:

[0028] When the scene conversion parameter of the image to be processed is a distant scene conversion parameter, obtaining a blank area in the initial scene image;

[0029] Marking and filling labels on the blank area to obtain a scene-marked image;

[0030] The scene mark image is input into an image filling model to obtain a target scene image.

[0031] In a possible implementation manner, redrawing the initial scene image to obtain a target scene image further includes:

[0032] When the scene conversion parameters of the image to be processed are near-scene conversion parameters or mid-scene conversion parameters, the initial scene image is input into an image restoration model to obtain a target scene image.

[0033] In a second aspect, the present application provides a scene image processing device, the device comprising:

[0034] An acquisition module, used for acquiring an image to be processed;

[0035] A recognition model is used to input the image to be processed into a key point recognition model to obtain a plurality of key points of the image to be processed and coordinate information corresponding to the key points;

[0036] The adjustment module is used to adjust the image to be processed according to the coordinate information of the key points to obtain a target scene image.

[0037] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the scene image processing method described in any one of the embodiments of the first aspect are implemented.

[0038] In a fourth aspect, the present application further provides a computer storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the scene image processing method described in any one of the embodiments of the first aspect are implemented.

[0039] The above-mentioned technical scheme provided by the embodiment of the present application has the following advantages over the prior art: the scene image processing method provided by the embodiment of the present application, in the process of generating scene images, uses a key point recognition model to automatically identify the key points in the image to be processed, and then adjusts the image based on the key points to obtain the scene image. The acquisition process of the scene image does not require human intervention, which effectively reduces the errors caused by human factors, greatly improves the efficiency and accuracy of scene image processing, and ensures a high degree of consistency between the generated scene image and the original image content; in addition, the coordinate transformation principle is adopted in the image adjustment stage, and the coordinate information of the target key points is accurately calculated and converted into corresponding scene coordinate information, so as to realize accurate scaling, cropping or filling processing of the original image, which not only ensures that the scene conversion of the image meets professional requirements, but also retains the key information of the image to enhance the integrity of the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0042] One or more embodiments are exemplarily described by the images in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0043] Figure 1 A schematic diagram of a flow chart of a scene image processing method provided in an embodiment of the present application;

[0044] Figure 2 An example diagram of key points provided for the embodiments of the present application;

[0045] Figure 3 A schematic flow chart of another scene image processing method provided in an embodiment of the present application;

[0046] Figure 4 An example diagram of close-up image processing provided in an embodiment of the present application;

[0047] Figure 5 An example diagram of mid-range and close-range image processing provided in an embodiment of the present application;

[0048] Figure 6An example diagram of mid-range image processing provided in an embodiment of the present application;

[0049] Figure 7 An example diagram of panoramic image processing provided in an embodiment of the present application;

[0050] Figure 8 An example diagram of distant image processing provided in an embodiment of the present application;

[0051] Fig. 9 A schematic diagram of the image filling process provided in an embodiment of the present application;

[0052] Fig.10 A schematic diagram of the image restoration process provided by an embodiment of the present application;

[0053] Fig.11 A schematic diagram of the structure of a scene image processing device provided in an embodiment of the present application;

[0054] Fig.12 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0056] The disclosure below provides many different embodiments or examples to implement different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0057] In order to solve the problem in the prior art that when generating a shot image of a specific scene, it is often necessary to manually extract the corresponding picture frames from a large amount of film and television video materials and perform detailed annotation work, which not only consumes time and energy, but also requires a large amount of manpower investment. Even if a large amount of manpower and material resources are invested, due to the inevitable errors in the manual extraction and annotation process, the consistency between the resulting scene image and the original image content is still low, resulting in the technical problem that the final generated image cannot completely and accurately reproduce the scene and details in the original video, the present application provides a scene image processing method, device, electronic device and storage medium, which effectively reduce the errors caused by human factors, greatly improve the efficiency and accuracy of scene image processing, and at the same time ensure that the generated scene image is highly consistent with the original image content, and realize accurate scaling, cropping or filling processing of the original image, which not only ensures that the scene conversion of the image meets professional requirements, but also retains the key information of the image and enhances the integrity of the image.

[0058] Figure 1 A schematic diagram of a flow chart of a scene image processing method provided in an embodiment of the present application is shown as follows: Figure 1 As shown, the method specifically includes:

[0059] S101, obtaining an image to be processed;

[0060] In this embodiment, an image to be processed is first determined, and the image is processed to identify the content in the image, and further the lens object in the image to be processed is determined for subsequent processing and analysis.

[0061] Specifically, by observing and analyzing the image, the main elements and features in the image are identified, thereby determining the specific position and properties of the lens object to ensure accurate identification and positioning of the lens object.

[0062] S102, inputting the image to be processed into a key point recognition model to obtain a plurality of key points of the image to be processed and coordinate information corresponding to the key points;

[0063] In this embodiment, the image to be processed that needs to be converted into a specific scene image is input into a key point recognition model. The key point recognition model can be a pre-trained deep learning model for automatically identifying key points in the image and outputting specific position information of multiple key points in the image.

[0064] It should be noted that the key point recognition model can be a deep learning model, such as a convolutional neural network (CNN), which is used to automatically identify key points in an image. The key points output by the key point recognition model can be not only significant feature points in the image, but also details in the image. The model outputs the coordinate information corresponding to the key points, which records the specific position of each key point in the image in detail. Through these coordinate data, we can further analyze, process and understand the image, thereby achieving more accurate image recognition and processing tasks.

[0065] For example, when the target object of the scene conversion in the image to be processed is a person, facial feature points, body posture key points or other image feature points representative of the person can be used as key points. By identifying these key points, the target person can be accurately adjusted and processed.

[0066] like Figure 2 As shown, for example, the open source project Open Pose detects the coordinates of key points of the human body in the panoramic image. The head coordinates, left shoulder coordinates, right shoulder coordinates, hip coordinates, left knee coordinates, right knee coordinates, left foot coordinates, and right foot coordinates are key coordinate data in subsequent steps.

[0067] S103: adjusting the image to be processed according to the coordinate information of the key points to obtain a target scene image.

[0068] In this embodiment, the image to be processed is adjusted according to the coordinate information of the key points, so as to obtain a target scene image.

[0069] Specifically, by accurately locating the key points and analyzing the coordinate information, the image to be processed is cropped and scaled to ensure that the final generated target scene image meets the requirements of the expected scene image.

[0070] In this embodiment, we first obtain the coordinate information of each key point in the image to be processed. Based on the coordinate information of these key points, we perform a series of adjustment operations on the image, and finally obtain an image of the target scene that meets the expectations.

[0071] The scene image processing method provided in this embodiment first obtains the image to be processed, and then inputs the image into the key point recognition model, which can automatically identify the key points in the image to be processed; based on these key points, the image is adjusted accordingly to generate the required scene image, and the entire scene image acquisition process does not require human intervention at all, which effectively reduces the errors caused by human operation, not only greatly improves the efficiency of scene image processing, but also significantly improves the accuracy of processing; in addition, in the image adjustment stage, this embodiment adopts the coordinate transformation principle, by accurately calculating the coordinate information of the target key point and converting it into the corresponding scene coordinate information, it realizes the accurate scaling, cropping or filling processing of the original image. This method not only ensures that the scene conversion of the image meets professional requirements, but also retains the key information in the image, thereby ensuring that the generated scene image maintains a high degree of consistency with the original image in terms of content.

[0072] In an optional solution of the embodiment of the present invention, adjusting the image to be processed according to the coordinate information of the key point to obtain the target scene image includes:

[0073] According to the coordinate information of the key point, obtaining the scene coordinate information corresponding to the image to be processed;

[0074] In this embodiment, the coordinate information of the key points of the image to be processed is converted into the scene coordinates of the target scene image by using the conversion rule of converting the image to be processed into the scene coordinates of the target scene image.

[0075] In this embodiment, the key point coordinate information in the image to be processed is extracted, and then converted into the scene coordinates in the target scene image according to the key point coordinate information, so that the scene coordinates can be used to perform image processing on the image to be processed to obtain the scene image.

[0076] Specifically, this conversion process involves complex mathematical operations and image processing techniques to ensure that the coordinate information remains accurate and consistent during the conversion process, including determining the target key points according to the scene requirements, and converting the coordinate information of the target key points in the image to be processed to obtain the scene coordinate information of the target key points in the scene image. After the conversion is completed, these scene coordinates are used to perform further image processing operations on the original image to be processed to generate the required scene image.

[0077] According to the scene coordinate information, the image to be processed is adjusted to obtain an initial scene image;

[0078] In this embodiment, after the scene coordinate information is obtained through conversion, the image to be processed is scaled and adjusted using the scene coordinate information to obtain an initial scene image.

[0079] In this embodiment, after the scene coordinate information is acquired through the conversion process, the scene coordinate information is used to scale and adjust the image to be processed, thereby obtaining a preliminary scene image.

[0080] Specifically, the scaling adjustment process is performed based on the acquired scene coordinate information, with the aim of ensuring that specific areas or objects in the image can be correctly located and displayed, so as to facilitate subsequent image processing steps, ensure the accuracy of the image's visual effects and content, and thus provide a good foundation for subsequent image processing.

[0081] The initial scene image is redrawn to obtain a target scene image.

[0082] In this embodiment, the initial scene image obtained by preliminary scaling may be blurred or distorted to a certain extent, so the initial scene image is redrawn to improve the image quality and ensure the clarity of the target scene image.

[0083] Specifically, the redrawing process can adopt a variety of image enhancement technologies, such as super-resolution reconstruction, image denoising, edge sharpening, etc., to improve the clarity and detail of the image.

[0084] In addition, the redrawing process can also utilize deep learning models, such as generative adversarial networks (GANs) or convolutional neural networks (CNNs), to optimize the initial scene images, which can quickly learn the features of high-quality images and apply these features to the initial scene images, thereby generating clearer and more restored target scene images.

[0085] The scene image processing method provided in this embodiment first determines the scene coordinate information corresponding to the image to be processed based on the coordinate information of the key points, and then uses the scene coordinate information to perform corresponding image adjustment operations on the original image to be processed, thereby obtaining a preliminary scene image. In order to further improve the image quality, the initial scene image is redrawn to finally obtain a clear and high-quality target scene image. In the image processing process, coordinate conversion and image redrawing are used to improve the intelligence and efficiency of image processing, and the key point coordinate information can be quickly and accurately identified, and the image can be intelligently adjusted to achieve the best visual effect, and a high-quality target scene image can be generated in a shorter time, which greatly improves the efficiency and effect of image processing.

[0086] Figure 3 A flowchart of another scene image processing method provided in an embodiment of the present application is shown as follows: Figure 3 As shown, the method specifically includes:

[0087] S301, obtaining scene conversion parameters of the image to be processed, wherein the scene conversion parameters include near scene conversion parameters, mid scene conversion parameters, or far scene conversion parameters;

[0088] In this embodiment, the scene conversion parameters of the image to be processed are obtained, and these scene conversion parameters include close-up conversion parameters, mid-range conversion parameters, and distant conversion parameters. Specifically, the close-up conversion parameters are used to adjust the visual effect of the foreground object in the image to make it more prominent and clear; the mid-range conversion parameters are used to optimize the detail performance of the objects in the middle distance in the image to make it more coordinated and natural visually; and the distant conversion parameters focus on adjusting the visual effect of the background part of the image to make it deeper and more layered; by using the scene conversion parameters, the overall visual effect of the image can be effectively improved, and the panoramic image can be converted into multiple scene images of different scene types.

[0089] It should be noted that in this plan, the medium shot actually covers three different shot sizes, namely medium shot, medium close shot and medium full shot. In order to simplify the expression, these three shot sizes are collectively referred to as medium shot to facilitate more convenience and efficiency in the discussion and implementation process.

[0090] S302, selecting a target key point corresponding to the scene conversion parameter from a plurality of key points according to the coordinate information of the key point;

[0091] In this embodiment, target key points that match specific scene conversion parameters are selected from a large number of key points based on the coordinate position information of the key points to ensure that the key points are completely consistent with the required scene conversion parameters, thereby ensuring that the selected key points can accurately reflect and adapt to changes in scene, and achieve coordination and consistency in visual effects and scene transitions.

[0092] like Figures 4 to 8 As shown, taking a person as an example, the head coordinates, left shoulder coordinates, right shoulder coordinates, hip coordinates, left knee coordinates, right knee coordinates, left foot coordinates, and right foot coordinates are key coordinate data in subsequent steps;

[0093] When the scene image is a close-up image, the close-up is defined as the head area occupying most of the area of ​​the picture, and the head key points, left shoulder key points and right shoulder key points are selected as target key points accordingly;

[0094] When the scene image is a medium scene image, the medium scene is defined as the area above the waist occupying most of the image area, and the head key point, left shoulder key point, right shoulder key point and hip key point are selected as the target key points;

[0095] When the scene image is a medium-close-up image, the medium-close-up is defined as the area above the head and chest occupying most of the image area, and the head key points, left shoulder key points, right shoulder key points and hip key points are selected as target key points;

[0096] When the scene image is a medium panoramic image, the medium panoramic image is defined as the area above the knees occupying most of the image area, and the head key points, left shoulder key points, right shoulder key points, hip key points, left knee key points and right knee key points are selected as target key points;

[0097] When the scene image is a distant view image, the distant view is defined as a human body that occupies less than one ninth of the image area, and correspondingly all key points of the human body are selected as target key points.

[0098] S303, determining a coordinate transformation rule of the image to be processed according to the scene conversion parameter;

[0099] According to the described scene transformation parameters, each key point in the image is mapped from the original coordinate system to the new coordinate system to achieve image scaling, rotation, translation and other transformations, thereby achieving the desired visual effect and ensuring that the image maintains its original geometric structure and visual information during the transformation process to avoid distortion or deformation.

[0100] like Figures 4 to 8 As shown, taking a person as an example, when the scene conversion parameter is a close-up conversion parameter, the coordinate conversion rule of the image to be processed is that the average of the left shoulder and the right shoulder is approximately the neck coordinate, and the head coordinate is subtracted from the neck coordinate and multiplied by 2, which is approximately equal to the head height value of the part to be cropped.

[0101] When the scene conversion parameter is a medium-close scene conversion parameter, the coordinate conversion rule of the image to be processed is the neck coordinate minus the hip coordinate, which is approximately the total height of the human torso, divided by two and added to the height of the head, which is the required half-body cropping range.

[0102] When the scene conversion parameter is a medium scene conversion parameter, the coordinate conversion rule of the image to be processed is to use the previously calculated torso and head height values ​​for cropping, and the bottom edge takes the coordinates of the hips.

[0103] When the scene conversion parameter is a medium panoramic conversion parameter, the coordinate conversion rule of the image to be processed is to subtract the mean coordinates of the left and right knees from the hip coordinates to approximately get the height of the thigh, and then use the previously calculated total height of the upper body for cropping, and the bottom edge takes the mean coordinates of the left and right knees.

[0104] When the scene conversion parameter is the distant view conversion parameter, the coordinate conversion rule of the image to be processed is to take three times the height of the human body in the original picture as the resolution basis of the new picture, which approximately meets the definition requirement of the area, and leaves space around it to be filled in the subsequent steps.

[0105] S304: using the coordinate conversion rule, convert the coordinate information of the target key point into scene coordinate information corresponding to the image to be processed.

[0106] In this embodiment, the coordinate information of the target key points is accurately converted according to the coordinate transformation rules, and the coordinate data of the target key points is recalculated and mapped according to the established transformation rules, so as to obtain the scene coordinate information corresponding to the image to be processed, ensuring that it can accurately reflect the position of the target object in the image to be processed in the new image coordinate system, thereby achieving consistency of the target object in images of different scene coordinates.

[0107] The scene image processing method provided in the embodiment of the present application obtains the scene conversion parameters of the image to be processed, and then selects the target key points corresponding to the scene conversion parameters from multiple key points based on the coordinate information of the key points; in this process, the coordinate conversion rules of the image to be processed are also determined based on the scene conversion parameters; the coordinate conversion rules are further used to convert the coordinate information of the target key points into the scene coordinate information corresponding to the image to be processed, effectively converting the image to be processed into an image of different scene with consistent content, thereby realizing the scene adjustment of the image.

[0108] In an optional solution of the embodiment of the present invention, the step of performing image adjustment on the image to be processed according to the scene coordinate information to obtain an initial scene image includes:

[0109] In the case where the scene conversion parameter of the image to be processed is a distant scene conversion parameter, the image to be processed is reduced according to the scene coordinate information to obtain a reduced scene image;

[0110] In this embodiment, when the scene of the image to be processed needs to be converted from the original state to the distant view, the preset distant view conversion parameters are first referred to to guide the image processing on how to convert the image to be processed into the target scene image;

[0111] Specifically, the scene parameter defines the key point coordinate scaling ratio and image processing method required in the process of converting the image to be processed into a distant scene image. Furthermore, according to the scene conversion parameter, the original image to be processed will be reduced in size to meet the distant visual effect.

[0112] It should be noted that during the zooming and translation process, it is necessary to ensure that the important areas defined by the key point coordinates, such as the head and torso, remain in the center of the picture, and adjust the size and position of the image to achieve the desired visual effect.

[0113] Performing blank filling processing on the reduced scene image to obtain an initial scene image;

[0114] The initial scene image has the same size as the image to be processed.

[0115] In this embodiment, after the reduction process is completed, a reduced scene image will be obtained. Since blank areas may appear at the edges of the image during the reduction process, the blank areas are filled to make the edge of the image look more natural and complete, thereby avoiding the appearance of inconsistent blank areas in the final image.

[0116] The scene image processing method provided in the embodiment of the present application, when the image to be processed needs to be converted into a distant view image, reduces the image to be processed according to the scene coordinate information obtained by the conversion, and fills the blank area of ​​the reduced image to obtain an initial scene image with the same size and consistent content as the image to be processed, thereby realizing the generation of high-quality distant view images using panoramic images to show the details and features of distant scenes.

[0117] In an optional solution of the embodiment of the present invention, the step of performing image adjustment on the image to be processed according to the scene coordinate information to obtain an initial scene image includes:

[0118] When the scene conversion parameter of the image to be processed is a near scene conversion parameter or a mid scene conversion parameter, the image to be processed is enlarged according to the scene coordinate information to obtain an enlarged scene image;

[0119] In this embodiment, when it is necessary to convert the scene of the image to be processed from the original state to a close-up or a medium-up, the preset close-up conversion parameters or the medium-up conversion parameters are first referred to to guide the image processing on how to convert the image to a close-up image or a medium-up image;

[0120] Specifically, the scene parameter defines the key point coordinate scaling ratio and translation direction required in the process of converting the image to be processed into a close-up scene image or a medium-view scene image. Furthermore, according to the scene conversion parameter, the original image to be processed will be enlarged and translated to meet the visual effect of a medium-close-up or medium-view scene.

[0121] It should be noted that during the zooming and panning process, ensure that the important areas defined by the key point coordinates, such as the head and torso, remain in the center of the picture, and adjust the size and position of the image to achieve the desired visual effect.

[0122] Performing cropping processing on the enlarged scene image to obtain an initial scene image;

[0123] The initial scene image has the same size as the image to be processed.

[0124] In this embodiment, the key point coordinate cropping range and image processing method required in the process of converting the image to be processed into a close-up or medium-shot scene image are cropped according to the scene conversion parameters to meet the visual effects of the close-up or medium-shot.

[0125] It should be noted that during the cropping process, it is ensured that the important areas defined by the key point coordinates, such as the head, shoulders, and knees, are retained, while the redundant parts are removed to achieve the desired visual effect.

[0126] In an optional solution of the embodiment of the present invention, the redrawing of the initial scene image to obtain the target scene image includes:

[0127] When the scene conversion parameter of the image to be processed is a distant scene conversion parameter, obtaining a blank area in the initial scene image;

[0128] In this embodiment, when processing an image, if the scene conversion parameters of the image to be processed are set to perspective conversion parameters, blank areas may appear at the edge of the image due to the reduction in image size caused by the reduction process, and the blank areas in the initial scene image are obtained.

[0129] Marking and filling labels on the blank area to obtain a scene-marked image;

[0130] The scene mark image is input into an image filling model to obtain a target scene image.

[0131] In this embodiment, the blank areas in the initial scene image are first filled and marked, and then deep learning technology is used to train an image filling model, which can intelligently fill the blank areas according to the surrounding image content; by inputting the marked scene marked image, the model can generate image content consistent with the style of the original image to fill the blank areas.

[0132] In addition, during the redrawing process of the target scene image, the image filling model will take into account the overall style and color distribution of the image to ensure that the newly generated content is seamlessly integrated with the original image, making the generated image content more realistic.

[0133] like Fig. 9 As shown in the figure, when processing the distant picture, in order to clearly distinguish the area that needs to be filled and the existing image content, the blank area around the picture is carefully marked with a mask; after the mask marking is completed, the diffusion model generation technology is used, such as the Stable Diffusion model, to redraw the mask-marked area, so that the filled area can be seamlessly integrated with the original picture, maintaining the overall quality and visual effect of the image.

[0134] It should be noted that, in the process of redrawing, in order to ensure the coherence and consistency of the image, image fusion technology can be used to seamlessly splice the newly generated image content with the original image, thereby avoiding obvious visual boundaries and unnatural transitions.

[0135] Exemplarily, the image fusion technology includes the following steps:

[0136] First, feature extraction is performed on the initial scene image and the target scene image to extract the key feature points and edge information in the image; then, the two images are aligned based on the extracted feature points and edge information to ensure that the newly generated image content is consistent with the original image in spatial position.

[0137] Furthermore, an image fusion algorithm is used to fuse the aligned images, wherein the image fusion algorithm can be a pixel-based fusion method or an image block-based fusion method; the pixel-based fusion method achieves fusion by calculating the weighted average of pixel values, while the image block-based fusion method divides the image into multiple blocks, fuses each block, and then recombines it into a complete image.

[0138] Finally, the fused image is post-processed, including steps such as color correction, sharpening, and noise reduction, to ensure that the generated image content is visually seamlessly integrated with the original image, achieving high-quality image conversion effects.

[0139] In an optional solution of the embodiment of the present invention, the redrawing of the initial scene image to obtain the target scene image further includes:

[0140] When the scene conversion parameters of the image to be processed are near-scene conversion parameters or mid-scene conversion parameters, the initial scene image is input into an image restoration model to obtain a target scene image.

[0141] In this embodiment, if the scene conversion parameters of the image to be processed are set to close-view conversion parameters or mid-view conversion parameters, then in this case, the initial scene image needs to be input into the image restoration model; the image restoration model has a powerful ability to process and convert images, so as to obtain the target scene image we need.

[0142] like Fig.10 As shown, since the close-up, medium-close-up, medium-shot, and medium-panorama are cropped contents, they will be blurred after being directly enlarged. Directly enlarging the image will cause the image to become blurred; it is necessary to use an image restoration model to restore the image, such as using a Stable Diffusion model to generate and redraw the image.

[0143] During the image restoration process, the Stable Diffusion model will set the noise reduction value below 0.1 to minimize changes to the original image content while ensuring that the quality of the restored image is as close to the original image as possible.

[0144] Fig.11 A schematic diagram of the structure of a scene image processing device provided in an embodiment of the present application is shown in FIG. Fig.11 As shown, the device specifically includes:

[0145] An acquisition module 1101 is used to acquire an image to be processed;

[0146] A recognition model 1102 is used to input the image to be processed into a key point recognition model to obtain a plurality of key points of the image to be processed and coordinate information corresponding to the key points;

[0147] The adjustment module 1103 is used to adjust the image to be processed according to the coordinate information of the key points to obtain a target scene image.

[0148] In a possible implementation, the adjustment module 1103 is further used to obtain the scene coordinate information corresponding to the image to be processed based on the coordinate information of the key points; perform image adjustment on the image to be processed based on the scene coordinate information to obtain an initial scene image; and redraw the initial scene image to obtain a target scene image.

[0149] In a possible implementation, the adjustment module 1103 is also used to obtain the scene conversion parameters of the image to be processed, the scene conversion parameters including close-up conversion parameters, mid-view conversion parameters, or long-view conversion parameters; according to the coordinate information of the key points, select the target key points corresponding to the scene conversion parameters from multiple key points; according to the scene conversion parameters, determine the coordinate conversion rules of the image to be processed; and using the coordinate conversion rules, convert the coordinate information of the target key points into the scene coordinate information corresponding to the image to be processed.

[0150] In a possible implementation, the adjustment module 1103 is further used to, when the scene conversion parameter of the image to be processed is a distant scene conversion parameter, perform a reduction process on the image to be processed according to the scene coordinate information to obtain a first scene image; and perform a blank-fill process on the first scene image to obtain an initial scene image; wherein the initial scene image has the same size as the image to be processed.

[0151] In a possible implementation, the adjustment module 1103 is further used to, when the scene conversion parameters of the image to be processed are close-view conversion parameters or mid-view conversion parameters, enlarge the image to be processed according to the scene coordinate information to obtain a first scene image; and crop the first scene image to obtain an initial scene image; wherein the initial scene image has the same size as the image to be processed.

[0152] In a possible implementation, the adjustment module 1103 is further used to obtain a blank area in the initial scene image when the scene conversion parameter of the image to be processed is a distant view conversion parameter; mark the blank area with a filling label to obtain a scene marked image; and input the scene marked image into an image filling model to obtain a target scene image.

[0153] In a possible implementation, the adjustment module 1103 is further configured to, when the scene conversion parameters of the image to be processed are close-view conversion parameters or mid-view conversion parameters, input the initial scene image into an image restoration model to obtain a target scene image.

[0154] The scene image processing device provided in this embodiment may be as follows Fig.11 The scene image processing device shown in can execute the following steps: Figure 1-10 All the steps of mid-view image processing are realized Figure 1-10 For details, please refer to Figure 1-10 For the sake of brevity, the relevant description is not repeated here.

[0155] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0156] Fig.12 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown in FIG. Fig.12As shown, an embodiment of the present application provides an electronic device, including a processor 1201, a communication interface 1202, a memory 1203 and a communication bus 1204, wherein the processor 1201, the communication interface 1202, and the memory 1203 communicate with each other through the communication bus 1204; the memory 1203 is used to store computer programs; the processor 1201 is used to implement the steps of the scene image processing method provided by any of the aforementioned method embodiments when executing the program stored in the memory 1203: obtaining an image to be processed; inputting the image to be processed into a key point recognition model to obtain multiple key points of the image to be processed and coordinate information corresponding to the key points; adjusting the image to be processed according to the coordinate information of the key points to obtain a target scene image.

[0157] In a possible implementation, based on the coordinate information of the key points, the scene coordinate information corresponding to the image to be processed is obtained; based on the scene coordinate information, the image to be processed is adjusted to obtain an initial scene image; and the initial scene image is redrawn to obtain a target scene image.

[0158] In a possible implementation, the scene conversion parameters of the image to be processed are obtained, the scene conversion parameters including close-up conversion parameters, mid-view conversion parameters, or long-view conversion parameters; based on the coordinate information of the key points, a target key point corresponding to the scene conversion parameter is selected from a plurality of the key points; based on the scene conversion parameters, a coordinate conversion rule of the image to be processed is determined; and using the coordinate conversion rule, the coordinate information of the target key point is converted into scene coordinate information corresponding to the image to be processed.

[0159] In one possible implementation, when the scene conversion parameters of the image to be processed are perspective conversion parameters, the image to be processed is reduced according to the scene coordinate information to obtain a first scene image; the first scene image is blank-filled to obtain an initial scene image; wherein the initial scene image has the same size as the image to be processed.

[0160] In a possible implementation, when the scene conversion parameters of the image to be processed are close-view conversion parameters or mid-view conversion parameters, the image to be processed is enlarged according to the scene coordinate information to obtain a first scene image; the first scene image is cropped to obtain an initial scene image; wherein the initial scene image has the same size as the image to be processed.

[0161] In a possible implementation, when the scene conversion parameters of the image to be processed are perspective conversion parameters, a blank area in the initial scene image is obtained; a filling label is marked on the blank area to obtain a scene marked image; and the scene marked image is input into an image filling model to obtain a target scene image.

[0162] In a possible implementation, when the scene conversion parameters of the image to be processed are near-scene conversion parameters or mid-scene conversion parameters, the initial scene image is input into an image restoration model to obtain a target scene image.

[0163] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0164] It should be understood that the terms used herein are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0165] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A scene image processing method, characterized in that: include: Get the image to be processed; Inputting the image to be processed into a key point recognition model to obtain a plurality of key points of the image to be processed and coordinate information corresponding to the key points; According to the coordinate information of the key points, the image to be processed is adjusted to obtain a target scene image.

2. The method according to claim 1, characterized in that The step of adjusting the image to be processed according to the coordinate information of the key points to obtain a target scene image includes: According to the coordinate information of the key point, obtaining the scene coordinate information corresponding to the image to be processed; According to the scene coordinate information, the image to be processed is adjusted to obtain an initial scene image; The initial scene image is redrawn to obtain a target scene image.

3. The method according to claim 2, characterized in that The step of obtaining scene coordinate information corresponding to the image to be processed according to the coordinate information of the key point includes: Acquiring scene conversion parameters of the image to be processed, wherein the scene conversion parameters include near-view conversion parameters, mid-view conversion parameters, or far-view conversion parameters; Selecting a target key point corresponding to the scene conversion parameter from a plurality of key points according to the coordinate information of the key point; Determining a coordinate transformation rule of the image to be processed according to the scene transformation parameters; The coordinate conversion rule is used to convert the coordinate information of the target key point into the scene coordinate information corresponding to the image to be processed.

4. The method according to claim 2, characterized in that: The step of adjusting the image to be processed according to the scene coordinate information to obtain an initial scene image includes: In the case where the scene conversion parameter of the image to be processed is a distant scene conversion parameter, the image to be processed is reduced according to the scene coordinate information to obtain a reduced scene image; Performing blank filling processing on the reduced scene image to obtain an initial scene image; The initial scene image has the same size as the image to be processed.

5. The method according to claim 2, characterized in that: The step of adjusting the image to be processed according to the scene coordinate information to obtain an initial scene image includes: When the scene conversion parameter of the image to be processed is a near scene conversion parameter or a mid scene conversion parameter, the image to be processed is enlarged according to the scene coordinate information to obtain an enlarged scene image; Performing cropping processing on the enlarged scene image to obtain an initial scene image; The initial scene image has the same size as the image to be processed.

6. The method according to claim 4, characterized in that The redrawing process of the initial scene image to obtain the target scene image includes: When the scene conversion parameter of the image to be processed is a distant scene conversion parameter, obtaining a blank area in the initial scene image; Marking and filling labels on the blank area to obtain a scene-marked image; The scene mark image is input into an image filling model to obtain a target scene image.

7. The method according to claim 5, characterized in that The redrawing process of the initial scene image to obtain the target scene image also includes: When the scene conversion parameters of the image to be processed are near-scene conversion parameters or mid-scene conversion parameters, the initial scene image is input into an image restoration model to obtain a target scene image.

8. A scene image processing device, characterized in that: include: An acquisition module, used for acquiring an image to be processed; A recognition model is used to input the image to be processed into a key point recognition model to obtain a plurality of key points of the image to be processed and coordinate information corresponding to the key points; The adjustment module is used to adjust the image to be processed according to the coordinate information of the key points to obtain a target scene image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the scene image processing method according to any one of claims 1 to 7 are implemented.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the scene image processing method according to any one of claims 1 to 7 are implemented.

Images