Image reconstruction method and system based on ROI and non-ROI, terminal and medium
Through object detection and semantic labeling processing, combined with adversarial generation network, the region of interest and non-interested areas are integrated, and the problem of unclear image transmission details caused by low terminal equipment performance is solved, and the visual effect of efficient transmission and clarity is achieved. .
Patent Information
- Application Number
- CN202411219982.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-02
- Publication Date
- 2024-12-06
AI Technical Summary
When the terminal equipment has low performance, it is easy to cause unclear image details during image transmission, and it is difficult for the prior art to effectively retain feature information in non-interest areas, resulting in poor visual effects.
The region of interest and non-interested areas are obtained through object detection, combined with semantic labeling processing and adversarial generation network, fused the region of interest and non-interested areas to generate reconstructed images to ensure the clarity and integrity of image transmission.
Effectively retain the feature information of the area of interest and non-areas of interest in the image, reduce the demand for network bandwidth of image transmission, improve the clarity and efficiency of image transmission, and adapt to scenes of different bandwidths.
Smart Images

Figure CN119091123A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing technology, and in particular to a method, system, terminal and medium for reconstructing an image based on ROI and non-ROI. Background Art
[0002] With the rapid development of Internet technology, the demand for image transmission is increasing. Due to the performance limitations of terminal devices and insufficient bandwidth of network links, it is easy to cause partial loss of image texture or incomplete global structure during transmission, resulting in poor visual effects for users.
[0003] At present, the optimization method for image transmission mainly extracts the important features corresponding to the region of interest to ensure the integrity of the main texture and global structure of the image. However, the information of the non-interested regions in the image is often lost, resulting in unclear image detail features during the image transmission process when the performance of the terminal device is low. Summary of the invention
[0004] In view of the above-mentioned deficiencies in the prior art, the purpose of the present invention is to provide a method, system, terminal and medium for reconstructing images based on ROI and non-ROI, aiming to solve the problem in the prior art that when the performance of the terminal device is low, the image details will be unclear during the image transmission process.
[0005] In order to achieve the above object, the first aspect of the present invention provides a method for reconstructing an image based on ROI and non-ROI, comprising:
[0006] Get the original image;
[0007] Performing target detection on the original image to obtain a target detection mask;
[0008] Based on the target detection mask, obtaining a region of interest;
[0009] Performing semantic labeling and merging operations on the original image to generate a composite image;
[0010] Obtaining a non-interest region based on the synthesized image and the object detection mask;
[0011] The region of interest and the region of no interest are fused to obtain a reconstructed image.
[0012] In one implementation, performing target detection on the original image to obtain a target detection mask includes:
[0013] Based on the preset image transmission bandwidth, a corresponding number of target categories are set;
[0014] Based on the target category, target detection is performed on the original image using a preset target detection model to obtain a target detection mask.
[0015] In one implementation, obtaining a region of interest based on the target detection mask includes:
[0016] The object detection mask is subjected to a pixel-by-pixel AND operation with the original image to obtain a region of interest.
[0017] In one implementation, the semantic labeling and merging operations are performed on the original images to generate a composite image, including:
[0018] Performing semantic labeling processing on the original image to obtain a plurality of semantic labels;
[0019] Performing rasterization processing on the original image to obtain a plurality of raster regions;
[0020] Assigning a semantic label to each grid area to obtain a semantic label map corresponding to each grid area;
[0021] A generative adversarial network is used to merge all the semantic label maps to generate a synthetic image.
[0022] In one implementation, obtaining a non-interest region based on the synthetic image and the target detection mask includes:
[0023] Performing a matching operation on the target detection mask and the synthetic image to obtain an adjusted target detection mask;
[0024] The adjusted target detection mask is used to perform a pixel-by-pixel AND operation with the synthetic image and then inverted, and a portion corresponding to the adjusted target detection mask is removed to obtain a non-interest region.
[0025] In one embodiment, fusing the region of interest with the region of no interest to obtain a reconstructed image includes:
[0026] Aligning the region of interest and the region of no interest according to pixel positions to obtain an initial superimposed image;
[0027] The initial superimposed images are fused using a preset image fusion algorithm to obtain a reconstructed image.
[0028] A second aspect of the present invention provides a system for reconstructing images based on ROI and non-ROI, the system comprising:
[0029] An information acquisition module, used to obtain original images;
[0030] An object detection module is used to perform object detection on the original image to obtain an object detection mask;
[0031] A region of interest extraction module, used to obtain a region of interest based on the target detection mask;
[0032] A semantic processing module, used for performing semantic labeling and merging operations on the original image to generate a composite image;
[0033] A non-interest region extraction module, used for obtaining a non-interest region based on the synthetic image and the target detection mask;
[0034] The image reconstruction module is used to fuse the region of interest and the region of no interest to obtain a reconstructed image.
[0035] In one embodiment, the target detection module includes a target category division unit and a mask generation unit, wherein the target category division unit is used to divide a corresponding number of target categories based on a preset image transmission bandwidth; the mask generation unit is used to perform target detection on the original image based on the target category using a preset target detection model to obtain a target detection mask.
[0036] A third aspect of the present invention provides an intelligent terminal, which includes a memory, a processor, and a ROI and non-ROI based image reconstruction program stored in the memory and executable on the processor, wherein the ROI and non-ROI based image reconstruction program, when executed by the processor, implements any one of the steps of the above-mentioned ROI and non-ROI based image reconstruction method.
[0037] A fourth aspect of the present invention provides a computer-readable storage medium, on which is stored a program for reconstructing images based on ROI and non-ROI. When the program for reconstructing images based on ROI and non-ROI is executed by a processor, any step of the above-mentioned method for reconstructing images based on ROI and non-ROI is implemented.
[0038] Compared with the prior art, the beneficial effects of this solution are as follows:
[0039] The present invention retains the fine feature information in the region of interest in the original image through target detection, and retains the feature information of the non-interest region by semantically labeling the original image, which can not only ensure the integrity and comprehensiveness of the information in the region of interest, but also retain important texture information and global structure on the basis of retaining the feature information of the non-interest region, and effectively reduce the demand for network bandwidth for image transmission, and finally achieve a visual effect that takes into account both efficient transmission and clarity. At the same time, the accuracy of target detection can be flexibly adjusted according to the size of the transmission bandwidth, which effectively improves the generalization ability of the model and can ensure the clarity of image transmission when the performance of the terminal device is low. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0041] Figure 1 A brief flow chart of the image reconstruction method based on ROI and non-ROI of the present invention;
[0042] Figure 2 A detailed flow chart of the image reconstruction method based on ROI and non-ROI of the present invention;
[0043] Figure 3 It is a schematic diagram of the image reconstruction system module based on ROI and non-ROI of the present invention;
[0044] Figure 4 It is a schematic diagram of the structure of the intelligent terminal of the present invention. DETAILED DESCRIPTION
[0045] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.
[0046] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.
[0047] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.
[0048] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.
[0049] As used in this specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.
[0050] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0051] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.
[0052] In view of the problem that the image details are not clear during the image transmission process when the performance of the terminal device is low, the present invention proposes a method for reconstructing images based on ROI and non-ROI. The method mainly comprises the following steps: in the process of transmitting images or videos, the transmission bandwidth is clarified by evaluating the current bandwidth and the device performance of the user, and target detection with different precisions is performed on the original image according to the size of the bandwidth to generate a corresponding number of target detection masks; at the same time, semantic segmentation is performed on the original image to obtain a number of semantic labels, and a generative adversarial network is used to merge all the semantic label maps to generate a synthetic image; the target detection mask is subjected to a pixel-by-pixel AND operation with the original image, and the part corresponding to the target detection mask in the original image is retained to obtain a region of interest; the target detection mask is subjected to a pixel-by-pixel AND operation with the synthetic image and the inversion is performed to eliminate the part corresponding to the target detection mask in the synthetic image to obtain a region of non-interest; the region of interest and the non-region of non-interest are fused to obtain a reconstructed image. The present invention retains the fine feature information in the region of interest in the original image through target detection, and retains the feature information of the non-interest region by semantically labeling the original image. This can not only ensure the integrity and comprehensiveness of the information in the region of interest, but also retain important texture information and global structure on the basis of retaining the feature information of the non-interest region, and effectively reduce the demand for network bandwidth for image transmission, ultimately achieving a visual effect that takes into account both efficient transmission and clarity.
[0053] The embodiment of the present invention provides a method for reconstructing an image based on ROI and non-ROI, which is deployed on electronic devices such as computers and servers and is applied to the transmission of images or videos in various bandwidth scenarios. Specifically, Figure 1 and Figure 2 As shown, the steps of the method in this embodiment include:
[0054] Step S100: obtaining an original image;
[0055] Specifically, an image to be transmitted in a target scene is obtained as an original image, wherein the target scene includes short video playback, autonomous driving, game development, virtual reality, television broadcasting, video conferencing, distance education, telemedicine and other scenes, and the transmission bandwidths of different scenes may be different or the same, and the image to be transmitted refers to one or more images, or is a plurality of images obtained by dividing the video to be transmitted according to video frames, such as by importing the video file into the OpenCV library, reading the video frame by frame and saving each frame as an image file. The present invention does not impose specific restrictions on the format of the image to be processed and the type of application scenario.
[0056] Step S200: performing target detection on the original image to obtain a target detection mask;
[0057] Specifically, based on a preset image transmission bandwidth, a corresponding number of target categories are set; based on the target categories, target detection is performed on the original image using a preset target detection model to obtain a target detection mask.
[0058] According to the image transmission bandwidth of the transmission network link, the instance registration is dynamically adjusted for the original image to be transmitted in the target scene. Specific objects (such as people, vehicles, trees, etc.) or areas (such as sidewalks, parking lots, etc.) in the original image are identified and marked through instance registration, so as to obtain target categories that match the current transmission bandwidth. Different target categories require different amounts of data to represent (for example, complex targets may require more pixels or feature information to represent). The wider the bandwidth, the more target categories are generated, the more regions the original image is divided into, or the more types of objects are marked. The number of target categories that need to be set is determined by evaluating the transmission bandwidth and the amount of data required for each target category. If the bandwidth is limited, choose to reduce the number of target categories or select important categories with lower data requirements.
[0059] In a preferred embodiment, the original image is input into a preset target detection model (such as Faster R-CNN) to perform target detection on the original image, and the category and position (such as a bounding box) of each detected target are output. For each detected target, a corresponding mask is generated on the original image according to the bounding box information of the target. In this embodiment, each mask represents a binary image of the same size as the original image, in which the target area is marked as 1 (or white) and the remaining area is marked as 0 (or black). The preset target detection model refers to a model built and trained based on the target detection model, and the trained model can perform target detection on the image according to the set target category. Different numbers of target categories will correspondingly obtain three different levels of target detection masks: high, medium and low. The high-level target detection mask contains detailed details, such as the precise contour of the edge, internal texture, etc., which is suitable for application scenarios requiring high-precision analysis; the medium-level target detection mask reduces some details, but can maintain the basic shape and position relationship of the object, which is suitable for application scenarios under general display and analysis requirements; the low-level target detection mask only contains the general outline and position information of the object, which is suitable for application scenarios with low bandwidth or low resolution display. That is, the accuracy of target detection can be flexibly adjusted according to the size of the transmission bandwidth, effectively improving the generalization ability of the model, and ensuring the clarity of image transmission when the performance of the terminal device is low.
[0060] Furthermore, before the original image is input into the preset target detection model, the original image may be preprocessed in advance such as scaling and cropping to obtain a preprocessed image to meet the input requirements of the target detection model. If the preprocessed image is preprocessed, the image input into the preset target detection model is the preprocessed image, and subsequent operations are all based on the preprocessed image.
[0061] This embodiment sets a corresponding number of target categories according to the transmission bandwidth of the target scene, and then uses the target detection model trained based on the target category to perform target detection on the image, thereby generating target detection masks with different accuracy requirements. It can retain image detail features and important features as much as possible when the performance of the terminal device is low, so as to optimize resource utilization and meet different bandwidth requirements, thereby effectively ensuring the image display effect of the receiving device.
[0062] Step S300: obtaining a region of interest based on the target detection mask;
[0063] Specifically, the target detection mask and the original image are subjected to a pixel-by-pixel AND operation, that is, the target detection mask and the original image are compared pixel by pixel, and the original image pixels at positions corresponding to the median values of 1 in the mask are retained, thereby obtaining a region of interest (ROI).
[0064] Step S400: performing semantic labeling and merging operations on the original image to generate a composite image;
[0065] Specifically, the original image is processed by semantic labeling to obtain a number of semantic labels, such as using a deep learning model (such as U-Net, Mask R-CNN, etc.) to perform pixel-level segmentation on the original image or the preprocessed image to identify different objects or regions in the image, and assign a unique identifier to each object or region; the original image is processed by rasterization to divide the original image into a number of raster regions; and a semantic label is assigned to each of the raster regions according to the image segmentation result to obtain a semantic label map corresponding to each of the raster regions. Furthermore, the semantic label map can also be optimized by denoising, filling holes, adjusting boundaries, etc. to ensure the accuracy and completeness of the label.
[0066] Then, a generative adversarial network based on semantic labels is used to merge all the semantic label maps to generate a synthetic image, that is, semantic labels are combined with basic information of the image, and language label features are learned under the training of the generative adversarial network to realize the synthesis of high-resolution images with semantic labels, and multi-level generators and multi-scale discriminators are used to improve the speed and resolution of image synthesis. Specifically, a generative adversarial network model is constructed, and the generative adversarial network is composed of a generator network and a discriminator network, wherein the generator network takes the semantic label map obtained after semantic labeling of the image as input and outputs a synthetic image corresponding thereto. The structure of the generator network usually includes an upsampling layer (such as a transposed convolution) and a convolution layer, which is used to gradually generate a high-resolution image from a low-resolution semantic label map. The discriminator network is used to distinguish between the generated synthetic image and the real image. It receives the image as input and outputs a probability value to distinguish whether the image is real or generated by the generator network. After the generative adversarial network model is constructed, the generative adversarial network is trained using paired training data, each pair including a merged semantic label map and a corresponding real image. The loss function uses adversarial loss and / or content loss, where the adversarial loss is used to encourage the generator to generate realistic images and encourage the discriminator to better distinguish between real and generated images. The content loss is used to ensure that the generated image is consistent with the semantic label map in content, such as L1 loss or L2 loss. The generator and discriminator are alternately trained using the loss function to obtain a trained adversarial generative network. Finally, the original image is semantically labeled, and the resulting semantic label map is input into the generator in the trained adversarial generative network. The generator merges the semantic label map to generate the corresponding synthetic image. The merging operation refers to pixel-level superposition of the semantic label map. Furthermore, machine learning methods such as conditional random fields can be used to optimize the merged label map to ensure smooth transition and logical consistency between labels.
[0067] In a preferred embodiment, the adversarial generative network model includes two main parts: a multi-level generator and a multi-scale discriminator, wherein the generator is used to generate synthetic images using semantic labels, and different accuracy levels can achieve step-by-step resolution improvement, while the discriminator is used to distinguish between real images and synthetic images according to different accuracy requirements in different scenarios. The generator adopts U-Net or a similar encoder-decoder architecture, in which the encoder gradually reduces the resolution of the image to extract features, and the decoder gradually restores the resolution of the image to generate an output image. In U-Net, jump connections are also included to help the decoder recover finer details. The generator is used to receive a semantic label map as input and output a corresponding synthetic image, with the goal of generating an output as close to the real image as possible while maintaining consistency with the input label map; the discriminator adopts a convolutional neural network (CNN), which receives a pair of images (real image and generated image or label map) as input and outputs a scalar value indicating the probability that the input pair is a real image pair. The discriminator is used to receive a real image and a synthetic image generated by the generator as input, and output the recognition results of the two input images. The goal is to accurately identify the real image and the synthetic image generated by the generator.
[0068] Step S500: obtaining a non-interest region based on the synthetic image and the target detection mask;
[0069] Specifically, the target detection mask and the synthetic image are matched to obtain an adjusted target detection mask; the adjusted target detection mask and the synthetic image are ANDed pixel by pixel and inverted to remove a portion corresponding to the adjusted target detection mask to obtain a non-interest area.
[0070] Since the target detection mask is generated based on the original image, if the synthetic image is different from the original image in size or content, the target detection mask and the synthetic image may not match in size or content. Therefore, the shape or position of the mask is adjusted to match the area in the semantic label map to adapt the target detection mask to the synthetic image so that the two are consistent in size, texture, resolution, color space, etc., thereby obtaining the adjusted target detection mask. Then, the adjusted target detection mask is used to perform pixel-by-pixel AND operations with the synthetic image and inverted to remove the corresponding target detection mask part in the synthetic image, thereby obtaining the non-ROI.
[0071] Step S600: Fusing the region of interest and the region of no interest to obtain a reconstructed image.
[0072] Specifically, since the non-interest region is obtained by performing a pixel-by-pixel AND operation with the target detection mask and the synthetic image and then negating them, that is, comparing the target detection mask and the synthetic image pixel by pixel, and the synthetic image is obtained by semantically labeling the original image, and the interest region is obtained by performing a pixel-by-pixel AND operation with the target detection mask and the original image, that is, comparing the target detection mask and the original image pixel by pixel, therefore, the region of interest and the non-interest region may have unclear boundaries or differences in color, brightness, etc., and if the region of interest and the non-interest region are directly superimposed, obvious seams may result.
[0073] Based on this, in order to improve the fusion effect of the region of interest and the non-region of interest, if the boundary between the region of interest and the non-region of interest is not obvious, the gradient effect is achieved by applying a gradient mask at the boundary of the ROI and the Non-ROI. Specifically, a gradient mask of the same size as the image is created, and the ROI and the non-ROI are weighted superimposed using the mask. The transition effect is optimized by adjusting the gradient parameters (such as gradient width, shape, etc.) so that the value of the mask gradually changes from 1 inside the ROI to 0 of the Non-ROI, thereby making the transition between the two regions smoother. If there is a relatively obvious difference in color or brightness between the ROI and the Non-ROI, an image enhancement strategy such as color balance, brightness / contrast adjustment is used to adjust one or both regions so that the two are more visually consistent. Then the processed region of interest and the non-region of interest are aligned according to the pixel position, such as by using the OpenCV image processing library, using array operations to copy the pixel value of the ROI to the corresponding position of the Non-ROI to obtain the initial superimposed image. Finally, the initial superimposed images are fused using a preset image fusion algorithm to obtain a reconstructed image. For example, a deep learning model is used to learn and predict the optimal fusion method between ROI and non-ROI. The model automatically extracts image features and learns fusion rules to generate a visually coherent reconstructed image that is rich in image details and texture information.
[0074] like Figure 3 As shown, corresponding to the above-mentioned ROI-based and non-ROI-based image reconstruction method, an embodiment of the present invention further provides a ROI-based and non-ROI-based image reconstruction system, and the above-mentioned ROI-based and non-ROI-based image reconstruction system includes:
[0075] An information acquisition module 310 is used to acquire an original image;
[0076] The target detection module 320 is used to perform target detection on the original image to obtain a target detection mask;
[0077] A region of interest extraction module 330, configured to obtain a region of interest based on the target detection mask;
[0078] A semantic processing module 340 is used to perform semantic labeling and merging operations on the original image to generate a composite image;
[0079] A non-interest region extraction module 350, configured to obtain a non-interest region based on the composite image and the target detection mask;
[0080] The image reconstruction module 360 is used to fuse the region of interest and the region of no interest to obtain a reconstructed image.
[0081] Specifically, in this embodiment, the specific functions of the above-mentioned system for reconstructing images based on ROI and non-ROI can also refer to the corresponding description in the above-mentioned method for reconstructing images based on ROI and non-ROI, which will not be repeated here.
[0082] In a preferred embodiment, the target detection module includes a target category division unit and a mask generation unit, wherein the target category division unit is used to divide a corresponding number of target categories based on a preset image transmission bandwidth; and the mask generation unit is used to perform target detection on the original image based on the target category using a preset target detection model to obtain a target detection mask.
[0083] Specifically, in this embodiment, the specific functions of the above-mentioned system for reconstructing images based on ROI and non-ROI can also refer to the corresponding description in the above-mentioned method for reconstructing images based on ROI and non-ROI, which will not be repeated here.
[0084] Based on the above embodiments, the present invention further provides an intelligent terminal, whose principle block diagram can be shown as follows: Figure 4 As shown. The above-mentioned intelligent terminal includes a processor, a memory, a network interface and a display screen connected through a system bus. Among them, the processor of the intelligent terminal is used to provide computing and control capabilities. The memory of the intelligent terminal includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a program for reconstructing images based on ROI and non-ROI. The internal memory provides an environment for the operation of the operating system and the program for reconstructing images based on ROI and non-ROI in the non-volatile storage medium. The network interface of the intelligent terminal is used to communicate with an external terminal through a network connection. When the program for reconstructing images based on ROI and non-ROI is executed by the processor, the steps of any one of the above-mentioned methods for reconstructing images based on ROI and non-ROI are implemented. The display screen of the intelligent terminal can be a liquid crystal display screen or an electronic ink display screen.
[0085] Those skilled in the art will understand that Figure 4The principle block diagram shown in the figure is only a block diagram of a partial structure related to the scheme of the present invention, and does not constitute a limitation on the smart terminal to which the scheme of the present invention is applied. The specific smart terminal may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0086] In one embodiment, a smart terminal is provided, comprising a memory, a processor, and a ROI- and non-ROI-based image reconstruction program stored in the memory and executable on the processor. When the ROI- and non-ROI-based image reconstruction program is executed by the processor, steps of any one of the ROI- and non-ROI-based image reconstruction methods provided in an embodiment of the present invention are implemented.
[0087] An embodiment of the present invention further provides a computer-readable storage medium, on which a program for reconstructing images based on ROI and non-ROI is stored. When the program for reconstructing images based on ROI and non-ROI is executed by a processor, the steps of any one of the methods for reconstructing images based on ROI and non-ROI provided in the embodiments of the present invention are implemented.
[0088] It should be understood that the serial numbers of the steps in the above embodiments do not imply a sequence of execution, and the execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0089] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the above-mentioned device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.
[0090] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0091] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0092] In the embodiments provided by the present invention, it should be understood that the disclosed apparatus / terminal equipment and method can be implemented in other ways. For example, the apparatus / terminal equipment embodiments described above are only illustrative, for example, the division of the above modules or units is only a logical function division, and in actual implementation, other division methods can be used, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.
[0093] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features therein may be replaced by equivalents. However, these modifications or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.
Claims
1. A method for reconstructing images based on ROI and non-ROI. It is characterized in that The following steps are involved: Get the original image; Performing target detection on the original image to obtain a target detection mask; Based on the target detection mask, obtaining a region of interest; Performing semantic labeling and merging operations on the original image to generate a composite image; Obtaining a non-interest region based on the synthesized image and the object detection mask; The region of interest and the region of no interest are fused to obtain a reconstructed image.
2. The method for reconstructing images based on ROI and non-ROI according to claim 1, It is characterized in that The performing target detection on the original image to obtain a target detection mask includes: Based on the preset image transmission bandwidth, a corresponding number of target categories are set; Based on the target category, target detection is performed on the original image using a preset target detection model to obtain a target detection mask.
3. The method for reconstructing images based on ROI and non-ROI according to claim 1, It is characterized in that The obtaining of a region of interest based on the target detection mask comprises: The object detection mask is subjected to a pixel-by-pixel AND operation with the original image to obtain a region of interest.
4. The method for reconstructing images based on ROI and non-ROI according to claim 1, It is characterized in that The performing semantic labeling and merging operations on the original images to generate a synthetic image includes: Performing semantic labeling processing on the original image to obtain a plurality of semantic labels; Performing rasterization processing on the original image to obtain a plurality of raster regions; Assigning a semantic label to each grid area to obtain a semantic label map corresponding to each grid area; A generative adversarial network is used to merge all the semantic label maps to generate a synthetic image.
5. The method for reconstructing images based on ROI and non-ROI according to claim 1, It is characterized in that The obtaining a non-interest region based on the synthetic image and the target detection mask comprises: Performing a matching operation on the target detection mask and the synthetic image to obtain an adjusted target detection mask; The adjusted target detection mask is used to perform a pixel-by-pixel AND operation with the synthetic image and then inverted, and a portion corresponding to the adjusted target detection mask is removed to obtain a non-interest region.
6. The method for reconstructing images based on ROI and non-ROI according to claim 1, It is characterized in that The step of fusing the region of interest with the region of no interest to obtain a reconstructed image includes: Aligning the region of interest and the region of no interest according to pixel positions to obtain an initial superimposed image; The initial superimposed images are fused using a preset image fusion algorithm to obtain a reconstructed image.
7. A system for reconstructing images based on ROI and non-ROI. It is characterized in that The system comprises: An information acquisition module, used to obtain original images; An object detection module is used to perform object detection on the original image to obtain an object detection mask; A region of interest extraction module, used to obtain a region of interest based on the target detection mask; A semantic processing module, used for performing semantic labeling and merging operations on the original image to generate a composite image; A non-interest region extraction module, used for obtaining a non-interest region based on the synthetic image and the target detection mask; The image reconstruction module is used to fuse the region of interest and the region of no interest to obtain a reconstructed image.
8. The ROI and non-ROI based image reconstruction system according to claim 7, It is characterized in that The target detection module includes a target category classification unit and a mask generation unit, wherein: The target category division unit is used to divide the target categories into a corresponding number based on a preset image transmission bandwidth; The mask generating unit is used to perform target detection on the original image based on the target category by using a preset target detection model to obtain a target detection mask.
9. A smart terminal, It is characterized in that The intelligent terminal includes a memory, a processor, and a ROI- and non-ROI-based image reconstruction program stored in the memory and executable on the processor. When the ROI- and non-ROI-based image reconstruction program is executed by the processor, the steps of the ROI- and non-ROI-based image reconstruction method according to any one of claims 1 to 6 are implemented.
10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a program for reconstructing images based on ROI and non-ROI. When the program for reconstructing images based on ROI and non-ROI is executed by a processor, the steps of the method for reconstructing images based on ROI and non-ROI are implemented as claimed in any one of claims 1 to 6.
