Method, apparatus, device, and storage medium for image processing

Abstract

According to an embodiment of the present disclosure, there are provided a method, an apparatus, a device, and a storage medium for image processing. The method for image processing includes segmenting a panoramic image into multiple sub-images and presenting the multiple sub-images. The method further includes receiving user markings for each of the multiple sub-images, and obtaining a marked panoramic image corresponding to the panoramic image based on the user markings for each of the multiple sub-images. In this way, the accuracy and efficiency of image marking can be significantly improved.

Description

Technical Field

[0001] Example embodiments of the present disclosure generally relate to the field of computers, and particularly to methods, apparatuses, devices, and computer-readable storage media for image processing. Background Art

[0002] Panoramic images can provide a wide-angle view of a specific scene, and some panoramic images can present visual information of 360° horizontally and 180° vertically in a specific scene. This novel display method is being applied in various industries. Panoramic images can be directly captured by a panoramic camera or obtained by stitching multiple images. Regardless of the source of the panoramic image, when presented tiled on a plane, some objects in the image may be distorted and different from the picture seen by the normal human eye. Summary of the Invention

[0003] According to an example embodiment of the present disclosure, a solution for image processing is provided.

[0004] In a first aspect of the present disclosure, a method for image processing is provided. The method includes: dividing a panoramic image into multiple sub-images; presenting the multiple sub-images; receiving user marks for each of the multiple sub-images; and determining a marked panoramic image corresponding to the panoramic image based on the user marks for each of the multiple sub-images.

[0005] In a second aspect of the present disclosure, an apparatus for image processing is provided. The apparatus includes: a dividing module configured to divide a panoramic image into multiple sub-images; a presenting module configured to present the multiple sub-images; a receiving module configured to receive user marks for each of the multiple sub-images; and a determining module configured to determine a marked panoramic image corresponding to the panoramic image based on the user marks for each of the multiple sub-images.

[0006] In a third aspect of the present disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. The instructions, when executed by the at least one processing unit, cause the device to execute the method of the first aspect.

[0007] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided. A computer program is stored on the computer-readable storage medium and can be executed by a processor to implement the method of the first aspect.

[0008] It should be understood that the content described in the Summary of the Invention section is not intended to limit the key features or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. Brief Description of the Drawings

[0009] In combination with the accompanying drawings and with reference to the following detailed description, the above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent. In the drawings, the same or similar reference numerals denote the same or similar elements, where:

[0010] Figure 1 FIG. shows a schematic diagram of an exemplary environment in which embodiments of the present disclosure can be implemented;

[0011] Figure 2 FIG. shows a flowchart of a process for image processing according to some embodiments of the present disclosure;

[0012] Figure 3 FIG. shows a schematic diagram of an image processing flow according to some embodiments of the present disclosure;

[0013] Figures 4A to 4D FIG. shows a schematic diagram of an exemplary user interface for user marking according to some embodiments of the present disclosure;

[0014] Figure 5 FIG. shows a block diagram of an apparatus for image processing according to some embodiments of the present disclosure; and

[0015] Figure 6 FIG. shows a block diagram of a computing device in which one or more embodiments of the present disclosure can be implemented. Detailed Description of the Embodiments

[0016] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0017] In the description of the embodiments of the present disclosure, the term "including" and its similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". There may also be other explicit and implicit definitions below.

[0018] It can be understood that the data involved in the technical solution of the present disclosure (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of the corresponding laws, regulations and related provisions.

[0019] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, usage scopes, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained through appropriate means in accordance with relevant laws and regulations.

[0020] For example, when responding to an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested by the user will require obtaining and using the user's personal information, so that the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, an application program, a server, or a storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0021] As an optional but non-limiting implementation manner, the manner of sending a prompt message to the user in response to receiving an active request from the user may be, for example, in the form of a pop-up window, and the prompt message may be presented in text in the pop-up window. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0022] It is understandable that the above process of notifying and obtaining the user's authorization is only illustrative and does not limit the implementation manner of the present disclosure. Other manners that meet relevant laws and regulations can also be applied to the implementation manner of the present disclosure.

[0023] Figure 1 FIG. shows a schematic diagram of an example environment 100 in which the embodiments of the present disclosure can be implemented. In this example environment 100, the terminal device 110 can provide a marking function for the panoramic image 120 to output the marked panoramic image 122.

[0024] In some examples, the terminal device 110 may be installed with an image marking application 112, and this application can provide an interaction with the user 102 to implement the marking function for an image (including a panoramic image). In other examples, it may not be necessary to install the image marking application 112, and the terminal device 110 can provide an interaction with the user 102 in other ways, such as by accessing a web page, to implement the marking function for the panoramic image.

[0025] The panoramic image 120 is a two-dimensional image that presents visual information in a wide-angle manner and can have any format. In some embodiments, the panoramic image 120 can present visual information of 360° horizontally and 180° vertically in a specific scene. In other embodiments, the panoramic image 120 can also present visual information at other angles. In some embodiments, the panoramic image 120 can have an aspect ratio of 2:1, or can have any other appropriate size. The panoramic image 120 can be captured by a panoramic image acquisition device, or can be obtained by stitching multiple images captured in a specific scene. The embodiments of the present disclosure are not limited in this regard.

[0026] In many application scenarios, marking operations on the panoramic image 120 may be required, such as image publishing, image review, image-based scene review, etc., for various purposes such as avoiding privacy leakage, image aesthetics, and compliance. The marking operation on the panoramic image 120 can be determined based on the operation of the user 102.

[0027] The marking operation on the panoramic image 120 can include the review and marking of the entire panoramic image 120 or specific regions. For example, in some scenarios, before publishing the panoramic image 120, it may be necessary to perform privacy processing on some regions in the panoramic image 120, such as obscuring personal items, unwanted information, etc. (such as by mosaicing). In other scenarios, it may also be desirable to add content, such as text, specific graphics, or stickers, to specific regions of the panoramic image 120; or it may also be desirable to check whether each visual detail presented in the panoramic image 120 complies with relevant regulations.

[0028] In some embodiments, the terminal device 110 can communicate with a remote computing device 132 to implement the marking process of the panoramic image 120. For example, the computing device 132 can provide storage functions, specific processing tasks, etc. for the panoramic image 120 to expand the storage and processing capabilities of the terminal device 110. The computing device 132 can be various types of computing systems / servers capable of providing computing capabilities, including but not limited to mainframes, edge computing nodes, computing devices in cloud environments, etc. In Figure 1 the example shown, the computing device 132 can be located in the cloud environment 130.

[0029] The terminal device 110 can be any type of mobile terminal, fixed terminal or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), audio / video players, digital cameras / camcorders, positioning devices, television receivers, radio broadcast receivers, e-book devices, gaming devices, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. In some embodiments, the terminal device 110 can also support any type of user interface (such as a "wearable" circuit, etc.).

[0030] It should be understood that the structure and functions of the environment 100 are described only for exemplary purposes and do not imply any limitation on the scope of the present disclosure. For example, the terminal device 110 may not communicate with the remote computing device 132, so the computing device 132 may be omitted.

[0031] In a conventional scheme for marking panoramic images, the panoramic image is directly presented to the user for viewing, and editing-type markings are made on the panoramic image as needed, or the image is carefully checked for any illegal content, etc.

[0032] Due to the wide-angle characteristics of panoramic images, when a panoramic image is tiled and presented on a two-dimensional plane, some objects in the image may be distorted and distorted. When marking a panoramic image, users often need to carefully view and analyze visual details. The distortion and distortion in the image will cause great interference, reduce the marking efficiency, and easily lead to problems such as marking errors and missed markings.

[0033] According to an embodiment of the present disclosure, an improved scheme for marking panoramic images is proposed. In this scheme, the panoramic image is converted into multiple sub-images that are visually convenient for viewing for presentation and marking. Through panoramic image segmentation, the visual information presented by each sub-image is more in line with the picture observed by the human eye at a specific perspective. Therefore, the user markings performed on the basis of the sub-images can ensure higher accuracy, are more user-friendly visually, and avoid marking errors or spending too much time viewing in the distorted area, which can significantly improve the accuracy and efficiency of image marking.

[0034] Some exemplary embodiments of the present disclosure will be described below with continued reference to the accompanying drawings.

[0035] Figure 2 A flowchart of a process 200 for image processing according to some embodiments of the present disclosure is shown. The process 200 can be implemented at the terminal device 110. For ease of discussion, the process 200 will be described with reference to Figure 1 the environment 100.

[0036] At block 210, the terminal device 110 divides the panoramic image 120 into multiple sub-images.

[0037] The panoramic image 120 can be provided from various image sources to the terminal device 110 for processing. According to an embodiment of the present disclosure, instead of directly presenting the panoramic image 120 to the user for marking, the panoramic image 120 is subjected to image segmentation to obtain multiple sub-images. Compared with the panoramic image 120, each sub-image presents visual information at a smaller visual angle. After panoramic image segmentation, the distortion and aberration in the panoramic image 120 will be processed so that each sub-image can present distortion-free and aberration-free visual information at a specific visual angle.

[0038] To better understand some embodiments of the present disclosure, Figure 3 A schematic diagram of an image processing flow 300 according to some embodiments of the present disclosure is shown. As Figure 3 shown, the panoramic image 120 is divided into multiple (N) sub-images 310-1, 310-2,..., 310-N, where N is an integer greater than 1. For ease of discussion, hereinafter, the sub-images 310-1, 310-2,..., 310-N will be collectively or individually referred to as sub-image 310.

[0039] Various segmentation algorithms for panoramic images can be used to perform the segmentation of the panoramic image 120. In some embodiments, multiple sub-images 310 that conform to human visual perception can be segmented from the panoramic image 120.

[0040] Specifically, multiple views associated with the scene presented by the panoramic image 120 can be determined. The views involved in the panoramic image 120 can be related to the angles presented by the panoramic image 120. Depending on the specific scene captured by the panoramic image 120, the specific number of associated views may vary. In one example, assume that the panoramic image 120 is a hexahedron panoramic image or a spherical panoramic image and presents visual information of 360° horizontally and 180° vertically in the presented scene. Such a panoramic image can include, for example, a panoramic image of a house for presenting the entire view of the house space. In such an example, the panoramic image 120 can be associated with the top view, bottom view, left view, right view, front view, and rear view of the scene. Accordingly, the panoramic image 120 can be segmented according to the hexahedron slicing method to obtain multiple sub-images 310. In another example, if the panoramic image 120 presents visual information of 270° horizontally and 180° vertically in the scene, then it may be associated with the top view, bottom view, left view, right view, and front view of the scene.

[0041] After determining multiple views associated with the panoramic image 120, the terminal device 110 may divide the panoramic image 120 into multiple sub-images 310 corresponding to the multiple views. The number of sub-images 310 may be the same as the number of views. The visual information presented under each view is visual information that conforms to the normal viewing angle of the human eye. For example, for a hexahedron panoramic image or a spherical panoramic image, it may be divided into six sub-images corresponding to the elevation view, top view, left view, right view, front view, and rear view of the scene respectively.

[0042] When dividing multiple sub-images corresponding to multiple views, an image area corresponding to the view may be divided out from the panoramic image 120, and subsequent image processing, such as deformation, distortion removal, size scaling, etc., may be performed on the image area based on the view. Various currently available or future-developed panoramic image segmentation algorithms can be applied to the segmentation of the panoramic image 120.

[0043] In some embodiments, the multiple sub-images 310 may have any appropriate size, for example, a size suitable for being presented to the user for marking. Embodiments of the present disclosure are not limited in this regard.

[0044] In block 220, the terminal device 110 presents multiple sub-images 310. In some embodiments, the multiple sub-images 310 may be presented by an output device of the terminal device 110, such as a display screen. The multiple sub-images 310 may be presented on the user interface for the user 102 to view. The user 102 may mark each sub-image 310 of the panoramic image 120.

[0045] Example scenarios for performing panoramic image marking may include image publishing, image review, image-based scene review, etc. As an example, when publishing a panoramic image on a public platform, it may be necessary to edit certain areas in the image. An example of an editing operation may include privacy processing of areas in the image that contain privacy information, such areas including, for example, faces, personal items, cluttered areas, computer screens, or documents. Other examples of editing operations include overlaying visual markers, such as text, stickers, various graphics, etc., on certain areas of the image.

[0046] In some other scenarios, it may be necessary to mark whether the panoramic image conforms to a predetermined rule or whether the scene presented by the panoramic image conforms to a predetermined rule, so as to decide whether to allow the panoramic image to be published or the next operation for the panoramic image or the scene. The predetermined rule may include whether the panoramic image contains sensitive information, whether the panoramic image conforms to a certain image classification rule (for example, in an image classification scenario), whether the panoramic image conforms to image publishing regulations, or any other predetermined rule. These rules can be set according to specific scenario requirements and are not limited herein.

[0047] As a specific example, before publishing a panoramic image of a house for sale or rent, it may be necessary to mark whether the panoramic image complies with the publishing rules, anonymize the privacy information presented therein, and / or overlay certain marks on the panoramic image, etc.

[0048] In various scenarios, marking a panoramic image involves checking details in the panoramic image. In an embodiment of the present disclosure, through the segmentation of the panoramic image 120, multiple sub-images 310 that are more in line with human visual perception can be presented to the user for marking.

[0049] In some embodiments, multiple sub-images 310 obtained by segmenting the panoramic image 120 can be presented in whole or in part on the user interface for the user 102 to mark. In some embodiments, one or more sub-images 310 can be presented in response to a user trigger. For example, the user 102 can initiate a trigger to present the subsequent one or more sub-images 310 after completing the marking of one sub-image 310.

[0050] In some embodiments, multiple panoramic images may need to be marked in a marking scenario, and these panoramic images can be respectively segmented into multiple sub-images. The terminal device 110 can record the association relationship between each panoramic image and its segmented multiple sub-images. When presenting sub-images to the user, the sub-images corresponding to a single panoramic image can be presented in sequence, or the entire set of sub-images of multiple panoramic images can be presented in any order. Thus, at the sub-image level of presentation, the user can focus on the presented sub-images without having to focus on the entire panoramic image.

[0051] In some embodiments, to facilitate the user to understand more information about the sub-images, the terminal device 110 can present multiple sub-images 310 in association with indications of multiple views. In this way, when marking each sub-image 310, the user 102 can know which specific perspective in the overall presentation angle of the panoramic image 120 the sub-image is.

[0052] For ease of understanding, the following will be further discussed with reference to user interface diagrams.

[0053] Figure 4A A schematic diagram of an example user interface 400 for user marking according to some embodiments of the present disclosure is shown. Figure 4A The user interface 400 can be the user interface of the image marking application 112 on the terminal device 110, or other user interfaces that can be presented on the terminal device 110. The user interface 400 is used to present the sub-images 310 and can provide corresponding function options for the user to mark the sub-images 310.

[0054] It should be understood that Figure 4AThe user interfaces shown in the following figures and subsequent figures are merely examples, and various interface designs may actually exist. Each graphical element in the user interface may have different arrangements and different visual representations, one or more of which may be omitted or replaced, and there may also be one or more other elements. Embodiments of the present disclosure are not limited in this regard.

[0055] In Figure 4A the example of, it is assumed that the panoramic image 120 is divided into six sub-images 310 corresponding to the front view, rear view, bottom view, top view, left view, and right view respectively. The user interface 400 presents thumbnails 420-1, 420-2, 420-3, 420-4, 420-5, and 420-6 of the six sub-images 310 (collectively or individually referred to as thumbnails 420 for ease of discussion). Note that the details of the thumbnails and sub-images in the figure are merely examples, and the visual information of some thumbnails is omitted.

[0056] On each thumbnail 420 of the sub-image 310, an indication 402 regarding the corresponding view is also presented, such as the text indications "front view", "rear view", "bottom view", "top view", "left view", and "right view". This can clearly indicate the view corresponding to each sub-image.

[0057] By detecting a user selection of each thumbnail 420, the corresponding sub-image 310 is presented on the user interface 400. As shown in the figure, in the case where the thumbnail 420-1 is selected, the corresponding sub-image 310-1 is presented on the user interface 400. On the presented sub-image 310, the user 102 can perform corresponding markings according to the marking needs.

[0058] Continuing to return to Figure 2 process 200, after presenting the sub-image 310, at block 230, the terminal device 110 receives user markings for each of the multiple sub-images 310.

[0059] When presenting the sub-image 310, it is possible to detect the marking operation of the user 102 on the presented sub-image 310. In some embodiments, user markings can be received via an input device (such as a display screen, mouse, etc.) of the terminal device 110. For each sub-image 310, it is possible to detect user editing markings on one or more regions of the sub-image 310 by the user 102, including privacy processing markings and / or visualization markings. It is also possible to additionally or alternatively detect user determination markings of the user 102 for each sub-image 310 to indicate whether the sub-image 310 conforms to a predetermined rule.

[0060] In some embodiments, marking functions executable on the sub-image 310 can be presented on the user interface of the terminal device 110 for the user to select. For example, Figure 4AThe user interface 400 also presents a list of marking functions to list the marking operations that can be performed on the sub-image 310. The editing area 410 in the list of marking functions includes a privacy processing function 412 that can be performed on the sub-image 310 and an overlay visualization marking function 414. Under the overlay visualization marking function 414, overlayable visualization marks are also provided, including text marks 411, sticker marks 413, and graphic marks 415. In addition, the marking list also includes a rule determination area 418, where a determination by the user on whether the sub-image 310 conforms to a predetermined rule can be received, for example, by detecting the user's selection of option "Yes" 417 or option "No" 419.

[0061] It should be understood that these marking functions are only examples, and more or fewer marking functions can be provided as needed. For example, in some cases where it is not necessary for the user to determine whether the panoramic image or the sub-image conforms to a predetermined rule, the rule determination area 418 can be omitted from the user interface 400 accordingly. In addition, the user interface 400 also provides a "Submit" option 405, and the user's selection of this option indicates that the user has completed the marking of the current sub-image 310.

[0062] In Figure 4A On the user interface 400 shown, the user 102 can perform corresponding markings on each sub-image 310 as needed. In some embodiments, if the user does not perform an edit mark or a mark regarding rule determination on the current sub-image 310, after detecting the user's selection of the "Submit" option 405, it can be determined that the user mark for the sub-image 310 is empty. In some embodiments, the user mark can indicate a privacy processing mark for one or more regions of the sub-image 310, one or more visualization marks superimposed on one or more regions of the sub-image 310, and / or a user determination mark that indicates whether the sub-image 310 conforms to a predetermined rule.

[0063] Figure 4B and Figure 4D FIG. shows a schematic diagram of an example user interface for user marking according to some embodiments of the present disclosure. In Figure 4B the example, a privacy processing mark 440 for the area 430 of the currently presented sub-image 310-1 is detected. For example, the user can indicate through an input device such as a mouse or a touch screen that the area 430 of the sub-image 310-1 needs to be processed for privacy, and then a privacy processing mark 440 can be added to this area. In this example, the privacy processing mark 440 can include a mosaic mark. In other examples, the privacy processing mark 440 can also include other marks for specific regions of the sub-image 310 to block or hide the visual information presented in that region.

[0064] In Figure 4CIn the example, a visual marker, such as text marker 442 containing input text, superimposed by the user on a region of sub-image 310-1 can be detected. In Figure 4D In the example, a user determination marker of the user for sub-image 310-1 can be detected, which indicates that image 310-1 does not conform to a predetermined rule. In addition, a visual marker, such as bounding box marker 446, superimposed by the user on a region of sub-image 310-1 can be detected to indicate that the region causes image 310-1 not to conform to the predetermined rule (e.g., due to containing privacy information).

[0065] Continuing to return to reference Figure 2 In process 200, after receiving user markers for each of multiple sub-images 310, at block 240, based on the user markers for each of the multiple sub-images 310, terminal device 110 determines a marked panoramic image 122 corresponding to panoramic image 120.

[0066] Depending on the type of the specific user marker, the marked panoramic image 122 can be determined in different ways. For user editing markers, it may be necessary to utilize the user editing markers to generate new sub-images, and generate the edited panoramic image 122 based on the new sub-images. If the user markers for multiple sub-images 310 only include user determination markers and do not include user editing markers, this means that no editing of the sub-images is required. The user determination markers can be recorded or provided as input to subsequent tasks.

[0067] Specifically, in some embodiments, for a certain sub-image 310, if the user marker for the sub-image 310 includes a user editing marker (e.g., a privacy processing marker or a superimposition of a visual marker), terminal device 110 can generate a marked sub-image based on the user editing marker. In some embodiments, the received user editing marker can indicate the coordinate information of one or more regions where the user editing marker is applied in sub-image 310, and the type indication of the user editing marker. Terminal device 110 can generate a marked sub-image locally or with the aid of a remote device (e.g., computing device 132) based on the coordinate information and the type indication of the user editing marker, such that the marked sub-image has the corresponding user editing marker. In some embodiments, if the user marker of a certain sub-image 310 among multiple sub-images indicates null, then terminal device 110 can determine the sub-image 310 as the marked sub-image.

[0068] If the user markings of one or more of the multiple sub-images 310 include user editing markings, the terminal device 110 may merge the marked multiple sub-images to generate a marked panoramic image 122. That is, when a sub-image 310 is edited, the visual information of the marked sub-image is different from the original sub-image 310. Therefore, it is necessary to perform inverse composition (or stitching) on the marked multiple sub-images to obtain the marked panoramic image 122 with user editing markings. In some embodiments, the merging process of the marked multiple sub-images may be the inverse process of the segmentation process of the multiple sub-images, so that the marked panoramic image 122 may have the same presentation, size, and proportion as the panoramic image 120.

[0069] In some embodiments, if the user markings of the multiple sub-images 310 for the panoramic image 120 do not include other user markings except user determination markings, this means that the visual information of the multiple sub-images 310 will not change due to user markings. Then, the terminal device 110 may determine the panoramic image 120 as the marked panoramic image 122 without performing a composition operation based on the sub-images.

[0070] In some embodiments, if in addition to user editing markings, the user markings for a certain sub-image 310 further include user determination markings, then the terminal device 110 may also record these user determination markings.

[0071] In some embodiments, when the user markings include user determination markings, the terminal device 110 may determine the overall user determination markings for the panoramic image 120 (or the marked panoramic image 122) based on the user determination markings for the multiple sub-images 310. For example, if the user determination markings of a certain sub-image 310 indicate that the sub-image does not conform to a predetermined rule, then the terminal device 110 may determine that the panoramic image 120 (or the marked panoramic image 122) does not conform to the predetermined rule. If the user determination markings of all the sub-images 310 indicate that these sub-images conform to the predetermined rule, then the terminal device 110 may determine that the panoramic image 120 (or the marked panoramic image 122) conforms to the predetermined rule.

[0072] In some embodiments, the terminal device 110 may store the marked panoramic image 122, such as storing it locally and / or storing it in a remote device (e.g., the computing device 132). In some embodiments, the marked panoramic image 122 may be provided as an input to a subsequent task, such as being published, or being provided for other processing, etc. There is no limitation here.

[0073] In some embodiments involving user determination marks, the terminal device 110 may store the marked panoramic image 122 in association with the overall user determination mark. In some embodiments, additionally or alternatively, the terminal device 110 may also store the user determination marks for each sub-image 310.

[0074] Reference Figure 3 In the image processing flow 300 of , after the segmented sub-images 310 are marked by the user, depending on the type of the user marks, there may be different processing branches. Specifically, if the user marks for multiple sub-images 310 include user editing marks, multiple marked sub-images 320-1, 320-2,..., 320-N (collectively or individually referred to as the marked sub-images 320) corresponding to the multiple sub-images 310 respectively may be generated. The multiple marked sub-images 320 are merged to obtain the marked panoramic image 122. If the user marks for multiple sub-images 310 only include user determination marks and do not include user editing marks, then the panoramic image 120 may be directly determined as the marked panoramic image 122.

[0075] In some embodiments, in the branch where the user marks only include user determination marks, the overall user determination mark 322 for the marked panoramic image 122 may also be determined based on the user determination marks for each sub-image 310, and the overall user determination mark 322 may be stored in association with the marked panoramic image 122. In some embodiments, in the branch where the user marks include user editing marks, there may also be user determination marks. At this time, the overall user determination mark 324 for the marked panoramic image 122 may also be determined in a similar manner and stored in association with the marked panoramic image 122.

[0076] In some embodiments, in addition to the overall user determination mark, the individual user determination marks for the sub-images may also be stored, which can be determined according to actual application needs and are not specifically limited herein.

[0077] In some embodiments, with respect to the presentation and marking of the sub-images 310, the segmentation of the panoramic image 120 and the synthesis of the marked panoramic image 122 may be performed asynchronously in the background and are transparent to the user 102. In other words, for the user performing the marking, only the marking of the presented sub-images needs to be concerned, and the input panoramic image and the output marked panoramic image do not need to be concerned. This can significantly improve the annotation efficiency and accuracy.

[0078] In a scenario where marking of multiple panoramic images needs to be performed, the segmentation and synthesis of each panoramic image can be automatically performed in the manner of a message queue. In this way, the processing capacity of the terminal device 110 can be improved. It should be understood that the terminal device 110 can also process the marking process of panoramic images in any appropriate serial or parallel manner according to the processing capacity and the needs of processing tasks, and the embodiments of the present disclosure are not limited in this regard.

[0079] Figure 5 FIG. 4 shows a schematic structural block diagram of an apparatus 500 for image processing according to some embodiments of the present disclosure. The apparatus 500 can be implemented as or included in the terminal device 15. Each module / component in the apparatus 500 can be implemented by hardware, software, firmware, or any combination thereof.

[0080] As shown in the figure, the apparatus 500 includes a segmentation module 510 configured to segment a panoramic image into multiple sub-images, and a presentation module 520 configured to present the multiple sub-images. The apparatus 500 further includes a receiving module 530 configured to receive user markings for each of the multiple sub-images, and an obtaining module 540 configured to obtain a marked panoramic image corresponding to the panoramic image based on the user markings for each of the multiple sub-images.

[0081] In some embodiments, the segmentation module 510 includes: a view obtaining module configured to determine a plurality of views associated with the scene presented by the panoramic image; and a view-based segmentation module configured to segment the panoramic image into multiple sub-images corresponding to the plurality of views.

[0082] In some embodiments, the plurality of views of the scene include at least one of the following: a top view, a bottom view, a left view, a right view, a front view, and a rear view.

[0083] In some embodiments, the presentation module 520 is configured to: present the multiple sub-images in association with indications of the plurality of views.

[0084] In some embodiments, the user markings for each of the multiple sub-images include at least one of the following: user editing markings for at least one region in the multiple images, and user determination markings indicating whether the multiple sub-images respectively conform to a predetermined rule.

[0085] In some embodiments, the user editing markings include at least one of the following: privacy processing markings for at least one region in the multiple images, and visual markings superimposed on at least one region in the multiple images.

[0086] In some embodiments, the acquisition module 540 includes: a sub-image generation module configured to generate multiple tagged sub-images based on user editing tags if the user tag includes user editing tags; and a merging module configured to generate a tagged panoramic image by merging the multiple tagged sub-images.

[0087] In some embodiments, the acquisition module 540 is configured to: if the user tag does not include other user tags except user determination tags, obtain the panoramic image as the tagged panoramic image.

[0088] In some embodiments, the user tag for each of the multiple sub-images includes a user determination tag for each of the multiple sub-images. The apparatus 500 further includes: an overall determination generation module configured to determine an overall user determination tag for the tagged panoramic image based on the user tags for each of the multiple sub-images; and a storage module configured to store the overall user determination tag and the tagged panoramic image in association with each other.

[0089] Figure 6 A block diagram of a computing device 600 in which one or more embodiments of the present disclosure may be implemented is shown. It should be understood that Figure 6 The illustrated computing device 600 is merely exemplary and should not impose any limitation on the functions and scope of the embodiments described herein. Figure 6 The illustrated computing device 600 can be used to implement Figure 1 the terminal device 110.

[0090] As Figure 6 shown, the computing device 600 is in the form of a general-purpose computing device. The components of the computing device 600 may include, but are not limited to, one or more processors or processing units 610, a memory 620, a storage device 630, one or more communication units 640, one or more input devices 650, and one or more output devices 660. The processing unit 610 may be an actual or virtual processor and is capable of performing various processes according to the programs stored in the memory 620. In a multi-processor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing ability of the computing device 600.

[0091] Computing device 600 generally includes multiple computer storage media. Such media can be any available media accessible to computing device 600, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 620 can be volatile memory (e.g., registers, caches, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 630 can be removable or non-removable media and can include machine-readable media such as flash drives, magnetic disks, or any other media that can be capable of storing information and / or data (e.g., training data for training) and can be accessed within computing device 600.

[0092] Computing device 600 can further include additional removable / non-removable, volatile / non-volatile storage media. Although not shown in Figure 6 it, a disk drive for reading from or writing to a removable, non-volatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, non-volatile optical disk can be provided. In these cases, each drive can be connected to a bus (not shown) by one or more data media interfaces. Memory 620 can include a computer program product 625 having one or more program modules that are configured to execute the various methods or actions of the various embodiments of the present disclosure.

[0093] Communication unit 640 enables communication with other computing devices via a communication medium. Additionally, the functions of the components of computing device 600 can be implemented in a single computing cluster or multiple computer machines that are capable of communicating via a communication connection. Thus, computing device 600 can operate in a networked environment using a logical connection with one or more other servers, network personal computers (PCs), or another network node.

[0094] Input device 650 can be one or more input devices such as a mouse, keyboard, trackball, etc. Output device 660 can be one or more output devices such as a display, speaker, printer, etc. Computing device 600 can also communicate with one or more external devices (not shown) as needed via communication unit 640, external devices such as storage devices, display devices, etc., communicate with one or more devices that enable a user to interact with computing device 600, or communicate with any device that enables computing device 600 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication can be performed via an input / output (I / O) interface (not shown).

[0095] According to an exemplary implementation of the present disclosure, there is provided a computer-readable storage medium having computer-executable instructions stored thereon, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, there is also provided a computer program product, the computer program product being tangibly stored on a non-transitory computer-readable medium and including computer-executable instructions, and the computer-executable instructions being executed by a processor to implement the method described above.

[0096] Aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of methods, apparatuses, devices, and computer program products according to the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and the combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0097] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine such that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device is produced that implements the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other devices to work in a specific manner. Thus, the computer-readable medium storing the instructions includes a manufactured article that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0098] The computer-readable program instructions can be loaded onto a computer, other programmable data processing device, or other device, such that a series of operation steps are executed on the computer, other programmable data processing device, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing device, or other device to implement the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0099] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various implementations of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of code, or a portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the accompanying drawings. For example, two consecutive blocks may in fact be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or acts, or by a combination of dedicated hardware and computer instructions.

[0100] The implementations of the present disclosure have been described above. The above description is exemplary, not exhaustive, and is not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The choice of terms used herein is intended to best explain the principles of the implementations, the practical application, or the improvement of technologies in the market, or to enable other ordinary skill in the art to understand the various implementation manners disclosed herein.

Claims

1. A method for image processing, comprising: Dividing a panoramic image into multiple sub-images, wherein dividing the panoramic image into the multiple sub-images includes: determining a plurality of views associated with the scene presented by the panoramic image; and dividing the panoramic image into multiple sub-images corresponding to the plurality of views; Presenting the multiple sub-images; Receiving user marks for each of the multiple sub-images; and Obtaining a marked panoramic image corresponding to the panoramic image based on the user marks for each of the multiple sub-images.

2. The method according to claim 1, wherein the plurality of views of the scene include at least one of the following: a top view, a bottom view, a left view, a right view, a front view, and a rear view.

3. The method according to claim 1, wherein presenting the multiple sub-images includes: Presenting the multiple sub-images in association with an indication of the plurality of views.

4. The method according to claim 1, wherein the user marks for each of the multiple sub-images include at least one of the following: User editing marks for at least one region in the multiple sub-images, and User determination marks, the user determination marks indicating whether the multiple sub-images respectively conform to a predetermined rule.

5. The method according to claim 4, wherein the user editing marks include at least one of the following: Privacy processing marks for at least one region in the multiple sub-images, and Visualization marks superimposed on at least one region in the multiple sub-images.

6. The method according to claim 4, wherein obtaining the marked panoramic image corresponding to the panoramic image includes: If the user marks include the user editing marks, generating marked sub-images based on the user editing marks; And Generating the marked panoramic image by combining the marked sub-images.

7. The method according to claim 4, wherein obtaining the marked panoramic image corresponding to the panoramic image includes: If the user marks do not include other user marks except the user determination marks, obtaining the panoramic image as the marked panoramic image.

8. The method according to claim 4, wherein the user marks for each of the multiple sub-images include user determination marks for each of the multiple sub-images, and the method further includes: Determining an overall user determination mark for the marked panoramic image based on the user marks for each of the multiple sub-images; And Storing the overall user determination mark in association with the marked panoramic image.

9. An apparatus for image processing, comprising: A segmentation module configured to divide a panoramic image into multiple sub-images; A presentation module configured to present the multiple sub-images; A receiving module configured to receive user marks for each of the multiple sub-images; And An obtaining module configured to obtain a marked panoramic image corresponding to the panoramic image based on the user marks for each of the multiple sub-images; The segmentation module includes: a view determination module configured to determine a plurality of views associated with the scene presented by the panoramic image; and a view-based segmentation module configured to segment the panoramic image into a plurality of sub-images corresponding to the plurality of views.

10. The apparatus according to claim 9, wherein the plurality of views of the scene include at least one of the following: a bottom view, a top view, a left view, a right view, a front view, and a rear view.

11. The apparatus according to claim 9, wherein the presentation module is configured to: present the plurality of sub-images in association with indications of the plurality of views.

12. The apparatus according to claim 9, wherein the user markings for each of the plurality of sub-images include at least one of the following: user-edited markings for at least one region in the plurality of sub-images, and user determination markings indicating whether the plurality of sub-images respectively conform to a predetermined rule.

13. The apparatus according to claim 12, wherein the user-edited markings include at least one of the following: privatization markings for at least one region in the plurality of sub-images, and visualization markings superimposed on at least one region in the plurality of sub-images.

14. The apparatus according to claim 12, wherein the acquisition module includes: a sub-image generation module configured to generate marked sub-images based on the user-edited markings if the user markings include the user-edited markings; and a merging module configured to generate the marked panoramic image by merging the marked sub-images.

15. The apparatus according to claim 12, wherein the acquisition module is configured to: acquire the panoramic image as the marked panoramic image if the user markings do not include other user markings except the user determination markings.

16. The apparatus according to claim 12, wherein the user markings for each of the plurality of sub-images include user determination markings for each of the plurality of sub-images, and the apparatus further includes: an overall determination generation module configured to determine an overall user determination marking for the marked panoramic image based on the user markings for each of the plurality of sub-images; and a storage module configured to store the overall user determination marking and the marked panoramic image in association with each other.

17. An electronic device, comprising: at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions when executed by the at least one processing unit causing the device to perform the method according to any one of claims 1 to 8.

18. A computer-readable storage medium having stored thereon a computer program, the computer program being executable by a processor to implement the method according to any one of claims 1 to 8.

Images