Image processing method, device and system

Abstract

Provided in the embodiments of the present application are an image processing method, device and system. The method comprises: displaying a setting interface, wherein the setting interface comprises a first region and a second region, the first region is configured to display first image data, and the second region is configured to display sub-image data corresponding to a local region in the first image data; and in response to a target region setting operation for the sub-image data, determining a target region.

Description

Image processing method, device and system TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of computer, in particular to an image processing method, device and system. BACKGROUND

[0002] With the rapid development of computer technology, the application range of computer is more and more extensive, which provides various conveniences for people's life, study, work and other aspects. At present, in some application scenarios such as life and study, video capture image and other methods are usually used for corresponding display. In the current video display process, object tracking technology can be used to track and process the tracking object, and then in the tracking and capturing process, a certain fixed area can be region-locked and captured, that is, the tracking object is not tracked and captured in these fixed areas. At present, in the actual interaction process of determining these fixed areas, the commonly used technical method is that the user sets the rotation angle of the image capture device such as a camera to realize it, but this setting process mainly depends on the user's experience. In the actual application process, the user needs to repeatedly determine and adjust the fixed area according to experience, and after the fixed area is determined, the user does not determine the specific corresponding distribution position, which is easy to cause multiple region position overlaps and other situations, resulting in low efficiency of fixed area setting and complex operation process.

[0003] SUMMARY

[0004] The present application aims to provide an image processing method, device and system.

[0005] In a first aspect, the present application provides an image processing method, which comprises:

[0006] displaying a setting interface, the setting interface comprising a first area and a second area, the first area being used for displaying first image data, and the second area being used for displaying sub-image data corresponding to a local area in the first image data;

[0007] in response to a target area setting operation on the sub-image data, determining a target area.

[0008] In a second aspect, the present application provides an image processing method, which comprises:

[0009] based on the determined at least one target area, performing object tracking on a target object by an image capture device; wherein the target area comprises a tracking forbidden area of the image capture device, and the tracking forbidden area is an area in which object tracking on the target object is paused during the object tracking on the target object by the image capture device;

[0010] In response to a tracking result of the target object indicating that the target object is located in a first tracking region, the image acquisition device is controlled to suspend object tracking for the target object, and the first tracking region matches at least one of the target regions.

[0011] In a third aspect, an embodiment of the present application provides an image processing device, and the device at least includes:

[0012] at least one processor;

[0013] a display connected with the processor;

[0014] and a memory connected with the processor in communication, and the memory stores one or more computer program instructions; when the instructions are executed by the processor, the processor is configured to:

[0015] generate a setting interface, and output the setting interface to the display, the setting interface includes a first region and a second region, the first region is used to display first image data, and the second region is used to display sub-image data corresponding to a local region in the first image data; in response to a target region setting operation for the sub-image data, determine a target region;

[0016] the display is configured to receive the setting interface output by the processor, and display the setting interface.

[0017] In a fourth aspect, an embodiment of the present application provides an image processing device, and the device includes:

[0018] at least one processor;

[0019] and a memory connected with the processor in communication, and the memory stores one or more computer program instructions; when the instructions are executed by the processor, the processor is configured to:

[0020] based on the determined at least one target region, perform object tracking on a target object by an image acquisition device; wherein the target region includes a tracking forbidden area of the image acquisition device, and the tracking forbidden area is a region in which object tracking for the target object is suspended in a process of performing object tracking on the target object by the image acquisition device;

[0021] in response to a tracking result of the target object indicating that the target object is located in a first tracking region, the image acquisition device is controlled to suspend object tracking for the target object, and the first tracking region matches at least one of the target regions.

[0022] In a fifth aspect, an embodiment of the present application provides an image processing system, and the system at least includes:

[0023] The image acquisition device is configured to realize object tracking and image acquisition of the target object, and is in communication connection with the image processing device.

[0024] The image processing device of the third aspect is implemented, and / or the image processing device of the fourth aspect is implemented

[0025] Through the above technical solution, the setting interface is displayed, the setting interface includes a first region and a second region, the first region is used to display first image data, and the second region is used to display sub-image data corresponding to a local region in the first image data. In response to a target region setting operation for the sub-image data, the target region is determined. In this way, after the setting interface including the first region used to display the first image data and the second region used to display the sub-image data of the first image data is displayed, the target region setting operation for the sub-image data is detected, and the target region is obtained in response to the target region setting operation. The visual setting of the tracking forbidden area is realized, the efficiency of setting the tracking forbidden area is improved, and the setting operation process of the tracking forbidden area is simplified. BRIEF DESCRIPTION OF DRAWINGS

[0026] The accompanying drawings, which are included to provide a further understanding of the present application, constitute a part of this application and illustrate embodiments of the present application and the description thereof, which serve to explain the present application, and do not constitute improper limitations on the present application. In the drawings:

[0027] FIG. 1 is a flow diagram of an image processing method provided by an embodiment of the present application;

[0028] FIG. 2 is a structural diagram of a setting interface according to an embodiment of the present application;

[0029] FIG. 3 is a structural diagram of a setting interface according to an embodiment of the present application;

[0030] FIG. 4 is a structural diagram of a setting interface according to an embodiment of the present application;

[0031] FIG. 5 is a structural diagram of a setting interface according to an embodiment of the present application;

[0032] FIG. 6 is a structural diagram of a setting interface according to an embodiment of the present application;

[0033] FIG. 7 is an interaction diagram of a setting interface according to an embodiment of the present application;

[0034] FIG. 8 is a structural diagram of a setting interface according to an embodiment of the present application;

[0035] FIG. 9 is a structural diagram of a setting interface according to an embodiment of the present application;

[0036] FIG. 10 is a flow diagram of another image processing method according to an embodiment of the present application;

[0037] FIG. 11 is a structural diagram of an image processing apparatus according to an embodiment of the present application;

[0038] FIG. 12 is a structural diagram of another image processing apparatus according to an embodiment of the present application;

[0039] FIG. 13 is a structural diagram of an image processing device according to an embodiment of the present application;

[0040] FIG. 14 is a structural diagram of another image processing device according to an embodiment of the present application;

[0041] FIG. 15 is a structural diagram of an image processing system according to an embodiment of the present application. DETAILED DESCRIPTION

[0042] The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0043] An embodiment of the present application provides an image processing method. Referring to FIG. 1, the method includes the following steps:

[0044] In step 101, a setting interface is displayed. The setting interface includes a first region and a second region. The first region is used to display first image data, and the second region is used to display sub-image data corresponding to a local region in the first image data.

[0045] In the embodiments of the present application, the method can be applied to an image processing device or an image processing system. The image processing device can be a device with a computing function and a display function, for example, a device with a computing function and a photographing function, for example, an image acquisition device such as a camera device or a video camera device, or a terminal device that can be in communication connection with the image acquisition device and can manage and control the image acquisition device, for example, a smart mobile terminal device such as a smart mobile phone, or various types of computer devices such as a tablet computer, a notebook computer, or a desktop computer. The camera device can be a camera device with a gimbal or a camera device without a gimbal.

[0046] The first image data can be an image captured by the image capturing device, or a panoramic image obtained by merging at least one image captured by the image capturing device. The image processing device generates a setting interface including two different display regions, i.e., a first region and a second region, and the setting interface is used to be displayed on a display screen, so as to be used to perform a corresponding setting operation management based on the setting interface, to realize a user-visualized setting of a target region. The image capturing device can be a camera, a video camera, or a camera module, etc.

[0047] In step 102, in response to a target region setting operation for the sub-image data, the target region is determined.

[0048] In the embodiments of the present application, the target region setting operation can be an operation of a user performing a corresponding region selection setting for the displayed setting interface, for example, can be an image region selection operation. When detecting the target region setting operation for the sub-image data of the first image data displayed in the first region, the target region setting operation determines the position distribution region of the sub-image data in the first image data, to obtain the target region.

[0049] For example, after the user performs the setting operation of the tracking forbidden area setting, the setting interface shown in FIG. 2 can be switched to, in the setting interface shown in FIG. 2, A is a first region for displaying the first image data, and B is a second region for displaying the sub-image data of the determined target region.

[0050] Based on the foregoing embodiments, in other embodiments of the present application, the first image data includes panoramic image data.

[0051] In the embodiments of the present application, the panoramic image data can be an image obtained by image stitching after the image capturing device captures an entire environment for object tracking. The panoramic image data can be spatial distribution data about the entire environment, or can be planar distribution data about the entire environment. The panoramic image data can specifically include panoramic image data in a long panoramic form, or can be panoramic image data in a spherical panoramic form, or can be other forms of panoramic image data obtained by the spherical panoramic form. In some application scenarios, the panoramic image data in the spherical panoramic form can be image data obtained by a fisheye lens. In some application scenarios, the panoramic image data can be static image data, or can be dynamic image data; the panoramic image data can be historical image data, or can be real-time image data; and the like.

[0052] Based on the foregoing embodiments, in other embodiments of the present application, the target region includes a tracking forbidden area of the image acquisition device, the tracking forbidden area being a region in which object tracking of the target object is suspended in an object tracking process of the target object based on the image acquisition device.

[0053] Based on the foregoing embodiments, in other embodiments of the present application, the image processing method further includes: displaying first identification information of the target region in the first region, the first identification information indicating a spatial correspondence relationship between the target region and the first image data.

[0054] In the embodiments of the present application, the first identification information is identification information for uniquely identifying each target region, and in some application scenarios, the first identification information can be some labels, and the label content can be a number, a name, or the like of each target region. After the target region is determined, each target region is displayed in the first region, and each target region is identified by using the first identification information. In this way, the first identification information can indicate the distribution of the target region in the first image data, so that the spatial correspondence relationship between the target region and the first image data is obtained, and the distribution of the set tracking forbidden area in the panoramic image can be presented.

[0055] For example, when the first image data displayed in the first region is panoramic image data, the first region further includes first identification information of one or more determined target regions. For example, referring to FIG. 3, 1, 2, 3, 5, and 6 in FIG. 3 are region numbers of six currently determined target regions, and the position of the number in the corresponding region can indicate the position of each target region in the panoramic image, so that the approximate position distribution of each target region in the panoramic image can be known. It should be noted that the actual image content in FIG. 3 is not shown.

[0056] Based on the foregoing embodiments, in other embodiments of the present application, the first identification information further indicates a region range of the target region.

[0057] In other embodiments of the present application, the first identification information can also be used to indicate the area range of the target area, and the first identification information can also be a closed graphic shape, so that the area distribution of the target area can be indicated. When the first identification information indicates the area range of the target area, the area range can be a regular shape, for example, a circle, a square, a rectangle, or an irregular shape, which can be determined by actual user setting operation. Further, when the first identification information indicates the area range of the target area, in order to enable the user to determine the target area more intuitively, the first identification information can also identify the target area in a color, filling, or other manner, so that the user setting the target area can intuitively know the position distribution of the target area in the first image data. For example, the first identification information can be a mask layer that can be distinguished from the first image data and indicates the position and area distribution of the target area, or a colored mask layer, for example, a green mask layer. Further, the mask layer can also be marked with the area number corresponding to the target area.

[0058] For example, for the six target areas in the first area A in FIG. 3, the area size range of each target area can also be identified by selecting the corresponding rectangular frame and using the shadow filling manner, which can be specifically shown in FIG. 4.

[0059] Based on the foregoing embodiments, in other embodiments of the present application, the image processing method further includes the following implementation steps: in response to the triggering operation on the first identification information, displaying the area adjustment control of the target area in the second area; and in response to the area adjustment operation on the area adjustment control, adjusting the area range of the target area.

[0060] In the embodiments of the present application, when the first identification information of one or more target areas is displayed in the first area, the user can adjust the first identification information. Specifically, when the user touches the display screen corresponding to the first identification information that needs to be adjusted, the triggering operation on the first identification information can be detected, so that in response to the triggering operation, the area adjustment control of the target area corresponding to the first identification information is displayed in the second area. In this way, the user can adjust the area range of the corresponding target area based on the displayed area adjustment control, that is, perform the area adjustment operation, so as to realize the re-editing of the target area, that is, adjust the area range of the target area to further meet the user's desired target area. In this way, the re-editing of the target area and the visualization of the editing and adjustment process are realized, and the user experience is ensured.

[0061] For example, when the user triggers the first identification information in the first region A1 shown in FIG. 4, for example, the region numbered 1, the sub-image data including the target region numbered 1 is displayed in the second region B, as shown in FIG. 5. At this time, the target region numbered 1 in the second region corresponds to an editable state. Thus, the user can adjust the region range of the target region numbered 1 displayed in the second region according to actual needs, for example, adjust the region to be larger, smaller, or change the region shape, or rotate adjustment, etc.

[0062] Based on the foregoing embodiments, in other embodiments of the present application, the region adjustment control adjusts the region range of the target region by adjusting at least one of the corner point, the edge, and the whole of the target region.

[0063] In the embodiments of the present application, when the region range of the target region is adjusted by the region adjustment control, the corner point, the edge, the whole, the corner point and the edge, the corner point and the whole, the corner point and the edge, and the corner point and the whole and the edge of the target region can be adjusted. The whole adjustment can be whole moving, whole shape adjustment, rotating the target region, etc. In this way, the quick adjustment of the region range of the target region is realized.

[0064] For example, the corner point can be the intersection of the four right-angle edges of the rectangular region of the target region numbered 1 shown in the second region B in FIG. 5, the edge can be the four edges of the rectangular region corresponding to the target region numbered 1, and the whole can be the whole region of the target region numbered 1, which is moved or rotated, etc. In some application scenarios, the delete shortcut operation of deleting the target region can also be included.

[0065] Based on the foregoing embodiments, in other embodiments of the present application, the first region is also used to prompt the spatial correspondence between the sub-image data and the first image data.

[0066] In the embodiments of the present application, the sub-image data corresponding to the target region is also displayed in the first region to present the distribution analysis of the sub-image data corresponding to the target region in the first image data, that is, to obtain the spatial correspondence between the sub-image data corresponding to the target region and the first image data. That is, when the image acquisition device currently acquires a corresponding sub-image data in the first image data, the preview is performed in the second region. Thus, the selection operation can be performed on the sub-image data previewed in the second region to determine the corresponding tracking region from the sub-image data displayed in the second region.

[0067] For example, as shown in FIG. 6, when the first image data is panoramic image data, the sub-image data B1 captured by the camera is displayed in the second region, and the sub-image data B1 displayed in the second region B can be identified in the first image data, so as to obtain the spatial correspondence between the sub-image data corresponding to the second region and the first image data.

[0068] Based on the foregoing embodiments, in other embodiments of the present application, the first region includes a local prompt control, and the first region prompts the spatial correspondence between the sub-image data and the first image data through the local prompt control.

[0069] Correspondingly, the step 102 determines the target region in response to the target region setting operation on the sub-image data, including: determining the local region selected in the first image data in response to the trigger operation on the local prompt control, and displaying the sub-image data corresponding to the local region in the second region; and determining the target region in response to the target region setting operation on the sub-image data.

[0070] In the embodiments of the present application, when the spatial correspondence between the sub-image data corresponding to the target region and the first image data is embodied in the first region, the local prompt control can be used to achieve this. When the local prompt control is touched by the user, a corresponding trigger operation is generated. At this time, the trigger operation corresponding to the local prompt control can be determined, the local region selected by the local prompt control in the first image data is determined, the sub-image data corresponding to the local region determined by the local prompt control is obtained and displayed in the second region. Further, if the target region setting operation of the target region determined by the sub-image data corresponding to the local region is detected, the region corresponding to the sub-image data is the target region. The target region setting operation can be achieved by corresponding touch operation on the second region, or can be achieved by directly continuing the corresponding operation on the local prompt control in the first region. The specific implementation can be determined by the actual situation, which is not limited here. In this way, the target region of the layout can be selected on the panoramic image, and the sub-image data corresponding to the selected target region can be updated in the second region. Further, the tracking forbidden area can be set according to the updated target region, which improves the efficiency of setting the tracking forbidden area and ensures the user experience.

[0071] For example, as shown in FIG. 7, when the panoramic image data displayed in the first region A includes six different target regions, and assuming that the user performs a trigger operation on the local prompt control D including the target region numbered 1, the image region corresponding to the region corresponding to D can be displayed in the second region B, so that the user can set a new tracking forbidden area in the updated second region.

[0072] In other embodiments of the present application based on the foregoing embodiments, the sub-image data displayed in the second region includes image data corresponding to the presentation board.

[0073] Correspondingly, the step 102 determines the target region in response to the target region setting operation on the sub-image data, including: displaying presentation board prompt information of the presentation board in the second region, the presentation board prompt information being used to prompt the presentation board in the sub-image data; and determining the target region in response to a target region setting operation on the presentation board prompt information.

[0074] In the embodiments of the present application, the sub-image data displayed in the second region includes image data corresponding to the presentation board, which can be obtained by recognizing the presentation board in a smart detection manner. After recognizing the presentation board, the image data corresponding to the presentation board is directly displayed in the second region to prompt the user that there is a presentation board and the distribution of the presentation board in the first image data. In this way, when a target region setting operation for setting the region where the presentation board is located as the target region is detected, the target region setting operation is responded to, and the target region where the presentation board is located is obtained. The presentation board may, for example, be a whiteboard, a transparent board, a semi-transparent board, a blackboard, a white wall, an electronic whiteboard, a screen, or any device that can display content.

[0075] In other embodiments of the present application based on the foregoing embodiments, the setting interface further includes a third region, the third region being used to determine a target region to be activated from the target regions; and correspondingly, the image processing method further includes the following steps: displaying second identification information of the at least one determined target region in the third region; determining the target region to be activated from the target regions in response to a selection operation on each second identification information; and controlling the image acquisition device based on the activated target region in response to a region activation operation on the target region to be activated.

[0076] In the embodiments of the present application, in addition to the first area and the second area, the setting interface can further include a third area, wherein the third area is used for selecting a target area to be activated from the determined one or more target areas, that is, through the displayed setting interface, the first image data with all the determined target areas and the sub-image data corresponding to the last determined target area are displayed, and the target area setting interface for performing activation setting from all the determined target areas is used. In this way, after all the target areas are determined, part or all of the target areas can be selected for activation processing according to actual needs, so as to subsequently perform corresponding object tracking operations according to the activated target areas. It should be noted that after detecting the area activation operation, the selected target area to be activated can be updated in the first image data, that is, only the activated target area is retained in the first image data; or all the determined target areas can be retained in the first image data, but the activated target area and the non-activated target area are marked with different identification information; further, the activated target area can be in an editable state, and the non-activated target area can be in a non-editable state. The specific implementation function can be set according to actual application needs, which is not limited here.

[0077] For example, the third area can be shown as area E in FIG. 8, wherein the switch button key E1 in the third area is an operation button for indicating whether the area is activated, for example, when the switch button key is in an open state, it is in an activated state, and when the switch button key is in a closed state, it is in a non-activated state. The number information displayed in area E2 in the corresponding third area is the target area that needs to be activated or the target area that does not need to be activated determined according to the state of the switch button key. Assuming that the switch button key E1 in FIG. 8 is currently in an open state, and the selected activated tracking forbidden area (which can be simply referred to as forbidden area) has numbers 1, 2, 4, and 6, then after selection, the tracking forbidden areas that are activated are the four areas with numbers 1, 2, 4, and 6.

[0078] Based on the foregoing embodiments, in other embodiments of the present application, the step 101 of displaying a setting interface includes a first area and a second area, the first area is used for displaying first image data, and the second area is used for displaying sub-image data corresponding to a local area in the first image data, which includes: displaying a setting interface including a first area and a second area, the first area displays an image drawing control, and the second area displays image data collected by an image collection device; in response to a trigger operation on the image drawing control, displaying first image data in the first area, and the image data displayed in the second area includes sub-image data corresponding to a local area in the first image data.

[0079] In the embodiments of the present application, the image drawing control is used to control the generation of the first image data. After the plurality of target regions are determined, the first region and the second region are included on the setting interface, at this time, the first image data has not been generated, when the user needs, the user can touch the image drawing control displayed at the first region, for example, a shortcut virtual button for instructing the generation of the first image data, based on the historical image data collected or the image data collected in real time, the first image data, i.e., the panoramic image data, is generated. In some application scenarios, the plurality of target regions determined can also be identified in the first image data.

[0080] For example, as shown in FIG. 9, when the first region has not generated the corresponding panoramic image data, the first region displays the image drawing control, i.e., the button for generating the panoramic image is clicked, in this way, when the user needs to generate the corresponding panoramic image data, the corresponding button is clicked to generate the corresponding panoramic image data.

[0081] Based on the foregoing embodiments, in other embodiments of the present application, the step 101 displays the setting interface including the first region and the second region, the first region is used to display the first image data, including: controlling the image acquisition device to rotate in a preset rotation range to acquire at least one second image data; performing image stitching processing on the at least one second image data to obtain the first image data; displaying the setting interface based on the first image data, the setting interface includes the first region, and the first region is used to display the first image data.

[0082] In the embodiments of the present application, when the image acquisition device is rotated in the preset rotation range, if the image acquisition device is a device with a gimbal device, for example, a camera device with a gimbal, the rotation step of the gimbal can be controlled to control the image acquisition device to rotate one circle in the maximum rotation range allowed by the gimbal, and each time the image acquisition device rotates, the image acquisition unit is controlled to capture the corresponding environmental object according to a preset acquisition frequency, so as to obtain at least one second image data; in some application scenarios, if the image acquisition device does not have a gimbal device, for example, a camera device without a gimbal, the image acquisition device can be controlled to rotate according to the actual rotation mode, for example, the user can hold the image acquisition device to rotate the image acquisition device, or the image acquisition device can rotate itself, and the image acquisition device can be controlled to rotate in the rotation range allowed by itself, which can be determined by the actual design structure of the image acquisition device, which is not limited here. After at least one second image data is obtained, the at least one second image data is subjected to image stitching processing to synthesize one image data, i.e., the first image data. After the first image data is obtained, the setting interface including the first region is generated based on the first image data. At this time, the first image data generated can be a long panoramic image data.

[0083] Based on the foregoing embodiments, in other embodiments of the present application, the image acquisition device includes at least two image acquisition units; correspondingly, step 101 displays a setting interface, and the setting interface includes a first region, and the first region is used to display first image data, including: acquiring second image data collected by each image acquisition unit respectively; performing image stitching processing on the at least two second image data to obtain the first image data; and displaying the setting interface based on the first image data, and the setting interface includes the first region, and the first region is used to display the first image data.

[0084] In the embodiments of the present application, when the image acquisition device includes at least two image acquisition units, the at least two image acquisition units can simultaneously perform image acquisition operations to obtain second image data collected by each image acquisition unit, and then the at least two second image data can be obtained, and the image stitching technology is used to perform image stitching processing on the at least two second image data, so that the first image data can be obtained. Illustratively, at least one image acquisition unit in the at least two image acquisition units can be a fisheye image acquisition lens, so that the obtained first image data can be spherical panoramic image data.

[0085] Based on the foregoing embodiments, in other embodiments of the present application, step of performing image stitching processing on the at least one second image data to obtain the first image data includes: determining a stitching region included in a feature overlapping region between any two adjacent second image data; based on each stitching region, performing color difference elimination processing on the two adjacent second image data to obtain third image data after color difference elimination; and performing image fusion processing on the third image data on both sides of each stitching region until the first image data is obtained.

[0086] In the embodiments of the present application, when the image stitching processing is performed on the at least two second image data, one implementable method is: analyzing any two spatially adjacent second image data to determine a region with the same feature, that is, a corresponding feature overlapping region, then analyzing the determined overlapping region to obtain a stitching region, then performing color difference elimination processing on the two adjacent images according to each stitching region to reduce the color difference between the two adjacent second image data, and then performing image fusion processing on the two images after color difference elimination until the at least two second image data are fused into the first image data. It should be noted that this is only one way of image fusion to obtain a panoramic image, and other image fusion methods such as spatial domain fusion or frequency domain fusion can also be applicable.

[0087] Exemplarily, the controller of the image processing device controls the camera, i.e., the aforementioned image acquisition device, to capture a series of photos at multiple angles, and a photo set is obtained, in which each photo has an overlapping area with at least one other photo. The photo set is stitched to obtain a panoramic image covering the tracking area. A corresponding relationship between the preview area and the panoramic image is established, so that the preview area and the tracking forbidden area determined by the preview area correspond to the panoramic image. The process of stitching the photo set can be summarized as follows: according to the internal parameters (referred to as internal parameters) and external parameters (referred to as external parameters) of the photos in the photo set, each photo in the photo set is mapped into the same panoramic image by using a projection model, and image fusion is performed based on the overlapping parts, so that the stitched panoramic image can be obtained. Correspondingly, a specific implementation process can be as follows:

[0088] The external parameters of each photo in the photo set obtained by controlling the camera to capture at multiple angles, i.e., the pose data of the camera, include the initial angle, Euler angle (pitch, yaw, roll), or a rotation matrix of the camera, and the internal parameters of the camera, including the focal length, curvature and other parameters of the camera. Based on the external parameters and internal parameters of each photo, a point-by-point inverse mapping relationship from the original photo to the panoramic image is established in the panoramic image, and the points in the original photo are projected into the panoramic image, including: first, according to the established projection model mapping function f, each point (u, v) in the panoramic image is mapped to a point (x, y, z) in the three-dimensional sphere: (x, y, z) = f (u, v), and then according to the rotation matrix R and the internal parameter matrix K, the coordinates (i, j) of the point in the original photo are calculated: In this way, after the photo set is projected into the same panoramic image, the corresponding overlapping area can be determined. Then, image stitching processing is performed on the overlapping area. A specific implementation process of the image stitching processing can be as follows: a dynamic programming method is used to find the best stitching seam in the overlapping area, a path is found, so that the pixel value difference sum of the two panoramic projected pictures on the path is the smallest; stitching is completed in a small resolution panoramic image, and a small resolution panoramic image with color difference eliminated is obtained by using gradient domain fusion; the small resolution color gain image of the panoramic projected picture is obtained by dividing the corresponding position of the small resolution panoramic image after the panoramic projected picture is reduced; the gain image is up-sampled and multiplied by the panoramic projected picture to obtain the picture after color difference correction, wherein the gradient domain fusion includes various fusion methods, such as Poisson fusion, multi-band fusion, etc.); when image fusion is performed, alpha blending can be used, i.e., the panoramic projected picture is weighted and fused according to the best seam on both sides, so that the fused panoramic image data can be obtained.

[0089] Based on the panoramic image data obtained as described above, the implementation method of establishing the corresponding relationship between the preview area and the panoramic image data can be: according to the intrinsic and extrinsic parameters of the photo group and the projection model used, the horizontal and vertical deflection angles of each point in the panoramic image in the unit spherical coordinates of the camera are determined, thereby establishing a connection with the attitude data such as the rotation angle of the camera, i.e. the extrinsic parameter. According to the relevant information of the intrinsic parameter of the camera, the position of each point in the panoramic image in the preview picture taken by the camera at the angle can be inferred, thereby realizing the association between the current preview picture, the tracking forbidden area and the corresponding area in the panoramic image.

[0090] Specifically, when the tracking forbidden area is selected on the panoramic image, for example, by dragging, the camera can be controlled to rotate to align with the corresponding area in the real environment. The specific implementation process is: according to the established projection model mapping function f, the three-dimensional spherical coordinates (x, y, z) of each point in the panoramic image in the real world can be obtained, thereby obtaining the horizontal and vertical angles (φ, θ) of the point. The angles are the yaw angle and the pitch angle of the camera. The corresponding relationship can be represented by the following three calculation formulas: θ = arccos(z / r).

[0091] Specifically, an implementation process of displaying the area of the preview picture to the panoramic image can be: the extrinsic parameter of the current preview picture, for example, can be a rotation matrix, and the intrinsic parameter of the camera when shooting the panoramic image is the same. In this way, the position (u, v) of each point (i, j) in the panoramic image can be obtained through the coordinate mapping relationship, thereby framing the preview picture or the selected area containing points in the corresponding position of the panoramic picture. First, according to the extrinsic parameter matrix R and the intrinsic parameter matrix K of the preview picture, the coordinates (x, y, z) of a certain point (i, j) in the preview picture under the three-dimensional spherical surface are obtained: Then, according to the established projection model mapping function g which is the inverse mapping of the aforementioned projection model mapping function f, the position (u, v) of the point (i, j) in the panoramic image is obtained: (u, v) = g(x, y, z).

[0092] In this way, the effective spatial position of the forbidden area can be determined more concisely and intelligibly by generating the panoramic image; and the user can clearly know which forbidden areas to be effective by displaying the determined tracking forbidden area in the generated panoramic image data. Further, the user can easily set the forbidden area region in the simplest and most intuitive way by selecting the tracking forbidden area according to the forbidden area, for example, by framing the tracking forbidden area, and the tracking forbidden area can be secondarily edited in the preview area, thereby improving the editing efficiency of the tracking forbidden area and simplifying the setting process of the tracking forbidden area. On the panoramic level, in addition to the beneficial effects on the interaction level, the tracking forbidden area or the preview area can be selected on the panoramic image, and the rotation of the camera can be controlled to match the corresponding real environment scene, thereby facilitating the control of the camera by the user.

[0093] The image processing method provided in the embodiments of the present application sets the setting interface, the setting interface includes a first area and a second area, the first area is used to display first image data, and the second area is used to display sub-image data corresponding to a local area in the first image data. In response to a target area setting operation on the sub-image data, the target area is determined. In this way, after the setting interface including the first area used to display the first image data and the second area used to display the sub-image data of the first image data is displayed, the target area is obtained in response to the target area setting operation after the target area setting operation on the sub-image data is detected. The tracking forbidden area is visually set, the efficiency of setting the tracking forbidden area is improved, and the operation process of setting the tracking forbidden area is simplified.

[0094] Based on the foregoing embodiments, the embodiments of the present application provide an image processing method. Referring to FIG. 10, the method includes the following steps:

[0095] In step 201, the target object is tracked by the image acquisition device based on the determined at least one target area.

[0096] The target area includes a tracking forbidden area of the image acquisition device, and the tracking forbidden area is a region in which the object tracking on the target object is paused in the process of tracking the target object by the image acquisition device.

[0097] In the embodiments of the present application, the determined at least one target area is the effective target area in the foregoing embodiments. After the at least one target area is determined, the tracking operation on the target object is performed. The target object can be one or more specific objects specified, for example, a user, or an object that needs to be processed by special tracking, for example, a display board, or a region that needs to be focused on, and the specific object can be determined by the actual situation, which is not limited here.

[0098] In step 202, in response to the tracking result of the target object indicating that the target object is located in the first tracking area, the image acquisition device is controlled to suspend tracking the target object, and the first tracking area matches the at least one target area.

[0099] In the embodiments of the present application, in the process of tracking the target object based on the image acquisition device, the tracking result of the target object is obtained. If the tracking result indicates that the target object is located in the first tracking area, the image acquisition device is controlled to suspend tracking the target object, and fixed image acquisition is performed on the first tracking area, and stable image data is output. The first tracking area belongs to one or more areas in the at least one target area.

[0100] Based on the foregoing embodiments, in other embodiments of the present application, in response to the tracking result of the target object indicating that the target object is located in the first tracking area, the image acquisition device is controlled to suspend tracking the target object, including: in response to the tracking result of the target object indicating that the target object is located in the first tracking area, the image acquisition device is controlled to suspend tracking the target object, and image acquisition is performed with the pose data corresponding to the first tracking area to obtain fourth image data; and the fourth image data is displayed.

[0101] In the embodiments of the present application, the pose data corresponding to the first tracking area is the pre-stored pose data of the image acquisition device. In this process, the image acquisition device can continuously perform image acquisition to obtain the fourth image data. The fourth image data can be an image data stream or a single frame of image data, which can be determined according to the actual application scenario. Displaying the fourth image data can be displaying the fourth image data in the corresponding display area. The display area can be a self-corresponding display screen, or other independent display screens such as a projection area of a projection device, or a device having a communication connection with the image acquisition device, etc. The specific situation can be determined according to the actual situation.

[0102] Based on the foregoing embodiments, in other embodiments of the present application, in step 201, based on the determined at least one target area, the target object is tracked by the image acquisition device, including: in the process of tracking the target object by the image acquisition device, a real-time position parameter of the target object is obtained; first pose data of the image acquisition device when the real-time position parameter is obtained is determined; distribution position parameters of the at least one target area and the real-time position parameter are analyzed, and second pose data of the image acquisition unit corresponding to the at least one target area and the first pose data are analyzed to track the target object.

[0103] In the embodiment of the present application, since the spatial distribution relationship data of the panoramic image data is obtained, when the image acquisition device performs object tracking on the target object, the actual position parameter of the target object in the panoramic image data can be determined, and at this time, the first attitude data of the image acquisition device can be directly determined, and the second attitude data of the image acquisition device corresponding to the at least one target region is matched with the current first attitude data, so as to realize object tracking on the target object.

[0104] Based on the foregoing embodiment, in other embodiments of the present application, the distribution position parameter and the real-time position parameter of the at least one target region, and the second attitude data and the first attitude data of the image acquisition unit corresponding to the at least one target region are analyzed to track the target object, including: if the real-time position parameter matches the position of the first tracking region included in the at least one target region, and the first attitude data matches the second attitude data of the first tracking region, the tracking result of the target object is that the target object is located in the first tracking region; if the real-time position parameter matches the position of at least two second tracking regions included in the at least one target region, the distribution proportion of the target detection frame used to identify the target feature of the target object in each second tracking region is determined; based on the at least two distribution proportions, the tracking result of the target object is determined.

[0105] In the embodiment of the present application, the target feature of the target object is a key feature that can identify the target object, for example, when the target object is a person, the target feature can be the facial feature of the person, and when the target object is a whiteboard, the corresponding target feature is the whiteboard feature. The target detection frame is a pre-set identification frame size experience value used to identify the target feature, which can be set according to actual conditions. When the real-time position parameter matches the position of the first tracking region included in the at least one target region, it can be that only the real-time position parameter is within the regional distribution range of the first tracking region, and at this time, the first attitude data of the image acquisition device is the same as the second attitude data collected when the first tracking region is collected, so it can be determined that the target object is located in the first tracking region. When the real-time position parameter is within the region range corresponding to at least two second tracking regions included in the at least one target region, one region needs to be determined as the first tracking region from the at least two second tracking regions, which can be analyzed and determined by determining the distribution proportion of the target detection frame used to identify the target feature of the target object in each second tracking region. The distribution proportion of the target detection frame in each second tracking region can be calculated by determining the overlapping region of the target detection frame and each second tracking region and the ratio of each second tracking region. It should be noted that each second tracking region belongs to the same order of magnitude.

[0106] In other embodiments of the present application based on the foregoing embodiments, the tracking result of the target object is determined based on the at least two distribution ratios, including: determining a target ratio from the at least two distribution ratios; determining a region corresponding to the target ratio from the at least two second tracking regions as the first tracking region, and obtaining the tracking result that the target object is located in the first tracking region.

[0107] In the embodiments of the present application, the target ratio can be the ratio with the largest value in the at least two distribution ratios. The second tracking region with the largest ratio is determined as the first tracking region.

[0108] In other embodiments of the present application based on the foregoing embodiments, the image processing method further includes: if the tracking result indicates that the target object leaves the first tracking region, continuing to perform object tracking by the image acquisition device.

[0109] In the embodiments of the present application, after the tracking result indicates that the target object leaves the first tracking region, the image acquisition device is controlled to continue to perform the object tracking operation for the target object. In this way, the reliability of the object tracking process is ensured, but the image is fixed and stable output for a specific scene, improving the user experience effect.

[0110] Based on the foregoing embodiments, the embodiments of the present application provide an image processing method, which is used to, after a user sets at least one tracking forbidden area based on the foregoing method, analyze the distribution information of the target detection frame of the object recognition for the target object in the image by using the pose coordinates of the camera corresponding to the tracking forbidden area, the region distribution coordinates of the tracking forbidden area, and the current pose coordinates of the camera in the object tracking process, to determine the relationship between the target object and the at least one tracking forbidden area determined, that is, to determine whether the target object is in the tracking forbidden area. If the target object is in the tracking forbidden area, the pose data of the camera can be directly locked as the pose data of the camera corresponding to the tracking forbidden area, so as to realize locking the corresponding display picture as the picture corresponding to the tracking forbidden area. If the target object is not in the tracking forbidden area or leaves the tracking forbidden area, the camera is controlled to continue to perform the object tracking operation for the target object.

[0111] It should be noted that in the implementation process, when the target detection frame is detected for each frame of acquired image data, it needs to be compared and analyzed with the several tracking forbidden areas set by the user.

[0112] In this way, the image processing in the device realizes the function of multiple field of view (MFOV) and the effect of automatic framing. Thus, in the application scenarios such as live broadcast and teaching, the user is provided with a more convenient tracking forbidden area experience. When the tracking object is near the tracking forbidden area, the picture will be stabilized, the picture board writing or the displayed object in the forbidden area will be locked, and the user's use experience effect is improved.

[0113] It should be noted that the descriptions of the same steps and contents in the embodiments and other embodiments are referred to the descriptions in other embodiments, and will not be described here.

[0114] The image processing method provided by the embodiments of the present application determines at least one target region, and performs object tracking on the target object by the image acquisition device. In response to the tracking result of the target object indicating that the target object is located in the first tracking region, the image acquisition device is controlled to suspend the object tracking on the target object. In this way, after at least one target region for suspending tracking is determined, when the object tracking on the target object is performed, if it is detected that the target object is located in the first tracking region in the at least one target region, the object tracking on the target object is suspended, and the first tracking region is directly image-acquired, so that the content in the tracking forbidden area is stably acquired and the content in the specific region is stably displayed when the target object enters the tracking forbidden area.

[0115] Based on the foregoing embodiments, the embodiments of the present application provide an image processing device. The device can be applied to the image processing method provided by the foregoing embodiments. Referring to FIG. 11, the image processing device 3 can include a display unit 31 and a first response unit 32. Wherein:

[0116] The display unit 31 is configured to display a setting interface. The setting interface includes a first region and a second region. The first region is configured to display first image data, and the second region is configured to display sub-image data corresponding to a local region in the first image data.

[0117] The first response unit 32 is configured to determine a target region in response to a target region setting operation on the sub-image data.

[0118] In other embodiments of the present application, the first image data includes panoramic image data.

[0119] In other embodiments of the present application, the target region includes a tracking forbidden area of the image acquisition device. The tracking forbidden area is a region in which the object tracking on the target object is suspended during the object tracking on the target object by the image acquisition device.

[0120] In other embodiments of the present application, the display unit is configured to display first identification information of the target region in the first region, the first identification information indicating a spatial correspondence between the target region and the first image data.

[0121] In other embodiments of the present application, the first identification information further indicates a region range of the target region.

[0122] In other embodiments of the present application, the first response unit is further configured to:

[0123] In response to a trigger operation on the first identification information, display a region adjustment control of the target region in the second region;

[0124] In response to a region adjustment operation on the region adjustment control, adjust the region range of the target region.

[0125] In other embodiments of the present application, the region adjustment control adjusts the region range of the target region by adjusting at least one of a corner point, an edge, and an entirety of the target region.

[0126] In other embodiments of the present application, the first region is further configured to prompt the spatial correspondence between the sub-image data and the first image data.

[0127] In other embodiments of the present application, the first region includes a local prompt control, and the first region prompts the spatial correspondence between the sub-image data and the first image data through the local prompt control.

[0128] The first response unit performs the step of determining the target region in response to a target region setting operation on the sub-image data, which can be implemented by the following steps:

[0129] In response to a trigger operation on the local prompt control, determine a local region selected in the first image data, and display sub-image data corresponding to the local region in the second region;

[0130] In response to a target region setting operation on the sub-image data, determine the target region.

[0131] In other embodiments of the present application, the sub-image data displayed in the second region includes image data corresponding to a presentation board;

[0132] The first response unit performs the step of determining the target region in response to a target region setting operation on the sub-image data, which can be implemented by the following steps:

[0133] Display presentation board prompt information of the presentation board in the second region, the presentation board prompt information being configured to prompt the presentation board in the sub-image data.

[0134] In response to a target region setting operation on the presentation board prompt information, determine the target region.

[0135] In other embodiments of the present application, the setting interface further includes a third region, the third region being configured to determine the target region to be activated from the target regions; and the display unit is further configured to display the second identification information of the determined at least one target region in the third region.

[0136] The first response unit is further configured to determine the target region to be activated from the target regions in response to a selection operation on each second identification information; and to control the image acquisition device based on the activated target region in response to a region activation operation on the target region to be activated.

[0137] In other embodiments of the present application, when the display unit displays the setting interface, the setting interface includes a first region and a second region, the first region is configured to display the first image data, and the second region is configured to display the sub-image data corresponding to the local region in the first image data. This can be achieved by the following steps:

[0138] The display unit displays the setting interface, the setting interface includes a first region and a second region, the first region displays the image drawing control, and the second region displays the image data acquired by the image acquisition device.

[0139] In response to a trigger operation on the image drawing control, the display unit displays the first image data in the first region, and displays the image data including the sub-image data corresponding to the local region in the first image data in the second region.

[0140] In other embodiments of the present application, when the display unit displays the setting interface, the setting interface includes a first region, the first region is configured to display the first image data. This can be achieved by the following steps:

[0141] The image acquisition device is controlled to rotate within a preset rotation range to acquire at least one second image data.

[0142] The at least one second image data is subjected to image stitching processing to obtain the first image data.

[0143] The display unit displays the setting interface based on the first image data, the setting interface includes a first region, the first region is configured to display the first image data.

[0144] In other embodiments of the present application, the image acquisition device includes at least two image acquisition units.

[0145] In other embodiments of the present application, when the display unit displays the setting interface, the setting interface includes a first region, the first region is configured to display the first image data. This can be achieved by the following steps:

[0146] The second image data acquired by each image acquisition unit is acquired respectively.

[0147] perform image splicing processing on the at least two pieces of second image data to obtain first image data;

[0148] display a setting interface based on the first image data, the setting interface including a first region for displaying the first image data.

[0149] In other embodiments of the present application, when the display unit performs the step of performing image splicing processing on the at least one piece of second image data to obtain first image data, it can be achieved by the following steps:

[0150] determining a splicing region included in a feature overlap region between any two adjacent pieces of second image data;

[0151] based on each splicing region, performing color difference elimination processing on the two adjacent pieces of second image data to obtain third image data after color difference elimination;

[0152] performing image fusion processing on the third image data on both sides of each splicing region until the first image data is obtained.

[0153] The interaction process between the units and modules in the embodiments of the present application can refer to the interaction process between the steps of the foregoing method embodiments, which will not be described in detail here.

[0154] Based on the foregoing embodiments, the embodiments of the present application provide an image processing device, which can be applied to the image processing method provided in the foregoing embodiments. Referring to FIG. 12, the image processing device 4 can include a tracking unit 41 and a second response unit 42; wherein:

[0155] The tracking unit 41 is configured to perform object tracking on the target object by the image acquisition device based on the determined at least one target region; wherein the target region includes a tracking forbidden area of the image acquisition device, and the tracking forbidden area is a region in which object tracking on the target object is temporarily suspended in the process of performing object tracking on the target object by the image acquisition device;

[0156] The second response unit 42 is configured to, in response to the tracking result of the target object indicating that the target object is located in the first tracking region, control the image acquisition device to temporarily suspend object tracking on the target object, and the first tracking region matches the at least one target region.

[0157] In other embodiments of the present application, the second response unit is specifically configured to implement the following steps:

[0158] In response to the tracking result of the target object indicating that the target object is located in the first tracking region, the image acquisition device is controlled to temporarily suspend tracking on the target object, and image acquisition is performed with the pose data corresponding to the first tracking region to obtain fourth image data;

[0159] displaying the fourth image data.

[0160] In other embodiments of the present application, the tracking unit is specifically configured to implement the following steps:

[0161] In the object tracking process of the target object by the image acquisition device, the real-time position parameter of the target object is acquired;

[0162] The first attitude data of the image acquisition device when the real-time position parameter is acquired is determined;

[0163] The distribution position parameter and the real-time position parameter of the at least one target region, and the second attitude data and the first attitude data of the image acquisition device corresponding to the at least one target region are analyzed to track the target object.

[0164] In other embodiments of the present application, when the tracking unit performs the step of analyzing the distribution position parameter and the real-time position parameter of the at least one target region, and the second attitude data and the first attitude data of the image acquisition device corresponding to the at least one target region to track the target object, the following steps can be implemented:

[0165] If the real-time position parameter matches the position of the first tracking region included in the at least one target region, and the first attitude data matches the second attitude data of the first tracking region, the tracking result of the target object is that the target object is located in the first tracking region.

[0166] If the real-time position parameter matches the position of at least two second tracking regions included in the at least one target region, the distribution proportion of the target detection frame used to identify the target feature of the target object in each second tracking region is determined; based on the at least two distribution proportions, the tracking result of the target object is determined.

[0167] In other embodiments of the present application, when the tracking unit performs the step of determining the tracking result of the target object based on the at least two distribution proportions, the following steps can be implemented:

[0168] A target proportion is determined from the at least two distribution proportions;

[0169] From the at least two second tracking regions, a region corresponding to the target proportion is determined as the first tracking region, and the tracking result that the target object is located in the first tracking region is obtained.

[0170] In other embodiments of the present application, the tracking unit is further configured to implement the following steps:

[0171] If the tracking result indicates that the target object leaves the first tracking region, the object tracking is continued by the image acquisition device.

[0172] The interaction process between units and modules in the embodiments of the present application can refer to the interaction process between steps in the foregoing method embodiments, which will not be described in detail here.

[0173] Based on the foregoing embodiments, in other embodiments of the present application, the embodiments of the present application provide an image processing device, which can be applied to the image processing method provided in the foregoing embodiments. Referring to FIG. 13, the image processing device 5 at least includes:

[0174] at least one processor 51;

[0175] a display 52 connected with the processor;

[0176] and a memory 53 in communication connection with the processor, the memory storing one or more computer program instructions; when the instructions are executed by the processor, the processor is configured to generate a setting interface, output the setting interface to the display, the setting interface including a first area and a second area, the first area being used to display first image data, and the second area being used to display sub-image data corresponding to a target area of the first image data; in response to a target area setting operation for the sub-image data, determining the target area; and the display is configured to receive the setting interface output by the processor, and display the setting interface, so as to realize the implementation process of the image processing method provided in FIG. 1 and corresponding other embodiments, which will not be described here.

[0177] It should be noted that the image processing device corresponding to FIG. 13 in the embodiments can be an image acquisition device itself with a display screen, or can be a terminal device with a display screen in communication connection with the image acquisition device. In some application scenarios, the image processing device realizing FIG. 1 and corresponding other embodiments can also not have a display, so that the corresponding image processing device can generate a setting interface, output the setting interface, and respond to the related content update in the setting interface according to the user operation actions or operation instructions sent by other devices capable of detecting the user operation for the setting interface. The specific situation can be determined by the actual situation, which will not be specifically limited here.

[0178] Based on the foregoing embodiments, the embodiments of the present application provide an image processing device, which is applied to the foregoing corresponding method embodiments. Referring to FIG. 14, the image processing device 6 includes:

[0179] at least one processor 61;

[0180] and a memory 62 in communication connection with the processor, the memory storing one or more computer program instructions; when the instructions are executed by the processor, the processor is configured to realize the implementation process of the image processing method provided in FIG. 2 and corresponding other embodiments, which will not be described in detail here.

[0181] It should be noted that the image processing device corresponding to FIG. 14 in the embodiments of the present application can be a device without a display in some application scenarios, and the image processing device can also be a device with a display in some application scenarios. The specific determination can be made according to the actual application scenario. The relationship between the image processing device and the image acquisition apparatus can be that the image acquisition apparatus and the image processing device are integrated, that is, the image acquisition apparatus is arranged on the image processing device, and the image processing device is the image acquisition apparatus itself, such as a camera device or a video camera device. The image acquisition apparatus and the image processing device can also be two independent devices, and the two devices are in communication connection. The image processing device realizes the corresponding process in the embodiments of the present application by controlling image acquisition.

[0182] Based on the foregoing embodiments, an image processing system is provided in the embodiments of the present application. The image processing system can be applied to the foregoing corresponding method embodiments. Referring to FIG. 15, the image processing system 7 at least includes an image acquisition apparatus 71 and an image processing device 72. Wherein:

[0183] The image acquisition apparatus 71 is configured to realize object tracking and image acquisition of a target object. The image acquisition apparatus is in communication connection with the image processing device.

[0184] The image processing device 72 can be configured to realize the implementation process of the image processing method provided in FIG. 1 and the corresponding embodiments thereof, and / or can realize the implementation process of the image processing method provided in FIG. 2 and the corresponding embodiments thereof.

[0185] It should be noted that when the image processing device realizes the implementation process of the image processing method provided in FIG. 1 and the corresponding embodiments thereof, and can realize the implementation process of the image processing method provided in FIG. 2 and the corresponding embodiments thereof, the image processing device realizing the implementation process of the image processing method provided in FIG. 1 and the corresponding embodiments thereof can be the same device as the image processing device realizing the implementation process of the image processing method provided in FIG. 2 and the corresponding embodiments thereof, or can not be the same device. For example, in some application scenarios, the image processing device realizing the image processing method provided in FIG. 1 and the corresponding embodiments thereof can be a touchable computer device, and the image processing device realizing the image processing method provided in FIG. 2 and the corresponding embodiments thereof can be a gimbal camera.

[0186] In FIG. 15, the image processing device and the image acquisition apparatus are two independent devices. The image acquisition apparatus can manage and control the image acquisition apparatus, and perform corresponding processing based on the image acquired by the image acquisition apparatus. The image processing device can be a device with a display or a device without a display. The specific determination can be made according to the actual application scenario, which is not limited here.

[0187] When the image processing device does not have a display, when it is necessary to display the corresponding setting interface, the image processing device can send the generated setting interface to a display device having a communication connection for display, and after the display device or other detection device detects the user's interactive control operation, the interactive control operation or the corresponding operation control instruction is sent to the image processing device to enable the image processing device to manage and control the image acquisition device.

[0188] Based on the foregoing embodiments, an embodiment of the present application provides a computer readable storage medium, referred to as a storage medium, which stores one or more programs executable by one or more processors to implement the implementation process of the image processing method provided with reference to the foregoing embodiments, which will not be described here again.

[0189] Based on the foregoing embodiments, an embodiment of the present application further provides a computer program product comprising a computer program executable by a processor of an image processing device to complete any of the foregoing method steps.

[0190] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware, or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0191] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be described here again.

[0192] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the above-described device embodiments are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0193] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0194] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit.

[0195] The functions, if realized in the form of software functional units and sold or used as independent products, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application or the part of the present application that essentially contributes to the prior art or the part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

[0196] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be limited by the protection scope of the claims.

Claims

1. An image processing method, the method comprising: displaying a setting interface, the setting interface comprising a first region and a second region, the first region being configured to display first image data, and the second region being configured to display sub-image data corresponding to a local region in the first image data; in response to a target region setting operation on the sub-image data, determining a target region.

2. The method of claim 1, wherein, The first image data comprises panoramic image data.

3. The method of claim 1, wherein, The target region comprises a tracking forbidden area of an image acquisition device, the tracking forbidden area being an area in which object tracking on a target object is suspended during an object tracking process on the target object based on the image acquisition device.

4. The method of claim 1, wherein, The method further comprises: displaying first identification information of the target region in the first region, the first identification information indicating a spatial correspondence between the target region and the first image data.

5. The method of claim 4, wherein, The first identification information further indicates a region range of the target region.

6. The method of claim 5, wherein, The method further comprises: in response to a trigger operation on the first identification information, displaying a region adjustment control of the target region in the second region; in response to a region adjustment operation on the region adjustment control, adjusting the region range of the target region.

7. The method of claim 6, wherein, The region adjustment control adjusts the region range of the target region by adjusting at least one of a corner point, an edge, and an entirety of the target region.

8. The method of claim 1, wherein, The first region is further configured to prompt a spatial correspondence between the sub-image data and the first image data.

9. The method of claim 8, wherein, The first region comprises a local prompt control, and the first region prompts the spatial correspondence between the sub-image data and the first image data through the local prompt control. The response to the target region setting operation on the sub-image data to determine the target region comprises: in response to a trigger operation on the local prompt control, determining a local region selected in the first image data, and displaying sub-image data corresponding to the local region in the second region; in response to a target region setting operation on the sub-image data, determining the target region.

10. The method of claim 1, wherein, The sub-image data displayed in the second region comprises image data of a presentation board; The response to the target region setting operation on the sub-image data to determine the target region comprises: displaying presentation board prompt information of the presentation board in the second region, the presentation board prompt information being configured to prompt the presentation board in the sub-image data; in response to a target region setting operation on the presentation board prompt information, determining the target region.

11. The method of claim 1, wherein, The setting interface further comprises a third region, the third region being configured to determine a target region to be activated from the target regions; and the method further comprises: displaying second identification information of at least one determined target region in the third region; in response to a selection operation on each of the second identification information, determining a target region to be activated from the target regions; in response to a region activation operation on the target region to be activated, controlling an image acquisition device based on the activated target region.

12. The method of claim 1, wherein, The display setting interface includes a first area and a second area, the first area is used for displaying first image data, and the second area is used for displaying sub-image data corresponding to a local area in the first image data. The display setting interface includes the first area and the second area, the first area displays an image drawing control, and the second area displays image data collected by an image collection device. In response to a triggering operation on the image drawing control, the first image data is displayed in the first area, and the image data displayed in the second area includes sub-image data corresponding to a local area in the first image data.

13. The method of claim 1, wherein, The display setting interface includes a first area, and the first area is used for displaying first image data. The image collection device is controlled to rotate in a preset rotation range to collect at least one second image data. The first image data is obtained by performing image splicing processing on at least one second image data. Based on the first image data, a setting interface is displayed, and the setting interface includes a first area used for displaying the first image data.

14. The method of claim 1, wherein, The image collection device includes at least two image collection units. The display setting interface includes a first area, and the first area is used for displaying first image data. Second image data collected by each image collection unit is obtained. The first image data is obtained by performing image splicing processing on at least two second image data. Based on the first image data, a setting interface is displayed, and the setting interface includes a first area used for displaying the first image data.

15. The method of claim 13, wherein, The first image data is obtained by performing image splicing processing on at least one second image data, including: A splicing area included in a feature overlapping area between any two adjacent second image data is determined. Based on each splicing area, color difference elimination processing is performed on two adjacent second image data to obtain third image data after color difference elimination. Image fusion processing is performed on the third image data on both sides of each splicing area until the first image data is obtained.

16. An image processing method, the method comprising: Based on the determined at least one target area, performing object tracking on a target object by an image collection device; wherein the target area includes a tracking forbidden area of the image collection device, and the tracking forbidden area is an area in which object tracking on the target object is temporarily suspended in the process of performing object tracking on the target object by the image collection device; In response to a tracking result of the target object indicating that the target object is located in a first tracking area, the image collection device is controlled to temporarily suspend object tracking on the target object, and the first tracking area matches at least one target area. In response to a tracking result of the target object indicating that the target object is located in a first tracking area, the image collection device is controlled to temporarily suspend object tracking on the target object, including:

17. The method of claim 16, wherein, ​ in response to the tracking result of the target object indicating that the target object is located in a first tracking area, controlling the image acquisition device to pause tracking the target object and to perform image acquisition with pose data corresponding to the first tracking area, to obtain fourth image data; displaying the fourth image data.

18. The method of claim 16, wherein, the object tracking of the target object by the image acquisition device based on the determined at least one target area comprises: acquiring real-time position parameters of the target object during the object tracking of the target object by the image acquisition device; determining first pose data of the image acquisition device at the time of acquiring the real-time position parameters; analyzing distribution position parameters of at least one of the target areas and the real-time position parameters, and second pose data of the image acquisition device corresponding to at least one of the target areas and the first pose data, to track the target object.

19. The method of claim 18, wherein, the analysis of the distribution position parameters of at least one of the target areas and the real-time position parameters, and the second pose data of the image acquisition device corresponding to at least one of the target areas and the first pose data, to track the target object, comprises: if the real-time position parameters match the position of a first tracking area included in at least one of the target areas, and the first pose data matches second pose data of the first tracking area, the tracking result of the target object is that the target object is located in the first tracking area; if the real-time position parameters match the position of at least two second tracking areas included in at least one of the target areas, a distribution proportion of a target detection frame used to identify a target feature of the target object in each of the second tracking areas is determined; based on at least two of the distribution proportions, a tracking result of the target object is determined.

20. The method of claim 19, wherein, the determination of the tracking result of the target object based on at least two of the distribution proportions comprises: determining a target proportion from at least two of the distribution proportions; from at least two of the second tracking areas, determining a region corresponding to the target proportion as the first tracking area, so that the tracking result is that the target object is located in the first tracking area. the method further comprises:

21. The method of claim 16, wherein, if the tracking result indicates that the target object leaves the first tracking area, continuing to perform object tracking by the image acquisition device.

22. An image processing device, the device comprising at least: at least one processor; a display connected to the processor; and a memory connected in communication with the processor, the memory storing one or more computer program instructions; when the instructions are executed by the processor, the processor is configured to: generate a setting interface, output the setting interface to the display, the setting interface comprising a first area and a second area, the first area being used to display first image data, and the second area being used to display sub-image data corresponding to a local area in the first image data; in response to a target area setting operation on the sub-image data, determine a target area; ​ The display is configured to receive the setting interface output by the processor and display the setting interface.

23. The apparatus of claim 22, wherein, The first image data includes panoramic image data.

24. The apparatus of claim 22, wherein, The target region includes a tracking forbidden area of the image acquisition device, and the tracking forbidden area is a region in which object tracking for a target object is suspended in an object tracking process based on the image acquisition device on the target object.

25. The apparatus of claim 22, wherein, The processor is further configured to: display first identification information of the target region in the first region, the first identification information indicating a spatial correspondence between the target region and the first image data.

26. The apparatus of claim 25, wherein, The first identification information further indicates a region range of the target region.

27. The apparatus of claim 26, wherein, The processor is further configured to: in response to a triggering operation on the first identification information, display a region adjustment control of the target region in the second region; in response to a region adjustment operation on the region adjustment control, adjust the region range of the target region.

28. The apparatus of claim 27, wherein, The region adjustment control adjusts the region range of the target region by adjusting at least one of a corner point, an edge, and an entirety of the target region.

29. The apparatus of claim 22, wherein, The first region is further used to prompt a spatial correspondence between the sub-image data and the first image data.

30. The apparatus of claim 28, wherein, The first region includes a local prompt control, and the first region prompts the spatial correspondence between the sub-image data and the first image data through the local prompt control. When the processor determines a target region in response to a target region setting operation on the sub-image data, the processor is configured to: in response to a triggering operation on the local prompt control, determine a local region selected in the first image data, and display sub-image data corresponding to the local region in the second region; determine the target region in response to a target region setting operation on the sub-image data.

31. The apparatus of claim 22, wherein, The sub-image data displayed in the second region includes image data corresponding to a display board; When the processor determines a target region in response to a target region setting operation on the sub-image data, the processor is configured to: display display board prompt information of the display board in the second region, the display board prompt information being used to prompt the display board in the sub-image data; determine the target region in response to a target region setting operation on the display board prompt information.

32. The apparatus of claim 22, wherein, The setting interface further includes a third region, and the third region is used to determine a target region to be validated from the target regions; the processor is further configured to: display second identification information of at least one determined target region in the third region; determine a target region to be validated from the target regions in response to a selection operation on each second identification information; control the image acquisition device based on the validated target region in response to a region validation operation on the target region to be validated.

33. The apparatus of claim 22, wherein, In a process of displaying the setting interface, the setting interface comprises a first area and a second area, the first area is used for displaying first image data, and the second area is used for displaying sub-image data corresponding to a local area in the first image data. In a process of displaying the setting interface, the setting interface comprises a first area and a second area, the first area is used for displaying first image data, and the second area is used for displaying sub-image data corresponding to a local area in the first image data. In a process of displaying the setting interface, the setting interface comprises a first area and a second area, the first area is used for displaying first image data, and the second area is used for displaying sub-image data corresponding to a local area in the first image data.

34. The apparatus of claim 22, wherein, In a process of displaying the setting interface, the setting interface comprises a first area and a second area, the first area is used for displaying first image data, and the second area is used for displaying sub-image data corresponding to a local area in the first image data. The image acquisition device comprises at least two image acquisition units. In a process of displaying the setting interface, the setting interface comprises a first area and a second area, the first area is used for displaying first image data, and the second area is used for displaying sub-image data corresponding to a local area in the first image data. In a process of displaying the setting interface, the setting interface comprises a first area and a second area, the first area is used for displaying first image data, and the second area is used for displaying sub-image data corresponding to a local area in the first image data.

35. The apparatus of claim 22, wherein, In a process of displaying the setting interface, the setting interface comprises a first area and a second area, the first area is used for displaying first image data, and the second area is used for displaying sub-image data corresponding to a local area in the first image data. In a process of displaying the setting interface, the setting interface comprises a first area and a second area, the first area is used for displaying first image data, and the second area is used for displaying sub-image data corresponding to a local area in the first image data.

37. An image processing device, the device comprising: at least one processor; and a memory connected with the processor in communication, the memory storing one or more computer program instructions; 36. The apparatus of claim 35, wherein, when the instructions are executed by the processor, the processor is configured to: based on the determined at least one target area, tracking a target object by an image acquisition device; wherein the target area comprises a tracking forbidden area of the image acquisition device, and the tracking forbidden area is an area in which object tracking for the target object is temporarily suspended in a process of tracking the target object by the image acquisition device. ​ ​ ​ ​ ​ ​ ​ In response to the tracking result of the target object indicating that the target object is located in a first tracking area, the image acquisition device is controlled to suspend object tracking for the target object, and the first tracking area matches at least one of the target areas.

38. The apparatus of claim 37, wherein, The response to the tracking result of the target object indicating that the target object is located in a first tracking area, the image acquisition device is controlled to suspend object tracking for the target object, comprising: In response to the tracking result of the target object indicating that the target object is located in a first tracking area, the image acquisition device is controlled to suspend tracking for the target object, and the first tracking area corresponds to the pose data of the image acquisition device to obtain fourth image data; Display the fourth image data.

39. The apparatus of claim 37, wherein, The image acquisition device is used to track the target object based on the determined at least one target area, comprising: During the object tracking process of the target object by the image acquisition device, the real-time position parameter of the target object is obtained; Determine the first pose data of the image acquisition device when the real-time position parameter is obtained; Analyze the distribution position parameter of at least one of the target areas and the real-time position parameter, and the second pose data of the image acquisition device corresponding to at least one of the target areas and the first pose data, to track the target object.

40. The apparatus of claim 39, wherein, The analysis of the distribution position parameter of at least one of the target areas and the real-time position parameter, and the second pose data of the image acquisition device corresponding to at least one of the target areas and the first pose data, to track the target object, comprising: If the real-time position parameter matches the position of a first tracking area included in at least one of the target areas, and the first pose data matches the second pose data of the first tracking area, the tracking result of the target object is that the target object is located in the first tracking area; If the real-time position parameter matches the position of at least two second tracking areas included in at least one of the target areas, determine the distribution proportion of the target detection frame used to identify the target feature of the target object in each of the second tracking areas; based on at least two of the distribution proportions, determine the tracking result of the target object.

41. The apparatus of claim 40, wherein, The determination of the tracking result of the target object based on at least two of the distribution proportions, comprising: Determine a target proportion from at least two of the distribution proportions; From at least two of the second tracking areas, determine the area corresponding to the target proportion as the first tracking area, and obtain the tracking result that the target object is located in the first tracking area.

42. The apparatus of claim 37, wherein, The processor is further configured to: If the tracking result indicates that the target object leaves the first tracking area, continue to perform object tracking by the image acquisition device.

43. An image processing system, the system comprising: an image acquisition device for realizing object tracking and image acquisition of a target object, the image acquisition device being in communication connection with the image processing device; The image processing device of any of claims 22-36, and / or the image processing device of any of claims 37-42.

Images