Image display method, device, apparatus, and storage medium

Abstract

The application provides an image display method, device, equipment and storage medium, target detection is performed on a panoramic image, a detail area in the panoramic image is determined, coordinates of a first center point of the panoramic image and a field angle of the panoramic image are used for performing straight line projection transformation processing on the coordinates of the first center point, the coordinates of a second center point of the first center point in a straight line projection mode are determined, the coordinates of the second center point are used as a fixed projection center, the target attitude angle and the target field angle of the detail area in the straight line projection mode are determined according to geometric information of the detail area and geometric information of the panoramic image, a transformation matrix is constructed according to the coordinates of the second center point, the target attitude angle and the target field angle, the detail area is transformed through the transformation matrix, the target image of the detail area in the straight line projection mode is determined and displayed. The technical scheme of the application can eliminate the distortion of the detail area in the display.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image display method, apparatus, device, and storage medium. Background Technology

[0002] With the continuous development of various technologies in the field of surveillance, higher requirements have been put forward for the images obtained after scene surveillance, namely, the need to obtain higher resolution images after scene surveillance. High resolution images are generally formed by stitching together images taken by multiple surveillance cameras to form a high resolution panoramic image.

[0003] In related technologies, in order to present high-resolution panoramic images, images captured by multiple surveillance cameras are usually stitched together and then presented using cylindrical projection to display the final panoramic image and images of detailed areas within the panoramic image.

[0004] However, the aforementioned techniques can lead to significant distortion in the detailed areas of the panoramic image, affecting the presentation of those areas. Summary of the Invention

[0005] This invention provides an image display method, apparatus, device, and storage medium to address the shortcomings of existing technologies that use cylindrical projection to display panoramic images and their detailed areas, resulting in significant distortion in the detailed areas and affecting the display effect. The invention achieves the effect of eliminating distortion in the detailed areas and improving the processing performance of the device by transforming the detailed areas of the panoramic image from cylindrical projection to linear projection.

[0006] This invention provides an image display method, comprising: Target detection is performed on the panoramic image to determine the detailed regions in the panoramic image; the detailed regions are the regions corresponding to the targets; the panoramic image is the image obtained by cylindrical projection and stitching together the initial images acquired by the multi-view cameras respectively. Based on the coordinates of the first center point corresponding to the panoramic image and the first field of view corresponding to the panoramic image, the coordinates of the first center point are transformed by linear projection to determine the coordinates of the second center point corresponding to the first center point under the linear projection method. Using the coordinates of the second center point as the fixed projection center, the target attitude angle and the target field of view angle of the detailed region under the linear projection method are determined based on the geometric information of the detailed region and the geometric information of the panoramic image. A transformation matrix is ​​constructed based on the coordinates of the second center point, the target attitude angle, and the target field of view. The transformation matrix is ​​then used to transform the detail region, and the target image corresponding to the detail region in the linear projection mode is determined and displayed.

[0007] According to an image display method provided by the present invention, the geometric information of the detailed region includes the coordinates of the starting point and the ending point of the detailed region, and the geometric information of the panoramic image includes the width and height of the panoramic image and the first field of view of the panoramic image. Determining the target attitude angle of the detailed region in a linear projection mode based on the geometric information of the detailed region and the geometric information of the panoramic image includes: The coordinates of the starting and ending points of the detail region are averaged along the same axis to determine the coordinates of the geometric center corresponding to the detail region, and the coordinates of the geometric center are used as the coordinates of the third center point. Based on the coordinates of the third center point of the detailed region, the first field of view of the panoramic image, and the width and height of the panoramic image, the target attitude angle corresponding to the detailed region in the linear projection method is determined.

[0008] According to an image display method provided by the present invention, determining the target pose angle of the detail region in linear projection mode based on the coordinates of the third center point of the detail region, the first field of view of the panoramic image, and the width and height of the panoramic image includes: Based on the width and height of the panoramic image, determine the coordinates of the first center point corresponding to the panoramic image; Based on the coordinates of the third center point, the coordinates of the first center point, the first field of view of the panoramic image, and the width and height of the panoramic image, determine the target attitude angle corresponding to the detailed region under the linear projection method.

[0009] According to an image display method provided by the present invention, the geometric information of the detailed region includes the coordinates of the starting point and the ending point of the detailed region, and the geometric information of the panoramic image includes the width and height of the panoramic image and the first field of view of the panoramic image. Determining the target field of view corresponding to the detailed region in a linear projection mode based on the geometric information of the detailed region and the geometric information of the panoramic image includes: Calculate the absolute value of the coordinate difference between the starting point and the ending point along the same axis; The second field of view corresponding to the detail region is determined based on the absolute value of the coordinate difference along each axis, the first field of view of the panoramic image, and the width and height of the panoramic image. Based on the second field of view of the detail region, determine the target field of view of the detail region in the linear projection mode.

[0010] According to an image display method provided by the present invention, the target attitude angle includes a target rotation angle in the rotation direction, the geometric information of the detail region includes the coordinates of the starting point and the ending point of the detail region, the geometric information of the panoramic image includes the width and height of the panoramic image, and determining the target attitude angle of the detail region in a linear projection mode based on the geometric information of the detail region and the geometric information of the panoramic image includes: The coordinates of the starting and ending points of the detail region are averaged along the horizontal axis to determine the coordinates of the third center point of the detail region along the horizontal axis. The width of the panoramic image is averaged to determine the coordinates of the first center point of the panoramic image on the horizontal axis. Based on the width and height of the panoramic image, determine the coordinates of the calibration point on the horizontal axis of the panoramic image, and determine the initial rotation angle based on the coordinates of the calibration point; Based on the coordinates of the third center point on the horizontal axis, the coordinates of the first center point on the horizontal axis, the width of the panoramic image, and the initial rotation angle, the target rotation angle corresponding to the detailed area in the linear projection mode is determined.

[0011] According to an image display method provided by the present invention, the method further includes: Obtain the object distance between the target in the detailed region and the set center point; the set center point is the imaging center point of the multi-view camera; Based on the object distance, determine the stitching parameters for stitching the initial images acquired by the multi-camera; the stitching parameters include the stitching distance between the initial images acquired by the multi-camera.

[0012] The present invention also provides an image display device, comprising the following modules: The detection module is used to perform target detection on the panoramic image and determine the detailed regions in the panoramic image; the detailed regions are the regions corresponding to the targets, and the panoramic image is the image obtained by cylindrical projection and stitching of the initial images captured by the multi-view cameras respectively. The projection center point determination module is used to perform linear projection transformation on the coordinates of the first center point based on the coordinates of the first center point corresponding to the panoramic image and the first field of view corresponding to the panoramic image, and determine the coordinates of the second center point corresponding to the first center point under the linear projection method. The attitude angle and field of view determination module is used to determine the target attitude angle of the detailed region and the target field of view of the detailed region under the linear projection method, based on the geometric information of the detailed region and the geometric information of the panoramic image, with the coordinates of the second center point as the fixed projection center. The projection transformation and display module is used to construct a transformation matrix based on the coordinates of the second center point, the target attitude angle, and the target field of view angle, and to transform the detail region through the transformation matrix to determine and display the target image corresponding to the detail region in the linear projection mode.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the image display method as described above.

[0014] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image display method as described above.

[0015] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the image display method as described above.

[0016] The image display method, apparatus, device, and storage medium provided by this invention obtain a stitched panoramic image by performing cylindrical projection and stitching on initial images acquired by multiple target cameras. Target detection is then performed on the panoramic image to determine the detail regions corresponding to the targets. Next, based on the coordinates of a first center point and its corresponding field of view in the panoramic image, a linear projection transformation is performed on the coordinates of the first center point to determine the coordinates of a second center point corresponding to the first center point under linear projection. Using the coordinates of the second center point as a fixed projection center, the target attitude angle and target field of view corresponding to the detail regions under linear projection are determined based on the geometric information of the detail regions and the geometric information of the panoramic image. Finally, a transformation matrix is ​​constructed based on the coordinates of the second center point, the target attitude angle, and the target field of view. This transformation matrix is ​​then used to transform the detail regions to determine and display the target image corresponding to the detail regions under linear projection. In this method, since the detailed areas in the multi-view stitched panoramic image can be transformed from cylindrical projection to linear projection and then displayed, the distortion problem of the final displayed detailed area image can be eliminated. At the same time, since the projection transformation is only performed on the image of this local area of ​​the detailed area, rather than on the entire panoramic image, the computational load of the device can be reduced, thus ensuring the processing performance of the device. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is one of the flowcharts illustrating the image display method provided in this embodiment of the invention.

[0019] Figure 2 This is a schematic diagram of the transformation from cylindrical projection to linear projection provided in an embodiment of the present invention.

[0020] Figure 3 This is a schematic diagram of a planar image projected onto a cylindrical surface according to an embodiment of the present invention.

[0021] Figure 4 This is a top view of the cylindrical projection provided in an embodiment of the present invention.

[0022] Figure 5 This is a schematic diagram of the similar triangle principle provided in an embodiment of the present invention.

[0023] Figure 6 This is the second flowchart illustrating the image display method provided in this embodiment of the invention.

[0024] Figure 7 This is an overall flowchart of the monitoring image processing provided in the embodiments of the present invention.

[0025] Figure 8 This is a schematic diagram of the structure of the image display device provided in an embodiment of the present invention.

[0026] Figure 9 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0028] With the rapid development of various technologies in the surveillance field, people are pursuing higher-quality visual experiences, leading to a comprehensive improvement in image clarity, from 1080P to 4K and even 8K or higher. This puts enormous pressure on image encoding and transmission. High-resolution videos are typically panoramic images or videos created by stitching together multiple cameras (or webcams). To present panoramic images, cylindrical projection is usually used, resulting in noticeable distortion at the image edges. Furthermore, in some scenarios, in addition to outputting panoramic images, attention is often paid to images of certain detailed areas. The aforementioned techniques also lead to significant distortion in these detailed areas of the final panoramic image, affecting the presentation of those areas. Therefore, this invention provides an image display method, apparatus, device, and storage medium to solve the above-mentioned technical problems.

[0029] The following is combined with Figures 1-7 This invention describes an image display method according to an embodiment of the invention.

[0030] It should be noted that the executing entity of the embodiments of the present invention can be an image display device, an electronic device, a processor in a camera, a camera itself, a central control device, or other devices or apparatus. No specific limitations are made here; the following embodiments will use an electronic device as an example. It is understood that the electronic device can be a terminal (e.g., a laptop computer, a camera, etc.), a server, etc.

[0031] Figure 1 This is one of the flowcharts illustrating the image display method provided by the present invention, such as... Figure 1 As shown, the method includes the following steps: Step 102: Perform target detection on the panoramic image to determine the detailed regions in the panoramic image; the detailed regions are the regions corresponding to the targets, and the panoramic image is the image obtained by cylindrical projection and stitching of the initial images collected by the multi-view cameras.

[0032] In this step, a multi-view camera refers to a camera with multiple cameras, or it can be a combination of multiple cameras with a single camera, or a combination of multiple cameras with both single and multiple cameras. Each camera in a multi-view camera can collect or monitor information from different locations within the monitored scene. During actual image acquisition, each camera captures an initial image of the current scene. Then, the initial image captured simultaneously by multiple cameras is cylindrically projected onto a cylinder to obtain a cylindrical image. These cylindrically projected images are then stitched together according to the camera's orientation and other information to obtain a stitched image, which is the panoramic image of the current scene.

[0033] Next, object detection processing can be performed on the panoramic image to obtain the region corresponding to the target, denoted as the detail region or detail region image. In other words, the detail region is the image of the region corresponding to the target in the panoramic image. The target here can be an object of interest in the current scene, such as a person, a vehicle, or other dynamic objects of interest. Object detection processing of the panoramic image can be performed using object detection algorithms or models. These algorithms and models can be based on deep learning, neural networks, or other methods; no specific limitations are made here.

[0034] When performing target detection processing on the panoramic image to obtain detailed regions, the corresponding geometric information of the detailed regions can also be obtained. Optionally, the geometric signal of the detailed region may include the coordinates of the starting point and the ending point of the detailed region.

[0035] Furthermore, when stitching panoramic images, the stitching can be performed according to pre-set stitching parameters, which may include predefined aspect ratios, widths and heights, and stitching distances between multiple initial images. After cylindrical projection and stitching, the geometric parameters of the panoramic image can also be obtained, including: aspect ratios, widths and heights, field of view, and attitude angles.

[0036] Step 104: Based on the coordinates of the first center point corresponding to the panoramic image and the first field of view corresponding to the panoramic image, perform linear projection transformation on the coordinates of the first center point to determine the coordinates of the second center point corresponding to the first center point under the linear projection method.

[0037] After obtaining the geometric parameters of the panoramic image, the coordinates of the center point of the panoramic image in cylindrical projection can be calculated using the width and height of the panoramic image. This center point can be denoted as the first center point, and its coordinates are calculated as follows: .

[0038] Wherein, OrgCenterX and OrgCenterY represent the coordinates of the first center point, specifically the coordinates of the first center point on the horizontal axis X and the coordinates on the vertical axis Y; Width represents the width of the panoramic image; and Heigth represents the height of the panoramic image.

[0039] In this step, both the panoramic image and its detailed areas are presented using cylindrical projection. This results in noticeable distortion at edges and other locations in both the panoramic and detailed images, affecting the final image presentation / display. Therefore, see [link to relevant documentation]. Figure 2 The diagram shown illustrates the conversion of cylindrical projection to linear projection. In this embodiment, it is proposed to convert the detailed areas from cylindrical projection to linear projection to eliminate distortion issues in the presentation / display of the detailed areas.

[0040] In the process of converting cylindrical projection to linear projection, it is necessary to first determine the projection center under linear projection. In this step, the center point of the panoramic image under cylindrical projection can be used as the projection center under cylindrical projection. This projection center can be converted to linear projection, that is, the center point and its coordinates corresponding to the first center point and its coordinates under linear projection are determined. The center point under linear projection can be recorded as the second center point, and the corresponding coordinates are the coordinates of the second center point.

[0041] For the process of converting points from cylindrical projection to linear projection, see [link to documentation]. Figure 3 The diagram shown illustrates the projection of a planar image onto a cylindrical surface. The quadrilateral GHEF represents the original image to be processed. After projection onto the cylindrical surface, it becomes the curved surface JDILCK (marked by the red dot in the diagram). See also... Figure 4 The diagram shows a top view of a cylindrical projection, where FCE is the image plane to be processed and KCL is the surface obtained from the cylindrical projection. In this embodiment, the main approach is back projection, which involves converting the KCL cylindrical projection surface to the FCE plane, i.e., converting it into a linear projection. The included angle is the field of view angle of the panoramic image under cylindrical projection, which can be denoted as the first field of view angle.

[0042] For a detailed explanation of converting cylindrical projection to linear projection, please refer to [link / reference needed]. Figure 5 The diagram illustrates the principle of similar triangles, where P and P1 are points in the plane of the image to be processed, P' and P1' are points in the panoramic image projected onto the cylindrical surface, and y and y' represent the coordinates of P and P' respectively along their vertical axes. Specifically, the principle of similar triangles can be used, combined with the first field of view of the panoramic image, to convert the coordinates of points in the panoramic image projected onto the cylindrical surface into the coordinates of points on the plane projected onto the linear surface.

[0043] For the first center point corresponding to the above panoramic image, the coordinates of the first center point can be calculated using the principle of similar triangles, combined with the coordinates of the first center point obtained above and the first field of view of the panoramic image. The point and its coordinates in the linear projection mode of the first center point can be recorded as the second center point and its coordinates.

[0044] Step 106: Using the coordinates of the second center point as the fixed projection center, determine the target attitude angle and the target field of view angle of the detailed region under the linear projection method based on the geometric information of the detailed region and the geometric information of the panoramic image.

[0045] In this step, after determining the second center point and its coordinates under the linear projection method, this second center point can be used as the fixed projection center under the linear projection method. Then, using the geometric information of the detailed region and the geometric information of the panoramic image, the target attitude angle and target field of view angle corresponding to the detailed region under the linear projection method can be calculated.

[0046] For example, the field of view of the detailed region under linear projection can be calculated using the geometric information related to the field of view from both the geometric information of the detailed region and the geometric information of the panoramic image. At the same time, the attitude angle of the detailed region under linear projection can be calculated using the geometric information related to the field of view and the information related to the center point coordinates from both the geometric information of the detailed region and the geometric information of the panoramic image.

[0047] Step 108: Construct a transformation matrix based on the coordinates of the second center point, the target attitude angle, and the target field of view angle. Then, transform the detail region using the transformation matrix to determine and display the target image corresponding to the detail region in the linear projection mode.

[0048] In this step, after obtaining the coordinates of the projection center (i.e., the coordinates of the second center point), the target attitude angle and the target field of view angle corresponding to the detailed region in the linear projection mode, the transformation matrix of the detailed region from cylindrical projection to linear projection can be directly constructed using these three parameters. Alternatively, the transformation matrix of the detailed region from cylindrical projection to linear projection can be constructed based on these three parameters and combined with other parameters. In short, the transformation matrix of the detailed region from cylindrical projection to linear projection can be constructed.

[0049] After obtaining the transformation matrix to convert the detailed region from cylindrical projection to linear projection, the coordinates of each point in the detailed region under cylindrical projection can also be obtained during target detection in the panoramic image. Then, the coordinates of each point in the detailed region under cylindrical projection can be transformed using the transformation matrix to obtain the coordinates of each point in the detailed region under linear projection. The pixel values ​​at each coordinate in the detailed region under cylindrical projection are then filled back into the corresponding coordinates under linear projection to obtain the final linear projection image of the detailed region, denoted as the target image. This target image can then be displayed. Here, the transformation matrix can be used to adjust the attitude angle and field of view of the detailed region under cylindrical projection, thus converting the cylindrical projection to linear projection.

[0050] For areas of the panoramic image other than the detailed regions, you can choose to continue displaying them using cylindrical projection. This only involves converting the projection method for the detailed regions, which requires less computation and thus avoids significant impact on device performance.

[0051] In this embodiment, a stitched panoramic image is obtained by cylindrically projecting and stitching together the initial images acquired by multiple target cameras. Target detection is then performed on the panoramic image to determine the detail regions corresponding to the targets. Based on the coordinates of the first center point and its corresponding field of view in the panoramic image, a linear projection transformation is performed on the coordinates of the first center point to determine the coordinates of the second center point under linear projection. Using the coordinates of the second center point as a fixed projection center, the target attitude angle and target field of view corresponding to the detail region under linear projection are determined based on the geometric information of the detail region and the panoramic image. A transformation matrix is ​​then constructed based on the coordinates of the second center point, the target attitude angle, and the target field of view. This transformation matrix is ​​used to transform the detail region to determine and display the target image corresponding to the detail region under linear projection. This method eliminates distortion in the final displayed detail region image because it transforms the detail region from cylindrical projection to linear projection. Furthermore, since the projection transformation is performed only on this local area of ​​the detail region, rather than the entire panoramic image, it reduces the computational load of the device and ensures its processing performance.

[0052] The following embodiments illustrate how to calculate the target attitude angle and target field of view of the detailed region under linear projection, given that the geometric information of the detailed region includes the coordinates of the starting point and the ending point of the detailed region, and the geometric information of the panoramic image includes the width and height of the panoramic image and the first field of view of the panoramic image.

[0053] In some embodiments, step 106 above, "determining the target pose angle of the detailed region in the linear projection mode based on the geometric information of the detailed region and the geometric information of the panoramic image," may include the following steps: The coordinates of the starting and ending points of the detail region along the same axis are averaged to determine the coordinates of the geometric center corresponding to the detail region, and the coordinates of the geometric center are used as the coordinates of the third center point. Based on the coordinates of the third center point of the detail region, the first field of view of the panoramic image, and the width and height of the panoramic image, the target attitude angle of the detail region under the linear projection method is determined.

[0054] Assuming the coordinates of the starting and ending points of the detail region are denoted as (startX, startY) and (endX, endY) respectively, the coordinates of the geometric center of the detail region can be calculated using the following formula. This geometric center (i.e., the center point of the geometric figure) can be denoted as the third center point, and its coordinates can be recorded as the coordinates of the third center point: .

[0055] Wherein, ROICenterX and ROICenterY represent the coordinates of the third center point, specifically the coordinates of the third center point on the horizontal axis X and the coordinates on the vertical axis Y.

[0056] After obtaining the coordinates of the center point of the detailed region, as an optional embodiment, the coordinates of the first center point of the panoramic image can be determined based on the width and height of the panoramic image. Specifically, the formula for calculating the coordinates of the first center point in step 104 above can be used. Then, based on the coordinates of the third center point, the coordinates of the first center point, the first field of view of the panoramic image, and the width and height of the panoramic image, the target attitude angle of the detailed region under linear projection can be determined. Specifically, the difference between the third center point and the first center point along the same axis can be calculated, and then compared with the width or height of the panoramic image along the corresponding axis. This ratio is then multiplied by the first field of view of the panoramic image along the corresponding axis to obtain the attitude angle of the detailed region along the corresponding axis.

[0057] Assuming the width and height of the panoramic image are denoted as Width and Height respectively, the first field of view of the panoramic image is denoted as (FovX, FovY), and the target pose angles of the detail region are denoted as (ROIpitch, ROIyaw, ROIroll), where pitch refers to the pose adjustment in the vertical / axial direction, yaw refers to the pose adjustment in the horizontal / axial direction, and roll refers to the adjustment in the rotational direction, then the ROIpitch and ROIyaw in the target pose angles of the detail region can be calculated in the following way: .

[0058] Here, ROIpitch and ROIyaw represent the attitude angles of the detailed region along the horizontal and vertical axes, respectively, in linear projection. For the attitude angle ROIroll in the rotation direction, no adjustment is needed; the attitude angle of the detailed region in the rotation direction under cylindrical projection can be used directly. Alternatively, the attitude angle in the rotation direction can be optimized; no specific limitations are specified here.

[0059] In summary, by calculating the attitude angle of the detailed region under linear projection in the above manner, and using the attitude angles on the horizontal and vertical axes to transform the coordinates of the detailed region under cylindrical projection (such as the coordinates of the third center point), the center point of the cylindrical projection can be mapped to the center of the linear projection after horizontal and vertical adjustment, thus realizing the conversion of the projection mode.

[0060] In this embodiment, the center point of the detail region is calculated using the start and end points of the detail region. Combined with information such as the width, height, and field of view of the panoramic image, the target attitude angle of the detail region under linear projection is calculated. This maps the center point of the cylindrical projection to the center of the linear projection, achieving a conversion of the projection method and thus improving the presentation effect of the detail region image. Alternatively, the center point of the panoramic image can be calculated first using its width and height, and then the target attitude angle of the detail region under linear projection can be calculated based on this. This refinement process improves the accuracy and efficiency of the projection method conversion, thereby enhancing the accuracy of the detail region image presentation.

[0061] Based on the geometric information of the detailed region and the geometric information of the panoramic image in the above embodiments, the target field of view corresponding to the detailed region in the linear projection mode can be calculated. The following embodiments illustrate this process.

[0062] In some embodiments, step 106 above, "determining the target field of view corresponding to the detailed region in the linear projection mode based on the geometric information of the detailed region and the geometric information of the panoramic image," may include the following steps: Calculate the absolute value of the coordinate difference between the starting point and the ending point along the same axis; determine the second field of view corresponding to the detail region based on the absolute value of the coordinate difference along each axis, the first field of view of the panoramic image, and the width and height of the panoramic image; determine the target field of view corresponding to the detail region in the linear projection mode based on the second field of view of the detail region.

[0063] In this context, assuming the width and height of the panoramic image are denoted as Width and Height respectively, the first field of view of the panoramic image is denoted as (FovX, FovY), and the coordinates of the starting and ending points of the detail region are denoted as (startX, startY) and (endX, endY) respectively, then the second field of view of the detail region can be calculated using the following formula: .

[0064] Wherein, ROIFovX and ROIFovY represent the second field of view, which can be the field of view of the detailed region on the horizontal axis X and the field of view on the vertical axis Y.

[0065] It is understandable that the second field of view of the detailed area here can be the field of view of the detailed area under the cylindrical projection, through the above... Figure 4 From the top view of the cylindrical projection, it can be deduced that the field of view of the detailed area is the same under both cylindrical and linear projection. Therefore, the second field of view of the detailed area under the cylindrical projection can be directly taken as the target field of view of the detailed area under the linear projection.

[0066] In this embodiment, the center point of the detail region is calculated by using the starting and ending points of the detail region, and the target field of view of the detail region under linear projection is calculated by combining the width, height and field of view of the panoramic image. In this way, the image of the detail region under cylindrical projection can be adjusted by using the target field of view, thereby improving the presentation effect of the detail region image.

[0067] The following examples illustrate the process of optimizing the attitude angle in the rotation direction of the target attitude angle.

[0068] Figure 6 This is a second schematic flowchart of the image display method provided by the present invention. The aforementioned target attitude angle includes the target rotation angle of the rotation direction. The aforementioned geometric information of the detail region includes the coordinates of the starting point and the ending point of the detail region. The aforementioned geometric information of the panoramic image includes the width and height of the panoramic image, such as... Figure 6 As shown, step 106 above, "determining the target pose angle of the detailed region in the linear projection mode based on the geometric information of the detailed region and the geometric information of the panoramic image," may include the following steps: Step 202: Average the coordinates of the starting and ending points of the detail region along the horizontal axis to determine the coordinates of the third center point of the detail region along the horizontal axis.

[0069] Here, assuming the coordinates of the starting point and the ending point of the detail region are denoted as (startX, startY) and (endX, endY) respectively, the coordinates of the third center point of the detail region on the horizontal axis X, ROICenterX, can be calculated using the following formula: .

[0070] Step 204: Average the width of the panoramic image to determine the coordinates of the first center point of the panoramic image on the horizontal axis.

[0071] In this step, assuming the width of the panoramic image is denoted as Width, the coordinates OrgCenterX of the first center point of the panoramic image on the horizontal axis X can be calculated using the following formula: .

[0072] Step 206: Determine the coordinates of the calibration point of the panoramic image on the horizontal axis based on the width and height of the panoramic image, and determine the initial rotation angle based on the coordinates of the calibration point.

[0073] In this step, we mainly focus on the transformation of the rotation direction along the horizontal axis. Therefore, we can first determine the two endpoints and their coordinates on the horizontal axis (e.g., the left and right direction) of the panoramic image based on the coordinates of each point in the panoramic image. Then, we can calculate the coordinates of the calibration points corresponding to these two points using the following formula: (Width / a, Height / b), where a and b are parameters set based on experience, such as a being 10 and b being 2.

[0074] Then, using the above calibration point as the center point and combining the coordinates of the two endpoints, the initial rotation angle can be calibrated and denoted as rollA.

[0075] Step 208: Determine the target rotation angle of the detailed area in the linear projection mode based on the coordinates of the third center point on the horizontal axis, the coordinates of the first center point on the horizontal axis, the width of the panoramic image, and the initial rotation angle.

[0076] In this step, after obtaining the coordinates of the third center point on the horizontal axis, the coordinates of the first center point on the horizontal axis, the width of the panoramic image, and the initial rotation angle, assuming the rotation angle of the panoramic image center point is 0, the target rotation angle ROIroll of the detail region under linear projection can be calculated using the following formula: .

[0077] Where c is a parameter set based on experience, such as c could be 0.4, 0.5, etc.; rollA is the initial rotation angle.

[0078] In this embodiment, the target rotation angle of the detail region is determined by combining the points marked in the panoramic image and the initial rotation angle marked, along with the center point coordinates of the detail region and the panoramic image on the horizontal axis. This method optimizes the rotation angle of the detail region, making the image of the detail region presented in the final linear projection more consistent with the actual situation, i.e., the displayed image is more realistic.

[0079] The following examples illustrate the process of optimizing the stitching parameters of multi-view cameras.

[0080] In some embodiments, the above method may further include the following steps: Obtain the object distance between the target in the detailed region and the set center point; the set center point is the imaging center point of the multi-view camera; based on the object distance, determine the stitching parameters for stitching the initial images acquired by the multi-view camera; the stitching parameters include the stitching distance between the initial images acquired by the multi-view camera.

[0081] In this multi-view camera imaging process, the center point is known. By acquiring the coordinates of the target in the actual scene within the detail region and then calculating the distance between them and the imaging center point, the object distance between the target in the detail region and the imaging center point can be obtained. This object distance can then be used to adjust the stitching distance—a parameter between the initial images acquired by the multi-view camera—for example, increasing the stitching distance if it's too large. Of course, other stitching parameters of the multi-view camera can also be adjusted using the object distance.

[0082] It is understandable that, during the process of the multi-view camera continuously acquiring initial images, the steps of the above embodiments can be repeatedly executed to adaptively adjust the observation area corresponding to the detail area, that is, to obtain the target image corresponding to the detail area at different times. Then, the stitching distance of multiple initial images can be dynamically adjusted continuously through the above-mentioned object distance adjustment stitching distance method to optimize the stitching seam of multiple initial images, thereby improving the presentation effect of the final obtained image.

[0083] In this embodiment, the stitching distance between multiple initial images of a multi-view camera can be adjusted by the object distance between the target in the detail region and the imaging center point. This optimizes the stitching seams between multiple initial images, so that the final detail region image has no obvious stitching abrupt changes, thus improving the overall presentation effect of the detail region image.

[0084] The following provides a complete implementation example of surveillance image processing for overall description. See also: Figure 7 The flowchart shown illustrates the overall process of surveillance image processing. Taking four images captured by four cameras as an example, the overall process of surveillance image processing can include the following steps: First, two stitching paths are created, each with its own path. The first path can cache the first frame image captured by the four cameras, for example, by caching it in a queue (i.e., a starting frame cache queue). Subsequently, all four real-time captured image frames can be cached. The second path performs object detection on the stitched panoramic image and sends the detected data back to the first path (i.e., real-time intelligent detection, and sends detection parameters, which can be geometric parameters of detailed regions in the panoramic image, or other geometric parameters of the panoramic image itself).

[0085] After receiving the detection parameters from the second path, the first path can match the frame number of the panoramic image corresponding to the detection parameters with the frame number of its own cached image. If the frame numbers do not match (e.g., the frame number is out of sequence), the corresponding four image frames are released directly. If the frame numbers match, the four image frames are stitched together, and detail channel encoding is performed simultaneously (i.e., transforming the detail region from cylindrical projection to central projection in the above embodiment) to obtain the target image corresponding to the detail region in linear projection. Then, the panoramic image and the target image of the detail region at the same time can be output synchronously, and the process is closed. This allows for the output of the target image of the detail region with better presentation while simultaneously outputting the panoramic image.

[0086] The image display device provided by the present invention will be described below. The image display device described below can be referred to in correspondence with the image display method described above.

[0087] Figure 8 This is a schematic diagram of the image display device provided by the present invention. See also: Figure 8 As shown, the device may include: The detection module 310 is used to perform target detection on the panoramic image and determine the detailed regions in the panoramic image; the detailed regions are the regions corresponding to the targets, and the panoramic image is an image obtained by cylindrical projection and stitching of the initial images collected by the multi-view cameras respectively. The projection center point determination module 320 is used to perform linear projection transformation on the coordinates of the first center point based on the coordinates of the first center point corresponding to the panoramic image and the first field of view corresponding to the panoramic image, and determine the coordinates of the second center point corresponding to the first center point under the linear projection method. The attitude angle and field of view determination module 330 is used to determine the target attitude angle of the detailed area and the target field of view of the detailed area under the linear projection method, based on the geometric information of the detailed area and the geometric information of the panoramic image, with the coordinates of the second center point as the fixed projection center. The projection transformation and display module 340 is used to construct a transformation matrix based on the coordinates of the second center point, the target attitude angle, and the target field of view angle, and to transform the detail area through the transformation matrix to determine and display the target image corresponding to the detail area in the linear projection mode.

[0088] In some embodiments, the geometric information of the detailed region includes the coordinates of the starting point and the ending point of the detailed region, the geometric information of the panoramic image includes the width and height of the panoramic image, and the first field of view of the panoramic image. The attitude angle and field of view determination module 330 includes: The attitude angle determination unit is used to average the coordinates of the starting point and ending point of the detail region along the same axis to determine the coordinates of the geometric center corresponding to the detail region, and use the coordinates of the geometric center as the coordinates of the third center point; based on the coordinates of the third center point of the detail region, the first field of view of the panoramic image, and the width and height of the panoramic image, the target attitude angle of the detail region under the linear projection method is determined.

[0089] Optionally, the above-mentioned attitude angle determination unit is specifically used to determine the coordinates of the first center point corresponding to the panoramic image based on the width and height of the panoramic image; and to determine the target attitude angle of the detail area in the linear projection mode based on the coordinates of the third center point, the coordinates of the first center point, the first field of view of the panoramic image, and the width and height of the panoramic image.

[0090] In some embodiments, the geometric information of the detailed region includes the coordinates of the starting point and the ending point of the detailed region, the geometric information of the panoramic image includes the width and height of the panoramic image, and the first field of view of the panoramic image. The attitude angle and field of view determination module 330 includes: The field of view determination unit is used to calculate the absolute value of the coordinate difference between the starting point and the ending point on the same axis; based on the absolute value of the coordinate difference on each axis, the first field of view of the panoramic image, the width and height of the panoramic image, the second field of view corresponding to the detail region is determined; based on the second field of view of the detail region, the target field of view corresponding to the detail region in the linear projection mode is determined.

[0091] In some embodiments, the target attitude angle includes a target rotation angle in the rotation direction, the geometric information of the detail region includes the coordinates of the starting point and the ending point of the detail region, the geometric information of the panoramic image includes the width and height of the panoramic image, and the attitude angle and field of view determination module 330 is specifically used to perform average processing on the coordinate values ​​of the starting point and the ending point of the detail region in the horizontal axis to determine the coordinates of the third center point of the detail region in the horizontal axis; perform average processing on the width of the panoramic image to determine the coordinates of the first center point of the panoramic image in the horizontal axis; determine the coordinates of the calibration point of the panoramic image in the horizontal axis according to the width and height of the panoramic image, and determine the initial rotation angle according to the coordinates of the calibration point; and determine the target rotation angle corresponding to the detail region in the linear projection mode according to the coordinates of the third center point in the horizontal axis, the coordinates of the first center point in the horizontal axis, the width of the panoramic image, and the initial rotation angle.

[0092] In some embodiments, the above-described apparatus further includes: The stitching optimization module is used to obtain the object distance between the target in the detail area and the set center point; the set center point is the imaging center point of the multi-view camera; based on the object distance, the stitching parameters for stitching the initial images acquired by the multi-view camera are determined; the stitching parameters include the stitching distance between the initial images acquired by the multi-view camera.

[0093] It should be noted that the apparatus provided in this embodiment of the invention can implement all the method steps implemented in the above method embodiment and can achieve the same technical effect. Therefore, the parts and beneficial effects that are the same as those in the method embodiment will not be described in detail here.

[0094] Figure 9 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 9 As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communications bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other through the communications bus 440. The processor 410 can call logical instructions in the memory 430 to execute an image display method, which includes: performing target detection on a panoramic image to determine a detail region in the panoramic image; the detail region is the target corresponding region, and the panoramic image is an image obtained by cylindrical projection and stitching together initial images acquired by multiple cameras; performing linear projection transformation on the coordinates of the first center point according to the coordinates of the first center point and the first field of view of the panoramic image to determine the coordinates of the second center point corresponding to the first center point under linear projection; using the coordinates of the second center point as a fixed projection center, determining the target attitude angle and the target field of view of the detail region under linear projection according to the geometric information of the detail region and the geometric information of the panoramic image; constructing a transformation matrix according to the coordinates of the second center point, the target attitude angle, and the target field of view, and transforming the detail region through the transformation matrix to determine and display the target image corresponding to the detail region under linear projection.

[0095] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0096] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the image display method provided by the above methods. The method includes: performing target detection on a panoramic image to determine a detail region in the panoramic image; the detail region is a target corresponding region, and the panoramic image is an image obtained by cylindrical projection and stitching together initial images acquired by multiple cameras; performing linear projection transformation on the coordinates of the first center point according to the coordinates of the first center point corresponding to the panoramic image and the first field of view corresponding to the panoramic image to determine the coordinates of the second center point corresponding to the first center point under the linear projection method; using the coordinates of the second center point as a fixed projection center, determining the target attitude angle and the target field of view corresponding to the detail region under the linear projection method according to the geometric information of the detail region and the geometric information of the panoramic image; constructing a transformation matrix according to the coordinates of the second center point, the target attitude angle, and the target field of view, and transforming the detail region through the transformation matrix to determine and display the target image corresponding to the detail region under the linear projection method.

[0097] In another aspect, the present invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the image display method provided by the above methods. The method includes: performing target detection on a panoramic image to determine a detail region in the panoramic image; the detail region being a target-corresponding region; the panoramic image being an image obtained by cylindrical projection and stitching together initial images acquired by multiple cameras; performing a linear projection transformation on the coordinates of the first center point according to the coordinates of the first center point and the first field of view corresponding to the panoramic image to determine the coordinates of a second center point corresponding to the first center point under the linear projection method; using the coordinates of the second center point as a fixed projection center, determining the target attitude angle and the target field of view corresponding to the detail region under the linear projection method based on the geometric information of the detail region and the geometric information of the panoramic image; constructing a transformation matrix based on the coordinates of the second center point, the target attitude angle, and the target field of view, and transforming the detail region using the transformation matrix to determine and display the target image corresponding to the detail region under the linear projection method.

[0098] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0099] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image display method, characterized in that, include: Target detection is performed on the panoramic image to determine the detailed regions in the panoramic image; the detailed regions are the regions corresponding to the targets; the panoramic image is an image obtained by cylindrical projection and stitching together the initial images acquired by the multi-view cameras respectively. Based on the coordinates of the first center point corresponding to the panoramic image and the first field of view corresponding to the panoramic image, the coordinates of the first center point are transformed by linear projection to determine the coordinates of the second center point corresponding to the first center point under the linear projection method. Using the coordinates of the second center point as a fixed projection center, the target attitude angle and the target field of view angle of the detailed region under the linear projection method are determined based on the geometric information of the detailed region and the geometric information of the panoramic image. A transformation matrix is ​​constructed based on the coordinates of the second center point, the target attitude angle, and the target field of view angle. The transformation matrix is ​​then used to transform the detail region to determine and display the target image corresponding to the detail region in the linear projection mode.

2. The image display method according to claim 1, characterized in that, The geometric information of the detailed region includes the coordinates of the starting point and the ending point of the detailed region. The geometric information of the panoramic image includes the width and height of the panoramic image and the first field of view of the panoramic image. Determining the target attitude angle of the detailed region in linear projection based on the geometric information of the detailed region and the geometric information of the panoramic image includes: The coordinates of the starting point and ending point of the detailed region along the same axis are averaged to determine the coordinates of the geometric center corresponding to the detailed region, and the coordinates of the geometric center are used as the coordinates of the third center point. Based on the coordinates of the third center point of the detailed region, the first field of view of the panoramic image, and the width and height of the panoramic image, the target attitude angle corresponding to the detailed region in the linear projection mode is determined.

3. The image display method according to claim 2, characterized in that, Determining the target pose angle of the detailed region in linear projection based on the coordinates of the third center point of the detailed region, the first field of view of the panoramic image, and the width and height of the panoramic image includes: Based on the width and height of the panoramic image, determine the coordinates of the first center point corresponding to the panoramic image; Based on the coordinates of the third center point, the coordinates of the first center point, the first field of view of the panoramic image, and the width and height of the panoramic image, the target attitude angle corresponding to the detailed region in the linear projection mode is determined.

4. The image display method according to claim 1, characterized in that, The geometric information of the detailed region includes the coordinates of the starting point and the ending point of the detailed region. The geometric information of the panoramic image includes the width and height of the panoramic image and the first field of view of the panoramic image. Determining the target field of view corresponding to the detailed region in the linear projection mode based on the geometric information of the detailed region and the geometric information of the panoramic image includes: Calculate the absolute value of the coordinate difference between the starting point and the ending point along the same axis; The second field of view corresponding to the detail region is determined based on the absolute value of the coordinate difference along each axis, the first field of view of the panoramic image, and the width and height of the panoramic image. Based on the second field of view of the detailed region, the target field of view corresponding to the detailed region in the linear projection mode is determined.

5. The image display method according to claim 1, characterized in that, The target attitude angle includes the target rotation angle in the rotation direction; the geometric information of the detail region includes the coordinates of the starting point and the ending point of the detail region; the geometric information of the panoramic image includes the width and height of the panoramic image; determining the target attitude angle of the detail region in linear projection based on the geometric information of the detail region and the geometric information of the panoramic image includes: The coordinates of the starting and ending points of the detailed region are averaged along the horizontal axis to determine the coordinates of the third center point of the detailed region along the horizontal axis. The width of the panoramic image is averaged to determine the coordinates of the first center point of the panoramic image along the horizontal axis. Based on the width and height of the panoramic image, determine the coordinates of the calibration point of the panoramic image on the horizontal axis, and determine the initial rotation angle based on the coordinates of the calibration point; Based on the coordinates of the third center point on the horizontal axis, the coordinates of the first center point on the horizontal axis, the width of the panoramic image, and the initial rotation angle, the target rotation angle corresponding to the detailed region in the linear projection mode is determined.

6. The image display method according to any one of claims 1 to 5, characterized in that, The method further includes: Obtain the object distance between the target in the detailed region and a set center point; the set center point is the imaging center point of the multi-view camera; Based on the object distance, stitching parameters are determined for stitching the initial images acquired by the multi-view camera; the stitching parameters include the stitching distance between the initial images acquired by the multi-view camera.

7. An image display device, characterized in that, include: The detection module is used to perform target detection on the panoramic image and determine the detail region in the panoramic image; the detail region is the region corresponding to the target, and the panoramic image is an image obtained by cylindrical projection and stitching of the initial images captured by the multi-view camera respectively; The projection center point determination module is used to perform linear projection transformation on the coordinates of the first center point based on the coordinates of the first center point corresponding to the panoramic image and the first field of view corresponding to the panoramic image, and determine the coordinates of the second center point corresponding to the first center point under the linear projection mode. The attitude angle and field of view determination module is used to determine the target attitude angle of the detailed region and the target field of view of the detailed region under the linear projection method, based on the geometric information of the detailed region and the geometric information of the panoramic image, with the coordinates of the second center point as the fixed projection center. The projection transformation and display module is used to construct a transformation matrix based on the coordinates of the second center point, the target attitude angle, and the target field of view angle, and to transform the detail region through the transformation matrix to determine and display the target image corresponding to the detail region in the linear projection mode.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the image display method as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the image display method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the image display method as described in any one of claims 1 to 6.

Images