A holographic video generation method and device, electronic equipment and storage medium

Abstract

The application discloses a holographic video generation method and device, an electronic device and a computer readable storage medium. The method comprises the following steps: acquiring an ultrasonic image; determining the corresponding relationship between each projection surface of a holographic projection device and each projection area in a holographic projection plane area; determining the projection image corresponding to each projection surface based on the ultrasonic image; placing the projection image corresponding to each projection surface in the corresponding projection area of the holographic projection plane area according to the corresponding relationship, so as to splice a holographic projection image; and generating a holographic video based on the holographic projection image. The application realizes the generation of a holographic video corresponding to an ultrasonic image in an ultrasonic device.

Description

Technical Field

[0001] This application relates to the field of ultrasound technology, and more specifically, to a holographic video generation method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] After an ultrasound examination, the doctor will provide the patient with an examination report, including two-dimensional or three-dimensional images and diagnostic text. After being printed out, the images on the report often do not display the original image information and can only show the image at a certain moment, failing to capture valuable images of the examination process.

[0003] Holographic imaging technology is an imaging technique that emerged with the development of computer media technology. It can realistically display a three-dimensional image of an observed object, creating a 3D effect where the object appears to float inside the device. Currently, however, holographic imaging technology is too specialized and not very practical, making it unsuitable for generating holographic videos from ultrasound images.

[0004] Therefore, how to generate holographic videos corresponding to ultrasound images is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0005] The purpose of this application is to provide a holographic video generation method, apparatus, electronic device, and computer-readable storage medium, which realizes the generation of holographic videos corresponding to ultrasound images in ultrasound equipment.

[0006] To achieve the above objectives, this application provides a holographic video generation method, comprising: acquiring an ultrasound image; determining the correspondence between each projection surface of a holographic projection device and each projection region in a holographic projection plane region; determining a projection image corresponding to each projection surface based on the ultrasound image; placing the projection image corresponding to each projection surface into the projection region corresponding to the holographic projection plane region according to the correspondence, so as to stitch together a holographic projection image; and generating a holographic video based on the holographic projection image.

[0007] The step of determining the correspondence between each projection surface of the holographic projection device and each projection area in the holographic projection plane region includes: calculating the position coordinates of each projection area in the holographic projection plane region based on the resolution of the holographic projection device; and determining the correspondence between each projection surface of the holographic projection device and the position coordinates of each projection area in the holographic projection plane region.

[0008] The step of calculating the position coordinates of each projection area in the holographic projection plane area based on the resolution of the holographic projection device includes: calculating the screen scaling factor based on the resolution of the holographic projection device or the resolution of the image in the ultrasound image; calculating the size of the projection screen based on the resolution of the holographic projection device and the screen scaling factor; and calculating the position coordinates of each projection area in the holographic projection plane area based on the resolution of the holographic projection device and the size of the projection screen.

[0009] The holographic projection device includes a first projection surface and at least one second projection surface. If the ultrasound image includes at least one two-dimensional ultrasound image, then determining the projection image corresponding to each projection surface based on the ultrasound image includes: determining the two-dimensional ultrasound image in the ultrasound image as the projection image corresponding to the first projection surface; determining a rotation matrix around the center point of the two-dimensional ultrasound image based on the correspondence; and performing rotation processing on the projection image corresponding to the first projection surface based on the rotation matrix to generate projection images corresponding to each second projection surface.

[0010] The acquisition of ultrasound images includes: acquiring three-dimensional ultrasound volume data, and performing three-dimensional volume rendering on the three-dimensional ultrasound volume data to generate an ultrasound image containing the rendered image; correspondingly, the determination of the projection image corresponding to each projection surface based on the ultrasound image includes: determining the projection image corresponding to each projection surface based on the rendered image.

[0011] The step of acquiring three-dimensional ultrasound volume data and performing three-dimensional volume rendering on the three-dimensional ultrasound volume data to generate an ultrasound image containing the rendered image includes: acquiring a single three-dimensional ultrasound volume data, determining an initial rotation angle and rotation step; and performing three-dimensional volume rendering on the three-dimensional ultrasound volume data based on the initial rotation angle and the rotation step to generate an ultrasound image containing the rendered image.

[0012] The determination of the initial rotation angle and rotation step includes: determining the same initial rotation angle and rotation step for all the projection surfaces; correspondingly, the projection surfaces of the holographic projection device include a first projection surface and at least one second projection surface; the determination of the projection image corresponding to each projection surface based on the rendered image includes: determining the rendered image as the projection image corresponding to the first projection surface; determining a rotation matrix around the center point of the rendered image based on the correspondence; and rotating the projection image corresponding to the first projection surface based on the rotation matrix to generate projection images corresponding to each of the second projection surfaces.

[0013] The determination of the initial rotation angle and rotation step includes: determining different initial rotation angles corresponding to each of the projection surfaces and the same rotation step for all the projection surfaces; correspondingly, the three-dimensional volume rendering of the three-dimensional ultrasound volume data based on the initial rotation angle and rotation step to generate an ultrasound video containing multiple rendered images includes: performing three-dimensional volume rendering of the three-dimensional ultrasound volume data based on the initial rotation angle and rotation step for each of the projection surfaces to generate an ultrasound image containing multiple rendered images for each of the projection surfaces; correspondingly, the projection surface of the holographic projection device includes a first projection surface and at least one second projection surface; the determination of the projection image corresponding to each of the projection surfaces based on the rendered images includes: determining the rendered image in the ultrasound image corresponding to the first projection surface as the projection image corresponding to the first projection surface; determining a rotation matrix based on the correspondence to rotate around the center point of the rendered image in the ultrasound image corresponding to each of the second projection surfaces; and rotating the rendered image in the ultrasound image corresponding to each of the second projection surfaces based on the rotation matrix to generate the projection image corresponding to each of the second projection surfaces.

[0014] The step of acquiring three-dimensional ultrasound volume data and performing three-dimensional volume rendering on the three-dimensional ultrasound volume data to generate an ultrasound image containing the rendered image includes: acquiring multiple three-dimensional ultrasound volume data and determining a target rotation angle; performing three-dimensional volume rendering on the multiple three-dimensional ultrasound volume data based on the target rotation angle to generate an ultrasound image containing the rendered image corresponding to each of the three-dimensional ultrasound volume data.

[0015] The determination of the target rotation angle includes: determining the same target rotation angle corresponding to all the projection surfaces; correspondingly, the projection surfaces of the holographic projection device include a first projection surface and at least one second projection surface; the determination of the projection image corresponding to each projection surface based on the rendered image includes: determining the rendered image as the projection image corresponding to the first projection surface; determining a rotation matrix around the center point of the rendered image based on the correspondence; and rotating the projection image corresponding to the first projection surface based on the rotation matrix to generate projection images corresponding to each second projection surface.

[0016] The determination of the target rotation angle includes: determining different target rotation angles corresponding to each of the projection surfaces; correspondingly, the step of performing three-dimensional volume rendering on multiple three-dimensional ultrasound volume data based on the target rotation angle to generate an ultrasound image containing a rendered image corresponding to each of the three-dimensional ultrasound volume data includes: performing three-dimensional volume rendering on multiple three-dimensional ultrasound volume data based on the target rotation angle corresponding to each of the projection surfaces to generate an ultrasound image containing multiple rendered images corresponding to each of the projection surfaces; correspondingly, the projection surface of the holographic projection device includes a first projection surface and at least one second projection surface; the step of determining the projection image corresponding to each of the projection surfaces based on the rendered image includes: determining the rendered image in the ultrasound image corresponding to the first projection surface as the projection image corresponding to the first projection surface; determining a rotation matrix around the center point of the rendered image in the ultrasound image corresponding to each of the second projection surfaces based on the correspondence; and performing rotation processing on the rendered image in the ultrasound image corresponding to each of the second projection surfaces based on the rotation matrix to generate a projection image corresponding to each of the second projection surfaces.

[0017] The process of performing three-dimensional volume rendering on the three-dimensional ultrasound volume data to generate an ultrasound image containing the rendered image includes: constructing a transformation matrix; performing three-dimensional volume rendering on the three-dimensional ultrasound volume data based on the transformation matrix to generate an ultrasound image containing the rendered image; and preprocessing the rendered image in the ultrasound image to adapt it to the size of the projection screen of the holographic projection device.

[0018] The construction of the transformation matrix includes: constructing a transformation matrix based on the display height of the three-dimensional ultrasound volume data; correspondingly, preprocessing the rendered image in the ultrasound image to adapt to the size of the projection screen of the holographic projection device includes: calculating an image scaling factor based on the size of the three-dimensional ultrasound volume data and the size of the projection screen; scaling the rendered image based on the image scaling factor to adapt to the size of the projection screen of the holographic projection device.

[0019] The construction of the transformation matrix includes: calculating the target rendering image size based on the resolution of the holographic projection device and the display height and display width of the three-dimensional ultrasound volume data, and constructing a transformation matrix based on the target rendering image size; correspondingly, preprocessing the rendered image in the ultrasound image to adapt to the size of the projection screen of the holographic projection device includes: cropping the rendered image in the ultrasound image to adapt to the size of the projection screen of the holographic projection device.

[0020] The process of acquiring the ultrasound image further includes: acquiring the video playback configuration of the holographic projection device; and when the video playback configuration is set to inverted playback, flipping the image in the ultrasound image vertically.

[0021] To achieve the above objectives, this application provides a holographic video generation device, comprising: a first acquisition module for acquiring ultrasound images; a first determination module for determining the correspondence between each projection surface of a holographic projection device and each projection area in a holographic projection plane region; a second determination module for determining a projection image corresponding to each projection surface based on the ultrasound images; a stitching module for placing the projection image corresponding to each projection surface into the projection area corresponding to the holographic projection plane region according to the correspondence, so as to stitch together a holographic projection image; and a generation module for generating a holographic video based on the holographic projection image.

[0022] To achieve the above objectives, this application provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of the holographic video generation method described above.

[0023] To achieve the above objectives, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the holographic video generation method described above.

[0024] The holographic video generation method provided in this application involves an ultrasound device generating projected images corresponding to each projection surface based on images in an ultrasound image. According to the correspondence between each projection surface and each projection area in the holographic projection plane region, the projected images corresponding to each projection surface are placed in the holographic projection plane region to obtain a holographic projection image. This holographic projection image is then used to generate a holographic video corresponding to the ultrasound video, which can be played in a holographic projection device. Therefore, this application achieves the generation of holographic videos corresponding to ultrasound images in an ultrasound device. This application also discloses a holographic video generation apparatus, an electronic device, and a computer-readable storage medium, which can achieve the same technical effects.

[0025] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this application. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. The drawings are used to provide a further understanding of this disclosure and constitute a part of the specification. They are used together with the following detailed description to explain this disclosure, but do not constitute a limitation of this disclosure. In the drawings:

[0027] Figure 1 This is a flowchart illustrating a first holographic video generation method according to an exemplary embodiment;

[0028] Figure 2 This is a schematic diagram illustrating a holographic projection image corresponding to a 270° holographic video and a holographic projection device according to an exemplary embodiment;

[0029] Figure 3 This is a schematic diagram illustrating a holographic projection image corresponding to a 360° holographic video and a holographic projection device according to an exemplary embodiment;

[0030] Figure 4 This is a schematic diagram illustrating the export of holographic video from an ultrasound device according to an exemplary embodiment;

[0031] Figure 5 This is a schematic diagram illustrating the export of holographic video using another ultrasound device according to an exemplary embodiment;

[0032] Figure 6 This is a schematic diagram illustrating a method for generating a video file when a 270° holographic video is output to multiple holographic projection devices according to an exemplary embodiment.

[0033] Figure 7 This is a schematic diagram illustrating a method for generating video files when outputting 360° holographic video to multiple holographic projection devices according to an exemplary embodiment;

[0034] Figure 8 This is a flowchart illustrating a second holographic video generation method according to an exemplary embodiment;

[0035] Figure 9 This is a flowchart illustrating a method for generating a holographic video file corresponding to a two-dimensional ultrasound video, according to an exemplary embodiment.

[0036] Figure 10 This is a flowchart illustrating a third holographic video generation method according to an exemplary embodiment;

[0037] Figure 11 This is a flowchart illustrating a method for generating a holographic video file corresponding to a single three-dimensional ultrasound volume data, according to an exemplary embodiment.

[0038] Figure 12 This is a flowchart illustrating a three-dimensional volume rendering according to an exemplary embodiment;

[0039] Figure 13 A flowchart illustrating another form of three-dimensional volume rendering according to an exemplary embodiment;

[0040] Figure 14This is a schematic diagram illustrating a holographic projection image corresponding to a 270° holographic video of a fetus according to an exemplary embodiment;

[0041] Figure 15 This is a schematic diagram illustrating a holographic projection image corresponding to a 360° holographic video of the fallopian tube according to an exemplary embodiment;

[0042] Figure 16 This is a schematic diagram illustrating a holographic projection image corresponding to a 360° holographic video of an early-pregnancy fetus according to an exemplary embodiment;

[0043] Figure 17 This is a rendering of a 360° holographic video according to an exemplary embodiment.

[0044] Figure 18 This is a practical rendering of another 360° holographic video according to an exemplary embodiment;

[0045] Figure 19 This is a flowchart illustrating a fourth holographic video generation method according to an exemplary embodiment;

[0046] Figure 20 This is a flowchart illustrating another method for generating a holographic video file corresponding to a single three-dimensional ultrasound volume data, according to an exemplary embodiment.

[0047] Figure 21 This is a schematic diagram of a holographic projection image corresponding to another 270° holographic video of a fetus, according to an exemplary embodiment.

[0048] Figure 22 This is a flowchart illustrating a fifth holographic video generation method according to an exemplary embodiment;

[0049] Figure 23 This is a flowchart illustrating a method for generating a holographic video file corresponding to a three-dimensional ultrasonic volumetric video, according to an exemplary embodiment.

[0050] Figure 24 This is a flowchart illustrating a sixth holographic video generation method according to an exemplary embodiment;

[0051] Figure 25 This is a flowchart illustrating another method for generating a holographic video file corresponding to a three-dimensional ultrasonic volumetric video, according to an exemplary embodiment.

[0052] Figure 26 This is a structural diagram of a holographic video generation apparatus according to an exemplary embodiment;

[0053] Figure 27 This is a structural diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0054] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Furthermore, in the embodiments of this application, "first," "second," etc., are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0055] This application discloses a method for generating holographic videos, in which the executing entity can be an ultrasound device, enabling the generation of holographic videos corresponding to ultrasound images within the ultrasound device. In other embodiments, the executing entity of the holographic video generation method can also be an electronic device other than an ultrasound device, which acquires ultrasound images from the ultrasound device and generates corresponding holographic videos based on the ultrasound images.

[0056] See Figure 1 A flowchart illustrating a first holographic video generation method according to an exemplary embodiment, as shown below. Figure 1 As shown, it includes:

[0057] S101: Acquire ultrasound images;

[0058] The purpose of this embodiment is to generate a holographic video corresponding to an ultrasound image and transmit it to a holographic projection device for playback. In specific implementation, the user can identify an object of interest in the ultrasound device and obtain an ultrasound image based on the identified object of interest. This ultrasound image can be a single image, a collection of multiple images, or a video composed of multiple images. When it is a video, the ultrasound image can include two-dimensional ultrasound video, three-dimensional volumetric ultrasound video, etc. In addition, in this embodiment, the image can be a two-dimensional ultrasound image or a rendered image after three-dimensional volumetric ultrasound data has been rendered in three dimensions.

[0059] In addition, users can also set the video playback configuration of the holographic projection device. That is, this embodiment also includes: obtaining the video playback configuration of the holographic projection device. The video playback configuration may include the holographic video type (270° or 360°), whether the pyramid device in the holographic projection device is inverted, the resolution of the holographic projection device (e.g., 2048×1536, 2388×1668, 1920×1200, 2560×1600, etc.), and the video format (e.g., avi, wmv, mp4, etc.). When the object of interest is a single three-dimensional volume data or a three-dimensional volume data video, it may also include the output projection image (the content of the projection images corresponding to different projection surfaces may be the same or different), the rendering mode (surface imaging, depth imaging, high-resolution imaging, contour imaging, or rainbow infusion imaging, etc.). When the object of interest is a single three-dimensional volume data, it may also include the rotation method (e.g., automatic rotation, light rotation, or volume rotation) and related parameters.

[0060] Furthermore, when the video playback is configured for inverted playback, the image in the ultrasound video is flipped vertically. That is, the ultrasound image is flipped before being mapped onto the projected image. It can be understood that the holographic video played by the holographic projection device is displayed in the shape of a pyramid. If the pyramid device is inverted, the image in the ultrasound video is flipped vertically, which can be achieved using an image flipping function, and the flipped image data is recorded in the video file. This image flipping function can be the `Flip` function in OpenCV. If the pyramid device is upright, it remains unchanged.

[0061] Optionally, acquiring ultrasound images includes: acquiring a two-dimensional ultrasound image, and rendering the two-dimensional ultrasound image to generate an ultrasound image containing the rendered image. The rendering of the two-dimensional ultrasound image can be achieved using surface rendering.

[0062] Optionally, acquiring ultrasound images includes: acquiring three-dimensional ultrasound volume data; and performing three-dimensional volume rendering on the three-dimensional ultrasound volume data to generate an ultrasound image containing the rendered image. The three-dimensional ultrasound volume data is data containing ultrasound three-dimensional volume information, and can be three-dimensional ultrasound volume data. Performing three-dimensional volume rendering on the three-dimensional ultrasound volume data can involve processing the three-dimensional ultrasound volume data according to specific color, angle, and lighting conditions to generate a rendered image that meets the requirements.

[0063] Furthermore, the angles of the 3D ultrasound volume data can be adjusted, and 3D volume rendering can be performed based on the angle-adjusted 3D ultrasound volume data to obtain the rendered image corresponding to angle θ1. Then, the angles of the 3D ultrasound volume data are adjusted again, and 3D volume rendering is performed again based on the angle-adjusted 3D ultrasound volume data to obtain the rendered image corresponding to angle θ2. This process can be repeated, with multiple angle adjustments to the 3D ultrasound volume data and 3D volume rendering performed on the angle-adjusted 3D ultrasound volume data. Even further, when only one rendering is performed, the resulting ultrasound image contains one rendered image; when multiple renderings are performed, the resulting ultrasound image contains multiple rendered images.

[0064] Optionally, acquiring three-dimensional ultrasound volume data and performing three-dimensional volume rendering on the three-dimensional ultrasound volume data to generate an ultrasound image containing the rendered image includes: acquiring a single three-dimensional ultrasound volume data, determining the initial rotation angle and rotation step; and performing three-dimensional volume rendering on the three-dimensional ultrasound volume data based on the initial rotation angle and rotation step to generate an ultrasound image containing the rendered image. The initial rotation angle can be any angle from 0 to 360°. The rotation step is used to control the rotation amplitude; a large rotation step allows the three-dimensional ultrasound volume data to be rotated one revolution faster than a small rotation step. The rotation step can be 10°, 20°, 30°, etc. It can be understood that rotating the three-dimensional ultrasound volume data can be equivalent to rotating the three-dimensional image corresponding to the three-dimensional ultrasound volume data. Once the angle is determined, performing three-dimensional volume rendering on the three-dimensional ultrasound volume data yields a rendered image from a certain viewpoint; rotating and rendering the three-dimensional ultrasound volume data according to the step yields a rendered image from another viewpoint; in this way, rendered images from different viewpoints can be obtained. Based on these rendered images, projected images are generated, and holographic videos can be obtained from these projected images. For a certain projection surface of a holographic projection device, it displays a projected image based on a single three-dimensional ultrasound volume data at a certain moment, such as displaying the image state of a fetus at a certain moment. This projected image can be continuously rotated to show the image state from various perspectives.

[0065] Optionally, acquiring three-dimensional ultrasound volume data and performing three-dimensional volume rendering on the three-dimensional ultrasound volume data to generate an ultrasound image containing the rendered image includes: acquiring multiple three-dimensional ultrasound volume data; performing three-dimensional volume rendering on the multiple three-dimensional ultrasound volume data to generate an ultrasound image containing the rendered image corresponding to each three-dimensional ultrasound volume data. The aforementioned multiple three-dimensional ultrasound volume data can be a sequence of ultrasound volume data, which can be: three-dimensional ultrasound volume data of a certain ultrasound detection object at different time sequences (e.g., three-dimensional ultrasound volume data of a certain fetus within a certain time period), or three-dimensional ultrasound volume data of different ultrasound detection objects (e.g., three-dimensional ultrasound volume data of different parts of a fetus at a certain moment, or three-dimensional ultrasound volume data of different fetuses at the same moment). For a certain projection surface of the holographic projection device, it displays the projected image of multiple three-dimensional ultrasound volume data at different time sequences, for example: displaying the image state of a certain fetus within a certain time period.

[0066] Optionally, three-dimensional ultrasound volume data is acquired, and three-dimensional volume rendering is performed on the three-dimensional ultrasound volume data to generate an ultrasound image containing the rendered image. This includes: acquiring multiple three-dimensional ultrasound volume data and determining a target rotation angle; performing three-dimensional volume rendering on the multiple three-dimensional ultrasound volume data based on the target rotation angle to generate an ultrasound image containing the rendered image corresponding to each three-dimensional ultrasound volume data. The target rotation angle can be 0° or not 0°. The target rotation angles corresponding to different three-dimensional ultrasound volume data can be the same or different. When the target rotation angles corresponding to different three-dimensional ultrasound volume data are the same, after rotating a certain three-dimensional ultrasound volume data based on the target rotation angle, it can be rendered to obtain a rendered image 1 of the three-dimensional ultrasound volume data at that angle, and a projected image under a certain projection plane is generated based on rendered image 1; after rotating another three-dimensional ultrasound volume data based on the target rotation angle, it can be rendered to obtain a rendered image 2 of the three-dimensional ultrasound volume data at that angle, and a projected image under another projection plane is generated based on rendered image 2; and so on, generating projected images of each three-dimensional ultrasound volume data under the corresponding projection plane. After arranging these projected images in chronological order and generating a holographic video, for a certain projection surface, what is displayed is the projected image of the ultrasonic testing object changing over time at a certain angle, i.e., the projected video.

[0067] S102: Determine the correspondence between each projection surface of the holographic projection device and each projection area in the holographic projection plane region;

[0068] In specific implementation, the holographic projection plane region includes multiple projection areas, each used to display a projected image corresponding to a projection surface. Each projection area in the holographic projection plane region can correspond one-to-one with each projection surface of the holographic projection device. It should be noted that the projection surfaces of the holographic projection device include a first projection surface and at least one second projection surface. The first projection surface is a front projection surface, and the second projection surfaces can include a rear projection surface, a left projection surface, and a right projection surface. This embodiment supports generating 270° holographic videos, with the projection surfaces including the front projection surface, the left projection surface, and the right projection surface. This embodiment also supports generating 360° holographic videos, with the projection surfaces including the front projection surface, the rear projection surface, the left projection surface, and the right projection surface. In this step, the correspondence between each projection surface of the holographic projection device and each projection area in the holographic projection plane region is determined. This correspondence can be the correspondence between the position coordinates of each projection surface and each projection area. In other words, as a feasible implementation method, this step includes: calculating the position coordinates of each projection area in the holographic projection plane area based on the resolution of the holographic projection device; and determining the correspondence between each projection surface of the holographic projection device and the position coordinates of each projection area in the holographic projection plane area. Specifically, this can be achieved by: calculating the position coordinates of the center point of each projection area in the holographic projection plane area based on the resolution of the holographic projection device; and determining the correspondence between each projection surface of the holographic projection device and the position coordinates of the center point of each projection area in the holographic projection plane area.

[0069] The resolution of the holographic projection device is nVideoPixelsW × nVideoPixelsH, and the resolution of the image in the ultrasound image is nImageWidth × nImageHeight. In specific implementation, firstly, the screen scaling factor ScreenScale is calculated based on the resolution of the holographic projection device or the resolution of the image in the ultrasound image. Secondly, the size of the projection screen, nDisplayW × nDisplayH, is calculated based on the resolution of the holographic projection device and the screen scaling factor. Furthermore, the position coordinates of each projection area in the holographic projection plane area are calculated based on the resolution of the holographic projection device and the size of the projection screen, and the correspondence between each projection surface and the position coordinates of each projection area is determined.

[0070] Understandably, a 270° holographic video provides three projection surfaces, while a 360° holographic video provides four. Therefore, the methods for calculating the position coordinates of the center point of the projected image corresponding to each projection surface within the holographic projection plane area are somewhat different.

[0071] A 270° holographic video provides three projection surfaces. The screen scaling factor (ScreenScale) is calculated based on the resolution of the holographic projection device as follows: The height and width of the holographic projection device are summed, and the ratio of the height of the holographic projection device to the summation result is used as the screen scaling factor. The specific calculation formula is as follows:

[0072] ScreenScale=nVideoPixelsH / (nVideoPixelsH+nVideoPixelsW).

[0073] The method for calculating the screen scaling factor (ScreenScale) based on the resolution of the ultrasound image is as follows: sum the height and width of the ultrasound image, and calculate the ratio of the height of the ultrasound image to the summation result. This ratio is used as the screen scaling factor. The specific calculation formula is: ScreenScale = nImageHeight / (nImageWidth + nImageHeight).

[0074] The height of the projection screen, nDisplayH, is calculated as follows: The product of the holographic projection device's height and half the screen scaling factor is calculated. This product is then rounded down. The result is multiplied by 2. Finally, this product and 2 are summed to obtain the projection screen height. The specific formula is as follows:

[0075] nDisplayH=int(nVideoPixelsH×ScreenScale / 2.0)×2+2.

[0076] The width nDisplayW of the projection screen is calculated as follows: the difference between the height of the holographic projection device and the height of the projection screen is used to obtain the width of the projection screen. The specific calculation formula is: nDisplayW = nVideoPixelsH - nDisplayH.

[0077] The coordinates of the center point of the projected image corresponding to the front projection plane in the holographic projection plane area are (nVideoPixelsW / 2, nVideoPixelsH-nDisplayH / 2), the coordinates of the center point of the projected image corresponding to the left projection plane in the holographic projection plane area are (nDisplayH / 2, nDisplayW / 2), and the coordinates of the center point of the projected image corresponding to the right projection plane in the holographic projection plane area are (nVideoPixelsW-DisplayH / 2, nDisplayW / 2).

[0078] The 360° holographic video provides four projection surfaces. The screen scaling factor (ScreenScale) is calculated based on the resolution of the holographic projection device as follows: The sum of twice the height and width of the holographic projection device is calculated, and the ratio of the height of the holographic projection device to the sum is used as the screen scaling factor. The specific calculation formula is: ScreenScale = nVideoPixelsH / (2 × nVideoPixelsH + nVideoPixelsW). Similarly, the screen scaling factor (ScreenScale) is calculated based on the resolution of the image in the ultrasound video as follows: The sum of twice the height and width of the image in the ultrasound image is calculated, and the ratio of the height of the image in the ultrasound image to the sum is used as the screen scaling factor. The specific calculation formula is: ScreenScale = nImageHeight / (2 × nImageHeight + nImageWidth).

[0079] The method for calculating the projection screen size nDisplayW×nDisplayH is the same as described above:

[0080] nDisplayH=int(nVideoPixelsH×ScreenScale / 2.0)×2+2;

[0081] nDisplayW=nVideoPixelsH-nDisplayH.

[0082] The coordinates of the center point of the projected image corresponding to the orthographic projection plane in the holographic projection plane area are:

[0083] (nVideoPixelsW / 2,nVideoPixelsH-nDisplayH / 2);

[0084] The coordinates of the center point of the projected image corresponding to the rear projection surface in the holographic projection plane area are: (nVideoPixelsW / 2, nDisplayH / 2);

[0085] The coordinates of the center point of the projected image corresponding to the left projection plane in the holographic projection plane area are: (nDisplayH / 2+(nVideoPixelsW-nVideoPixelsH) / 2,nVideoPixelsH / 2);

[0086] The coordinates of the center point of the projected image corresponding to the right projection plane in the holographic projection plane area are:

[0087] (nVideoPixelsW-nDisplayH / 2-(nVideoPixelsW-nVideoPixelsH) / 2,nVideoPixelsH / 2).

[0088] S103: Determine the projection image corresponding to each projection plane based on ultrasound imaging;

[0089] In this step, the projected image corresponding to each projection plane is determined based on the images in the ultrasound video. If the ultrasound video includes multiple two-dimensional ultrasound images, each two-dimensional ultrasound image is determined as the projected image corresponding to the orthographic projection plane, and the projected image corresponding to the orthographic projection plane is rotated to generate the projected images corresponding to other projection planes. Alternatively, some two-dimensional ultrasound images can be directly rotated to generate the projected images corresponding to other projection planes.

[0090] If the ultrasound video includes a three-dimensional volume image, the projected image corresponding to each projection plane is determined by the rendered image after three-dimensional volume rendering based on the three-dimensional ultrasound volume data. It should be noted that the content of the projected images corresponding to each projection plane can be completely identical, differing only in orientation; conversely, the content of the projected images corresponding to each projection plane can be different. This embodiment does not impose specific limitations. In specific implementation, the rendered image after three-dimensional volume rendering is used as the projected image corresponding to the orthographic projection plane. If the content of the projected images corresponding to each projection plane is the same, the projected image corresponding to the orthographic projection plane is rotated to generate projected images corresponding to other projection planes, with the rotation method being the same as described above. If the content of the projected images corresponding to each projection plane is different, a rendered image corresponding to each projection plane needs to be generated separately before rotation.

[0091] It is understandable that there is no restriction on the order of steps S102 and S103.

[0092] S104: According to the correspondence, place the projection image corresponding to each projection surface into the projection area corresponding to the holographic projection plane area to stitch together the holographic projection image.

[0093] In this step, the projected image corresponding to each projection surface determined in step S103 is copied to the corresponding position in the holographic projection plane area according to the correspondence determined in step S102. As a feasible implementation, the projected image corresponding to each projection surface can be copied to the corresponding position in the holographic projection plane area based on the position coordinates of each projection area within the holographic projection plane area.

[0094] For a 270° holographic video, three projected images need to be copied, such as... Figure 2 As shown, a 360° holographic video requires copying four projected images, such as... Figure 3As shown. Copying the projected image can be achieved using an image copy function, which copies the data of the projected image and establishes key-value pairs with the corresponding holographic projection plane region, then records the established key-value pairs into the video file. This image copy function can be the `copyTo` function in OpenCV, as shown in the following example:

[0095] Holo_Frame_Front.copyTo(holoFrame(roi_front));

[0096] Where Holo_Frame_Front is the projected image corresponding to the orthographic projection plane, holoFrame is the frame image in the holographic video, and roi_front is the position coordinate of the center point of the projected image corresponding to the orthographic projection plane in the holographic projection plane area. The above statement copies the projected image corresponding to the orthographic projection plane to the position corresponding to the holographic image frame.

[0097] S105: Generate holographic video based on holographic projection images.

[0098] Each stitching operation produces a holographic projection image that can serve as a frame of a holographic video. After stitching together all the holographic projection images, these frames can be sequentially strung together to form a video, thus obtaining the corresponding holographic video.

[0099] In this step, the holographic projection image can be written to a video file using an image writing function to generate a holographic video. This image writing function can be the write function in the VideoWriter class of OpenCV: writer.write(holoFrame).

[0100] Based on the set parameters of the holographic video, a corresponding holographic video is generated. The core of this process is to rationally divide the effective resolution of the holographic projection device according to the positions of three or four holographic projection modules, and place images at corresponding angles in different positions. This allows viewers to see content from different angles within a 270° or 360° range, thus producing a stereoscopic projection effect. The ultrasonic device transmits the generated holographic video to the holographic projection device, which uses a projector or LED screen as the playback device. A transparent panel covered with a holographic film serves as the carrier, projecting the video material onto the plane covered with the holographic projection film. Pure black pixels in the video material do not produce light, while the remaining pixel information is displayed on the transparent panel. The visual effect is both transparent and realistic, providing viewers with a completely different viewing experience.

[0101] As a possible implementation method, such as Figure 4As shown, during an ultrasound examination, the ultrasound equipment generates holographic videos from the real-time acquired ultrasound images and transmits them to a holographic projection device for holographic projection via a network in real time.

[0102] As another feasible implementation method, such as Figure 5 As shown, the user selects the object of interest in the patient management system of the ultrasound equipment, sets the relevant parameters of the holographic video, clicks the video export button, generates a holographic video, exports it to the holographic projection device, and projects the holographic video onto the holographic projection device.

[0103] It should be noted that holographic video can be output to a single holographic projection device for playback. This involves placing the projected image corresponding to each projection surface onto the holographic projection plane area to obtain a holographic projection image, which is then written into a single video file for playback on a single holographic projection device. Alternatively, holographic video can also be output to multiple holographic projection devices for playback. This means directly writing the projected images corresponding to each projection surface into a separate video file for each surface, and then outputting them to different holographic projection devices for playback. In this approach, the resolution of the image in the ultrasound video is the same as the resolution of the holographic projection device, both being nVideoPixelsW × nVideoPixelsH. The method for generating a 270° holographic video file is as follows: Figure 6 As shown, the projected image corresponding to the left projection plane is written into the left-side video file. The method for generating the video file for a 360° holographic video is as follows: Figure 7 As shown, the implementation method is the same as Figure 6 The method is similar.

[0104] The holographic video generation method provided in this application involves an ultrasound device generating projection images corresponding to each projection surface based on images in an ultrasound image. According to the correspondence between each projection surface and each projection region in the holographic projection plane region, the projection images corresponding to each projection surface are placed in the holographic projection plane region to obtain a holographic projection image. This allows the generation of a holographic video corresponding to the ultrasound video based on the holographic projection image, which can then be played on a holographic projection device. Therefore, this application embodiment achieves the generation of a holographic video corresponding to an ultrasound image within an ultrasound device.

[0105] This application discloses a method for generating a holographic video corresponding to a two-dimensional ultrasound video, specifically:

[0106] See Figure 8 A flowchart illustrating a second holographic video generation method according to an exemplary embodiment, as shown below. Figure 8 As shown, it includes:

[0107] S201: Acquire an ultrasound image containing multiple two-dimensional ultrasound images;

[0108] The purpose of this embodiment is to generate a holographic video corresponding to a two-dimensional ultrasound video. That is, the ultrasound image in this step is a collection of images composed of multiple two-dimensional ultrasound images, such as... Figure 9 As shown, each frame of the two-dimensional ultrasound image in the two-dimensional ultrasound video is read, and it is determined whether the pyramid device in the holographic projection device is inverted. If the pyramid device is inverted, the two-dimensional ultrasound image is flipped vertically; if the pyramid device is upright, it remains unchanged. Additionally, the two-dimensional ultrasound image can be rendered. The rendered image is still in the form of a two-dimensional ultrasound image, and then an ultrasound image is obtained based on the rendered image.

[0109] S202: Determine the correspondence between each projection surface of the holographic projection device and each projection area in the holographic projection plane region;

[0110] S203: Determine the two-dimensional ultrasound image in the ultrasound image as the projection image corresponding to the first projection plane, determine the rotation matrix around the center point of the two-dimensional ultrasound image based on the correspondence, and perform rotation processing on the projection image corresponding to the first projection plane based on the rotation matrix to generate projection images corresponding to each second projection plane.

[0111] In this step, the two-dimensional ultrasound images in the two-dimensional ultrasound video are identified as the projection images corresponding to the first projection plane, i.e., the orthographic projection plane. Based on the correspondence between each projection plane and its projection region, rotation matrices are determined for each second projection plane. The projection images corresponding to the orthographic projection plane are then rotated based on these rotation matrices to generate the projection images corresponding to each second projection plane. In practice, the two-dimensional ultrasound video can be opened using a video reading function, and the two-dimensional ultrasound images in the video can be acquired frame by frame, serving as the projection images corresponding to the orthographic projection planes. This video reading function can be implemented using the VideoCapture class object in OpenCV or the ffmpeg library. Additionally, rotation matrices corresponding to each second projection plane can be generated using a rotation matrix generation function, such as the getRotationMatrix2D function in OpenCV. Furthermore, the projection images corresponding to the orthographic projection planes can be rotated using an image rotation function, such as the warpAffine function in OpenCV.

[0112] For the projected image corresponding to the frontal projection plane, establish a coordinate system with the X-axis horizontal, the Y-axis vertical, and the Z-axis perpendicular to the projected image. For a 270° holographic video, the projected image corresponding to the left projection plane is the projected image corresponding to the frontal projection plane rotated 90° clockwise around the Z-axis, and the projected image corresponding to the right projection plane is the projected image corresponding to the frontal projection plane rotated 90° counterclockwise around the Z-axis. For a 360° holographic video, the projected image corresponding to the rear projection plane is the projected image corresponding to the frontal projection plane rotated 180° clockwise around the Z-axis, the projected image corresponding to the left projection plane is the projected image corresponding to the frontal projection plane rotated 90° clockwise around the Z-axis, and the projected image corresponding to the right projection plane is the projected image corresponding to the frontal projection plane rotated 90° counterclockwise around the Z-axis.

[0113] S204: According to the correspondence, place the projection image corresponding to each projection surface into the projection area corresponding to the holographic projection plane area to stitch together the holographic projection image;

[0114] S205: Generate holographic video based on holographic projection images.

[0115] In practice, the projection image corresponding to each projection surface is placed in the corresponding projection area of ​​the holographic projection plane area to obtain a holographic projection image corresponding to a frame of two-dimensional ultrasound image. Finally, the holographic projection image corresponding to each frame of two-dimensional ultrasound image is written into a video file in sequence to generate a holographic video.

[0116] Optionally, regarding the implementation of determining a two-dimensional ultrasound image from an ultrasound image as the projection image corresponding to the first projection plane, if the ultrasound image includes a single two-dimensional ultrasound image, then that two-dimensional ultrasound image can be directly determined as the projection image corresponding to the first projection plane; if the ultrasound image includes at least one two-dimensional ultrasound image, then one two-dimensional ultrasound image can be selected sequentially from these two-dimensional ultrasound images and determined as the projection image corresponding to the first projection plane. The implementation of sequentially selecting one two-dimensional ultrasound image from multiple two-dimensional ultrasound images can be: 1) random selection, 2) selection according to a specific order. For example, if these two-dimensional ultrasound images are rendered sequentially, then selection can be performed according to the rendering order.

[0117] This application discloses a method for generating a holographic video corresponding to a single three-dimensional ultrasound volume data, specifically:

[0118] See Figure 10 A flowchart illustrating a third holographic video generation method according to an exemplary embodiment, as shown below. Figure 10 As shown, it includes:

[0119] S301: Acquire single three-dimensional ultrasonic volume data and determine the same initial rotation angle and rotation step for all projection planes;

[0120] The purpose of this embodiment is to generate a holographic video corresponding to a single three-dimensional ultrasound volume data, wherein the content of the projected images corresponding to different projection surfaces is the same, and the processing flow is as follows: Figure 11 As shown. In specific implementation, the same initial rotation angle fRotateAngle and rotation step fRotateAngleStep are first determined for all projection surfaces.

[0121] S302: Perform three-dimensional volume rendering on the three-dimensional ultrasound volume data based on the initial rotation angle and rotation step to generate an ultrasound image containing the rendered image;

[0122] It's important to note that during holographic projection, the black pixels in the video do not emit light; the remaining pixels are displayed within a transparent material. This allows the viewer to see both the objects on the other side and the objects in the video, creating a visually clear and realistic effect. Therefore, in 3D rendering, the background color is set to black, and a high-transparency, high-resolution rendering mode is selected.

[0123] In practical implementation, the total number of rendered images, nFrameNum, is obtained by calculating the ratio of the total automatic rotation angle fRotateAngle to the rotation step fRotateAngleStep. The specific calculation formula is as follows:

[0124] nFrameNum=fRotateAngle / fRotateAngleStep.

[0125] The initial rotation angle is generally 0°. The three-dimensional ultrasonic volume data after each rotation is rendered. Each rendering generates a corresponding rendered image. That is, one rotation can obtain the corresponding rendered image. nFrameNum rotations will obtain nFrameNum rendered images.

[0126] As a feasible implementation, this step includes: constructing a transformation matrix; performing 3D volume rendering on the 3D ultrasound volume data based on the transformation matrix to generate an ultrasound image containing the rendered images; and preprocessing the rendered images in the ultrasound image to adapt them to the size of the projection screen of the holographic projection device. In a specific implementation, firstly, the 3D ultrasound volume data is rendered using a ray casting method based on the transformation matrix to generate nFrameNum rendered images. Secondly, the rendered images are preprocessed to adapt them to the size of the projection screen of the holographic projection device.

[0127] Two approaches can be used to obtain rendered images at different resolutions. One approach is to render the image according to the display resolution of the 3D ultrasound volume data, and then scale the rendered image according to the size of the projection screen. This involves constructing a transformation matrix, including: building a transformation matrix based on the display height of the 3D ultrasound volume data; and correspondingly, preprocessing the rendered image in the ultrasound image to adapt it to the size of the holographic projection device's screen, including: calculating an image scaling factor based on the size of the 3D ultrasound volume data and the size of the projection screen; and scaling the rendered image based on the image scaling factor to adapt it to the size of the holographic projection device's screen.

[0128] In specific implementation, such as Figure 12 As shown, the transformation matrix is ​​constructed first:

[0129] TransFormMatrix=ViewPortM·ProjM·View3D·ModelM

[0130] The initial values ​​for each parameter are:

[0131]

[0132]

[0133] Where WinSize is the display height nImageHeight of the three-dimensional ultrasound volume data.

[0134] Next, perform a three-dimensional rotation:

[0135] View3D=R y (θ)×View3D;

[0136] After substituting the rotated View3D into the TransformMatrix, the transformation matrix after volume rotation is obtained. Based on the transformation matrix and the current rotation angle, 3D volume rendering is performed to generate a single rendered image. The image scaling factor Scale is calculated based on the dimensions of the 3D ultrasound volume data (nImageWidth×nImageHeight) and the dimensions of the projection screen (nDisplayW×nDisplayH). First, the ratio between the width of the projection screen and the width of the 3D ultrasound volume data is calculated as the width scaling factor Scale_w. Second, the ratio between the height of the projection screen and the height of the 3D ultrasound volume data is calculated as the height scaling factor Scale_h. Finally, the minimum value between the width scaling factor and the height scaling factor is taken as the image scaling factor. The specific calculation formula is as follows:

[0137] Scale_w=nDisplayW / nImageWidth;

[0138] Scale_h=nDisplayH / nImageHeight;

[0139] Scale=min(Scale_w,Scale_h);

[0140] The rendered image is scaled based on the image scaling factor. It is then determined whether the pyramid device in the holographic projection device is upside down. If the pyramid device is upside down, the rendered image is flipped vertically. If the pyramid device is upright, it remains unchanged.

[0141] Another approach to obtaining rendered images at different resolutions is to calculate the target rendered image size based on the dimensions of the 3D ultrasound volume data and the projection screen size. This involves calculating the target rendered image size adapted to the projection screen size based on the width-to-height ratio of the 3D ultrasound volume data. Finally, the image is cropped according to the VOI (Volume of Interest) ratio. This approach yields higher image resolution. This involves constructing a transformation matrix, including: calculating the optimal rendered image size based on the resolution of the holographic projection device and the display height and width of the 3D ultrasound volume data, and constructing a transformation matrix based on the height of the optimal rendered image; correspondingly, preprocessing the rendered image in the ultrasound image to adapt it to the projection screen size of the holographic projection device, including: cropping the rendered image in the ultrasound image to fit the projection screen size of the holographic projection device.

[0142] In specific implementation, such as Figure 13 As shown, the optimal rendering image size RendeImageW × RenderImageH is first calculated based on the resolution of the holographic projection device and the display height and width of the 3D ultrasonic volume data. The specific process is as follows:

[0143]

[0144] Scale_h = nVideoPixelsH / (nImageHeight × nfVOIY); where nfVOIX, nfVOIY, and nfVOIZ are the sizes of VOI;

[0145] Scale=min(Scale_w,Scale_h);

[0146] nDisplayW=nVideoPixelsW / Scale;

[0147] nDisplayH=nVideoPixelsH / Scale;

[0148] RenderImageH = nDisplayH;

[0149] RenderImageW=nImageWidth / nImageHeight×RenderImageH.

[0150] Secondly, a transformation matrix is ​​constructed, using the same method as the previous approach, except that WinSize is set to the height of the optimal rendered image, RenderImageH. Based on the transformation matrix and the current rotation angle, a single rendered image is generated using 3D volume rendering. The rendered image, RendeImageW × RenderImageH, is then cropped to obtain an image of size nDisplayW × nDisplayH. It is then determined whether the pyramid device in the holographic projection is inverted. If the pyramid device is inverted, the rendered image is flipped vertically; if the pyramid device is upright, it remains unchanged.

[0151] S303: Determine the correspondence between each projection surface of the holographic projection device and each projection area in the holographic projection plane region;

[0152] S304: Determine the rendered image as the projection image corresponding to the first projection surface, determine the rotation matrix around the center point of the rendered image based on the correspondence, and perform rotation processing on the projection image corresponding to the first projection surface based on the rotation matrix to generate projection images corresponding to each second projection surface.

[0153] Since ultrasound images contain multiple rendered images, one of them can be selected as the projected image corresponding to the first projection plane, and rotated to generate projected images corresponding to each of the second projection planes. All projected images corresponding to the projection planes are used as the first frame in the holographic video. Then, the next image is selected as the projected image corresponding to the first projection plane, and rotated to generate projected images corresponding to each of the second projection planes. All projected images corresponding to the projection planes are used as the second frame in the holographic video. This process continues until all rendered images are associated with the projected images. At this point, the corresponding holographic image can be generated based on each frame.

[0154] S305: According to the correspondence, place the projection image corresponding to each projection surface into the projection area corresponding to the holographic projection plane area to stitch together the holographic projection image;

[0155] S306: Generate holographic video based on holographic projection images.

[0156] In practice, each frame of the rendered image in the ultrasound image is read sequentially. Based on the holographic video type, the rendered image is rotated to obtain the projection image corresponding to each projection plane. The rendered image is used as the projection image corresponding to the first projection plane, i.e., the orthographic projection plane. The projection image corresponding to the orthographic projection plane is rotated to generate projection images corresponding to each second projection plane. Then, the projection image corresponding to each projection plane is placed in the corresponding projection area of ​​the holographic projection plane region to obtain the holographic projection image corresponding to one frame of the rendered image. For example, the holographic projection image corresponding to a 270° holographic video is as follows: Figure 14 As shown (the content of the projected images corresponding to the three projection surfaces is the same), the holographic projection image corresponding to the 360° holographic video is as follows: Figure 15 and 16 As shown, the holographic projection image corresponding to each frame of the rendered image is then sequentially written into the video file to generate a holographic video. The actual effect of the 360° holographic video projection is as follows: Figure 17 and 18 As shown.

[0157] This application discloses another method for generating a holographic video corresponding to a single three-dimensional ultrasound volume data, specifically:

[0158] See Figure 19 A flowchart illustrating a fourth holographic video generation method according to an exemplary embodiment, as shown below. Figure 19 As shown, it includes:

[0159] S401: Acquire single three-dimensional ultrasonic volume data, determine the different initial rotation angles corresponding to each projection plane and the same rotation step corresponding to all projection planes;

[0160] The purpose of this embodiment is to generate a holographic video corresponding to a single three-dimensional ultrasound volume data, wherein the content of the projected image corresponding to different projection surfaces is different, and the processing flow is as follows: Figure 20 As shown. In specific implementation, firstly, the different initial rotation angles and the same rotation step are determined for all projection surfaces.

[0161] For a 270° holographic video, the initial rotation angle of the front projection plane is 0° (initial position), the initial rotation angle of the left projection plane is 90° around the Y-axis from the initial position, and the initial rotation angle of the right projection plane is -90° around the Y-axis from the initial position. For a 360° holographic video, the initial rotation angle of the front projection plane is 0° (initial position), the initial rotation angle of the right projection plane is 180° around the Y-axis from the initial position, the initial rotation angle of the left projection plane is 90° around the Y-axis from the initial position, and the initial rotation angle of the right projection plane is -90° around the Y-axis from the initial position.

[0162] S402: Perform three-dimensional volume rendering on the three-dimensional ultrasound volume data based on the initial rotation angle and rotation step corresponding to each projection plane to generate an ultrasound image containing multiple rendered images corresponding to each projection plane.

[0163] In this embodiment, the number of frames contained in the ultrasound image corresponding to each projection plane is the same. The only difference is the angle during 3D volume rendering. The specific rendering method is the same as in the previous embodiment, and will not be repeated here.

[0164] S403: Determine the correspondence between each projection surface of the holographic projection device and each projection area in the holographic projection plane region;

[0165] S404: Determine the rendered image in the ultrasound image corresponding to the first projection plane as the projected image corresponding to the first projection plane, determine the rotation matrix around the center point of the rendered image in the ultrasound image corresponding to each second projection plane based on the correspondence, and perform rotation processing on the rendered image in the ultrasound image corresponding to each second projection plane based on the rotation matrix to generate the projected image corresponding to each second projection plane.

[0166] S405: According to the correspondence, place the projection image corresponding to each projection surface into the projection area corresponding to the holographic projection plane area to stitch together the holographic projection image;

[0167] S406: Generate holographic video based on holographic projection images.

[0168] In practice, each frame of the rendered image in the ultrasound image corresponding to each projection plane is read sequentially. The rendered image is rotated according to the holographic video type to obtain the projection image corresponding to each projection plane. The rendered image in the ultrasound image corresponding to the first projection plane (i.e., the orthographic projection plane) is used as the projection image corresponding to the orthographic projection plane. The rendered images in the ultrasound images corresponding to each second projection plane are rotated to generate the projection images corresponding to each second projection plane. Then, the projection image corresponding to each projection plane is placed at the corresponding position in the holographic projection plane area to obtain a holographic projection image. The holographic projection image corresponding to a 270° holographic video is shown below. Figure 21 As shown (the content of the projected images corresponding to the three projection surfaces is different), the multiple holographic projection images are then written into the video file in sequence to generate a holographic video.

[0169] This application discloses a method for generating a holographic video corresponding to a three-dimensional ultrasound volumetric video, specifically:

[0170] See Figure 22 A flowchart illustrating a fifth holographic video generation method according to an exemplary embodiment, as shown below. Figure 22 As shown, it includes:

[0171] S501: Acquire multiple three-dimensional ultrasonic volume data and determine the same target rotation angle corresponding to all projection planes;

[0172] The purpose of this embodiment is to generate a holographic video corresponding to a three-dimensional ultrasonic volumetric video, wherein the content of the projected images corresponding to different projection surfaces is the same. The processing flow is as follows: Figure 23 As shown. In specific implementation, the same target rotation angle corresponding to all projection surfaces is first determined.

[0173] S502: Perform three-dimensional volume rendering on multiple three-dimensional ultrasound volume data based on the target rotation angle to generate an ultrasound image containing the rendered image corresponding to each three-dimensional ultrasound volume data.

[0174] S503: Determine the correspondence between each projection surface of the holographic projection device and each projection area in the holographic projection plane region;

[0175] S504: Determine the rendered image as the projection image corresponding to the first projection surface, determine the rotation matrix around the center point of the rendered image based on the correspondence, and perform rotation processing on the projection image corresponding to the first projection surface based on the rotation matrix to generate projection images corresponding to each second projection surface.

[0176] Optionally, the process of determining the projected images of each projection surface is illustrated as follows: When there are two second projection surfaces, the first three-dimensional ultrasonic volume data b11 is selected from the acquired multiple three-dimensional ultrasonic volume data in chronological order; the three-dimensional ultrasonic volume data b11 is rotated according to the target rotation angle; the three-dimensional ultrasonic volume data b11 after angle rotation is rendered in three dimensions to obtain the rendered image c11 corresponding to the first projection surface; the rendered image c11 is rotated twice based on the rotation matrix to obtain the projected images corresponding to the two second projection surfaces respectively. Based on this, the first frame image in the holographic video is obtained by stitching together the rendered images. Next, the next three-dimensional ultrasonic volume data b12 is selected from the acquired multiple three-dimensional ultrasonic volume data in chronological order, and the second frame image in the holographic video is obtained in the same way; this process continues until all three-dimensional ultrasonic volume data are associated with the frame images in the holographic video, at which point the holographic video can be generated based on each frame image. When there are three or more second projection surfaces, the implementation is similar to that of two, except that more rotations are required to obtain the projected image corresponding to each second projection surface.

[0177] S505: According to the correspondence, place the projection image corresponding to each projection surface into the projection area corresponding to the holographic projection plane area to stitch together the holographic projection image.

[0178] S506: Generate holographic video based on holographic projection images.

[0179] In practice, volume data with the target rotation angle is selected from all 3D ultrasound volume data for rendering, generating multiple rendered images. Each rendered image is read sequentially, and rotated according to the holographic video type to obtain the projection image corresponding to each projection plane. The rendered image is used as the projection image corresponding to the first projection plane, i.e., the orthographic projection plane. The projection image corresponding to the orthographic projection plane is rotated to generate projection images corresponding to each second projection plane. Then, the projection image corresponding to each projection plane is placed in the corresponding projection area of ​​the holographic projection plane region to obtain a holographic projection image corresponding to the rendered image. Finally, multiple holographic projection images are sequentially written into a video file to generate a holographic video.

[0180] This application discloses another method for generating holographic videos corresponding to three-dimensional ultrasound volumetric videos, specifically:

[0181] See Figure 24 A flowchart illustrating a sixth holographic video generation method according to an exemplary embodiment, as shown below. Figure 24 As shown, it includes:

[0182] S601: Acquire multiple three-dimensional ultrasonic volume data and determine the different target rotation angles corresponding to each projection plane;

[0183] The purpose of this embodiment is to generate a holographic video corresponding to a three-dimensional ultrasonic volumetric video, wherein the content of the projected image corresponding to different projection surfaces is different, and the processing flow is as follows: Figure 25 As shown. In practice, the different target rotation angles corresponding to all projection planes are first determined.

[0184] For a 270° holographic video, the target rotation angle on the front projection plane is used as the initial position. The target rotation angle on the left projection plane is the initial position rotated 90° around the Y-axis, and the target rotation angle on the right projection plane is the initial position rotated -90° around the Y-axis. For a 360° holographic video, the target rotation angle on the front projection plane is used as the initial position. The target rotation angle on the right projection plane is the initial position rotated 180° around the Y-axis, the target rotation angle on the left projection plane is the initial position rotated 90° around the Y-axis, and the target rotation angle on the right projection plane is the initial position rotated -90° around the Y-axis.

[0185] S602: Perform three-dimensional volume rendering on multiple three-dimensional ultrasonic volume data based on the target rotation angle corresponding to each projection plane to generate an ultrasonic image containing multiple rendered images corresponding to each projection plane.

[0186] S603: Determine the correspondence between each projection surface of the holographic projection device and each projection area in the holographic projection plane region;

[0187] S604: Determine the rendered image in the ultrasound image corresponding to the first projection plane as the projected image corresponding to the first projection plane, determine the rotation matrix around the center point of the rendered image in the ultrasound image corresponding to each second projection plane based on the correspondence, and perform rotation processing on the rendered image in the ultrasound image corresponding to each second projection plane based on the rotation matrix to generate the projected image corresponding to each second projection plane.

[0188] Optionally, the process of determining the projected image of each projection plane is illustrated as follows: When there are two second projection planes, the target rotation angles include rotation angle a1, rotation angle a2, and rotation angle a3; the first three-dimensional ultrasonic volume data b21 is selected from the acquired multiple three-dimensional ultrasonic volume data in chronological order; the three-dimensional ultrasonic volume data b21 is rotated according to rotation angle a1, and the three-dimensional ultrasonic volume data b21 after angle rotation is rendered in three dimensions to obtain the rendered image c21 corresponding to the first projection plane; the three-dimensional ultrasonic volume data b21 is rotated according to rotation angle a2, and the three-dimensional ultrasonic volume data b21 after angle rotation is rendered in three dimensions to obtain the rendered image c21 corresponding to the first projection plane; the three-dimensional ultrasonic volume data b21 is rotated according to rotation angle a2, and the three-dimensional ultrasonic volume data c ... 1. Perform 3D volume rendering to obtain a rendered image c22 corresponding to one of the second projection planes. 2. Rotate the 3D ultrasound volume data b21 by rotation angle a3. Perform 3D volume rendering on the rotated 3D ultrasound volume data b21 to obtain a rendered image c23 corresponding to the other second projection plane. 3. Rotate the rendered image c22 based on the rotation matrix to obtain a projected image corresponding to one of the second projection planes. 4. Rotate the rendered image c23 based on the rotation matrix to obtain a projected image corresponding to the other second projection plane. Based on this, the first frame of the holographic video is obtained by stitching together the rendered images c21, c22, and c23. Next, select the next 3D ultrasound volume data b22 from the acquired 3D ultrasound volume data in chronological order, and obtain the second frame of the holographic video in the same way. This process continues until all 3D ultrasound volume data are associated with the frame images in the holographic video. At this point, the holographic video can be generated based on each frame image. When there are three or more second projection surfaces, the implementation is similar to that of two, except that more rendering and rotation are required to obtain the projection image corresponding to each second projection surface.

[0189] S605: According to the correspondence, place the projection image corresponding to each projection surface into the projection area corresponding to the holographic projection plane area to stitch together the holographic projection image;

[0190] S606: Generate holographic video based on holographic projection images.

[0191] In practice, volume data corresponding to the target rotation angle for each projection plane in each 3D ultrasound volume data is selected for rendering, generating an ultrasound image containing multiple rendered images for each projection plane. Rendered images at the same position in each ultrasound image are read sequentially. The rendered image in the ultrasound image corresponding to the first projection plane (i.e., the orthographic projection plane) is used as the projected image for that plane. The rendered images in the ultrasound images corresponding to each second projection plane are rotated to generate projected images for each second projection plane. Then, the projected images for each projection plane are placed at their corresponding positions in the holographic projection plane region to obtain a holographic projection image. Finally, multiple holographic projection images are sequentially written into a video file to generate a holographic video.

[0192] The following describes a holographic video generation apparatus provided in an embodiment of this application. The holographic video generation apparatus described below and the holographic video generation method described above can be referred to each other.

[0193] See Figure 26 A structural diagram of a holographic video generation apparatus according to an exemplary embodiment is shown, such as... Figure 26 As shown, it includes:

[0194] The first acquisition module 100 is used to acquire ultrasound images;

[0195] The first determining module 200 is used to determine the correspondence between each projection surface of the holographic projection device and each projection area in the holographic projection plane area;

[0196] The second determining module 300 is used to determine the projection image corresponding to each projection plane based on the ultrasound image;

[0197] The splicing module 400 is used to place the projection image corresponding to each projection surface into the projection area corresponding to the holographic projection plane area according to the correspondence, so as to splice the holographic projection image.

[0198] The generation module 500 is used to generate holographic videos based on holographic projection images.

[0199] The holographic video generation apparatus provided in this application involves an ultrasound device generating projected images corresponding to each projection surface based on images in an ultrasound image. According to the correspondence between each projection surface and each projection area in the holographic projection plane region, the projected images corresponding to each projection surface are placed in the holographic projection plane region to obtain a holographic projection image. This allows the generation of a holographic video corresponding to the ultrasound video based on the holographic projection image, which can then be played on a holographic projection device. Therefore, this application embodiment achieves the generation of a holographic video corresponding to an ultrasound image within an ultrasound device.

[0200] Based on the above embodiments, as a preferred implementation, the first determining module 200 is specifically used to: calculate the position coordinates of each projection area in the holographic projection plane area according to the resolution of the holographic projection device; and determine the correspondence between each projection surface of the holographic projection device and the position coordinates of each projection area in the holographic projection plane area.

[0201] Based on the above embodiments, as a preferred implementation, the first determining module 200 includes: a first calculation unit, used to calculate a screen scaling factor based on the resolution of the holographic projection device or the resolution of the image in the ultrasound video; a second calculation unit, used to calculate the size of the projection screen based on the resolution of the holographic projection device and the screen scaling factor; a third calculation unit, used to calculate the position coordinates of the center point of the projection image corresponding to each projection surface in the holographic projection plane area based on the resolution of the holographic projection device and the size of the projection screen; and a first determining unit, used to determine the correspondence between the position coordinates of each projection surface of the holographic projection device and the position coordinates of each projection area in the holographic projection plane area.

[0202] Based on the above embodiments, as a preferred implementation, the projection surface of the holographic projection device includes a first projection surface and at least one second projection surface; if the ultrasound video includes at least one two-dimensional ultrasound image, the second determining module 300 is specifically used to: determine the two-dimensional ultrasound image in the ultrasound image as the projection image corresponding to the first projection surface; determine a rotation matrix around the center point of the two-dimensional ultrasound image based on the correspondence; and perform rotation processing on the projection image corresponding to the first projection surface based on the rotation matrix to generate projection images corresponding to each of the second projection surfaces.

[0203] Based on the above embodiments, as a preferred implementation, the first acquisition module 100 is specifically used to acquire three-dimensional ultrasound volume data and perform three-dimensional volume rendering on the three-dimensional ultrasound volume data to generate an ultrasound image containing the rendered image; correspondingly, the second determination module 300 is specifically used to determine the projection image corresponding to each projection surface based on the rendered image.

[0204] Based on the above embodiments, as a preferred implementation, the first acquisition module 100 includes: a second determining unit, used to acquire a single three-dimensional ultrasound volume data and determine an initial rotation angle and rotation step; and a first rendering unit, used to perform three-dimensional volume rendering on the three-dimensional ultrasound volume data based on the initial rotation angle and rotation step to generate an ultrasound image containing the rendered image.

[0205] Based on the above embodiments, as a preferred implementation, the second determining unit is specifically used to determine the same initial rotation angle and rotation step corresponding to all projection surfaces; correspondingly, the projection surfaces of the holographic projection device include a first projection surface and at least one second projection surface; the second determining module 300 specifically: determines the rendered image as the projection image corresponding to the first projection surface; determines the rotation matrix around the center point of the rendered image based on the correspondence; and performs rotation processing on the projection image corresponding to the first projection surface based on the rotation matrix to generate projection images corresponding to each of the second projection surfaces.

[0206] Based on the above embodiments, as a preferred implementation, the second determining unit is specifically used to determine different initial rotation angles corresponding to each projection surface and the same rotation step for all projection surfaces; correspondingly, the first rendering unit is specifically used to perform three-dimensional volume rendering on the three-dimensional ultrasound volume data based on the initial rotation angle and rotation step for each projection surface, so as to generate an ultrasound image containing multiple rendered images for each projection surface; correspondingly, the projection surface of the holographic projection device includes a first projection surface and at least one second projection surface; the second determining module 300 specifically: determines the rendered image in the ultrasound image corresponding to the first projection surface as the projected image corresponding to the first projection surface; determines a rotation matrix for rotating around the center point of the rendered image in the ultrasound image corresponding to each second projection surface based on the correspondence; and performs rotation processing on the rendered image in the ultrasound image corresponding to each second projection surface based on the rotation matrix to generate the projected image corresponding to each second projection surface.

[0207] Based on the above embodiments, as a preferred implementation, the first acquisition module 100 includes: a third determining unit, used to acquire multiple three-dimensional ultrasound volume data and determine the target rotation angle; and a second rendering unit, used to perform three-dimensional volume rendering on the multiple three-dimensional ultrasound volume data based on the target rotation angle, so as to generate an ultrasound image containing the rendered image corresponding to each three-dimensional ultrasound volume data.

[0208] Based on the above embodiments, as a preferred implementation, the third determining unit is specifically used to determine the same target rotation angle corresponding to all projection surfaces; correspondingly, the projection surfaces of the holographic projection device include a first projection surface and at least one second projection surface; the second determining module 300 is specifically used to: determine the rendered image as the projection image corresponding to the first projection surface; determine the rotation matrix around the center point of the rendered image based on the correspondence; and perform rotation processing on the projection image corresponding to the first projection surface based on the rotation matrix to generate projection images corresponding to each of the second projection surfaces.

[0209] Based on the above embodiments, as a preferred implementation, the third determining unit is specifically used to determine different target rotation angles corresponding to each projection surface; correspondingly, the second rendering unit is specifically used to perform three-dimensional volume rendering on multiple three-dimensional ultrasound volume data based on the target rotation angles corresponding to each projection surface, so as to generate an ultrasound image containing multiple rendered images corresponding to each projection surface; correspondingly, the projection surface of the holographic projection device includes a first projection surface and at least one second projection surface; the second determining module 300 is specifically used to: determine the rendered image in the ultrasound image corresponding to the first projection surface as the projected image corresponding to the first projection surface; determine a rotation matrix based on the correspondence to rotate around the center point of the rendered image in the ultrasound image corresponding to each second projection surface; and perform rotation processing on the rendered image in the ultrasound image corresponding to each second projection surface based on the rotation matrix to generate the projected image corresponding to each second projection surface.

[0210] Based on the above embodiments, as a preferred implementation, the first acquisition module 100 is specifically used to acquire three-dimensional ultrasound volume data, construct a transformation matrix, perform three-dimensional volume rendering on the three-dimensional ultrasound volume data based on the transformation matrix to generate an ultrasound image containing the rendered image; and preprocess the rendered image in the ultrasound image to adapt it to the size of the projection screen of the holographic projection device.

[0211] Based on the above embodiments, as a preferred implementation, the first acquisition module 100 is specifically used to acquire three-dimensional ultrasound volume data, construct a transformation matrix based on the display height of the three-dimensional ultrasound volume data, perform three-dimensional volume rendering on the three-dimensional ultrasound volume data based on the transformation matrix to generate an ultrasound image containing the rendered image; calculate the image scaling factor according to the size of the three-dimensional ultrasound volume data and the size of the projection screen; and scale the rendered image based on the image scaling factor to adapt to the size of the projection screen of the holographic projection device.

[0212] Based on the above embodiments, as a preferred implementation, the first acquisition module 100 is specifically used to acquire three-dimensional ultrasound volume data, calculate the target rendering image size according to the resolution of the holographic projection device and the display height and display width of the three-dimensional ultrasound volume data, and construct a transformation matrix based on the target rendering image size; perform three-dimensional volume rendering on the three-dimensional ultrasound volume data based on the transformation matrix to generate an ultrasound image containing the rendered image; and crop the rendered image in the ultrasound image to adapt to the size of the projection screen of the holographic projection device.

[0213] Based on the above embodiments, as a preferred embodiment, it further includes: a second acquisition module, used to acquire the video playback configuration of the holographic projection device; and a flipping module, used to flip the image in the ultrasound image vertically when the video playback configuration is inverted.

[0214] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0215] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiments of this application, the embodiments of this application also provide an electronic device. Figure 27 This is a structural diagram of an electronic device according to an exemplary embodiment, such as... Figure 27 As shown, the electronic device includes:

[0216] Communication interface 1 enables information exchange with other devices, such as network devices;

[0217] Processor 2 is connected to communication interface 1 to enable information interaction with other devices. When running a computer program, it executes the holographic video generation method provided by one or more of the above-mentioned technical solutions. The computer program is stored in memory 3.

[0218] Of course, in practical applications, the various components in an electronic device are coupled together through bus system 4. It can be understood that bus system 4 is used to achieve communication and connection between these components. In addition to the data bus, bus system 4 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 27 The general will label all buses as Bus System 4.

[0219] The memory 3 in this embodiment is used to store various types of data to support the operation of the electronic device. Examples of such data include any computer program used to operate on the electronic device.

[0220] It is understood that memory 3 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memory 3 described in the embodiments of this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0221] The methods disclosed in the embodiments of this application can be applied to processor 2, or implemented by processor 2. Processor 2 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 2 or by instructions in the form of software. The processor 2 may be a general-purpose processor, DSP, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 2 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory 3. Processor 2 reads the program in memory 3 and completes the steps of the aforementioned method in combination with its hardware.

[0222] When processor 2 executes the program, it implements the corresponding processes in the various methods of the embodiments of this application. For the sake of brevity, these will not be described in detail here.

[0223] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a memory 3 that stores a computer program, which can be executed by a processor 2 to complete the steps described in the aforementioned method. The computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM.

[0224] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0225] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0226] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for generating holographic video, characterized in that, include: Acquire ultrasound images; Determine the correspondence between each projection surface of the holographic projection device and each projection area in the holographic projection plane region; Based on the ultrasound images, a projection image corresponding to each projection surface is determined; According to the correspondence, the projection image corresponding to each projection surface is placed in the projection area corresponding to the holographic projection plane area to stitch together a holographic projection image. Generate holographic video based on the holographic projection image; The determination of the correspondence between each projection surface of the holographic projection device and each projection area in the holographic projection plane region includes: Calculate the screen scaling factor based on the resolution of the holographic projection device or the resolution of the image in the ultrasound image; The size of the projection screen is calculated based on the resolution of the holographic projection device and the screen scaling factor. Calculate the position coordinates of each projection area in the holographic projection plane region based on the resolution of the holographic projection device and the size of the projection screen. Determine the correspondence between the position coordinates of each projection surface of the holographic projection device and the position coordinates of each projection area in the holographic projection plane region; Wherein, if the ultrasound image includes at least one two-dimensional ultrasound image, then determining the projection image corresponding to each projection surface based on the ultrasound image includes: determining the two-dimensional ultrasound image in the ultrasound image as the projection image corresponding to the first projection surface; the projection surface of the holographic projection device includes the first projection surface and at least one second projection surface; determining a rotation matrix around the center point of the two-dimensional ultrasound image based on the correspondence; and performing rotation processing on the projection image corresponding to the first projection surface based on the rotation matrix to generate projection images corresponding to each second projection surface; If the ultrasound image includes a rendered image obtained by three-dimensional volume rendering of three-dimensional ultrasound volume data, then determining the projection image corresponding to each projection surface based on the ultrasound image includes: determining the projection image corresponding to each projection surface based on the rendered image.

2. The holographic video generation method according to claim 1, characterized in that, The acquisition of ultrasound images includes: Acquire three-dimensional ultrasound volume data, and perform three-dimensional volume rendering on the three-dimensional ultrasound volume data to generate an ultrasound image containing the rendered image.

3. The holographic video generation method according to claim 2, characterized in that, The step of acquiring three-dimensional ultrasound volume data and performing three-dimensional volume rendering on the three-dimensional ultrasound volume data to generate an ultrasound image containing the rendered image includes: Acquire single three-dimensional ultrasonic volume data to determine the initial rotation angle and rotation step; Based on the initial rotation angle and the rotation step, the three-dimensional ultrasound volume data is rendered in three dimensions to generate an ultrasound image containing the rendered image.

4. The holographic video generation method according to claim 3, characterized in that, The determination of the initial rotation angle and rotation step includes: Determine the same initial rotation angle and rotation step for all the projection surfaces; Accordingly, the projection surface of the holographic projection device includes a first projection surface and at least one second projection surface; determining the projection image corresponding to each projection surface based on the rendered image includes: The rendered image is determined to be the projection image corresponding to the first projection surface; Based on the correspondence, a rotation matrix is ​​determined to rotate around the center point of the rendered image; The projection image corresponding to the first projection surface is rotated based on the rotation matrix to generate projection images corresponding to each of the second projection surfaces.

5. The holographic video generation method according to claim 3, characterized in that, Determining the initial rotation angle and rotation step includes: Determine the different initial rotation angles corresponding to each of the projection planes and the same rotation step for all of the projection planes; Accordingly, the step of performing three-dimensional volume rendering on the three-dimensional ultrasound volume data based on the initial rotation angle and the rotation step to generate an ultrasound video containing multiple rendered images includes: Based on the initial rotation angle and rotation step corresponding to each of the projection planes, the three-dimensional ultrasound volume data is rendered in three dimensions to generate an ultrasound image containing multiple rendered images corresponding to each of the projection planes. Accordingly, the projection surface of the holographic projection device includes a first projection surface and at least one second projection surface; determining the projection image corresponding to each projection surface based on the rendered image includes: The rendered image in the ultrasound image corresponding to the first projection surface is determined as the projection image corresponding to the first projection surface. Based on the correspondence, a rotation matrix is ​​determined to rotate around the center point of the rendered image in the ultrasound image corresponding to each second projection plane. The rendered images in the ultrasound images corresponding to each second projection plane are rotated based on the rotation matrix to generate the projected images corresponding to each second projection plane.

6. The holographic video generation method according to claim 2, characterized in that, The step of acquiring three-dimensional ultrasound volume data and performing three-dimensional volume rendering on the three-dimensional ultrasound volume data to generate an ultrasound image containing the rendered image includes: Acquire multiple three-dimensional ultrasonic volume data and determine the target rotation angle; Based on the target rotation angle, three-dimensional volume rendering is performed on multiple three-dimensional ultrasound volume data to generate an ultrasound image containing a rendered image corresponding to each of the three-dimensional ultrasound volume data.

7. The holographic video generation method according to claim 6, characterized in that, Determining the target rotation angle includes: Determine the same target rotation angle corresponding to all the projection surfaces; Accordingly, the projection surface of the holographic projection device includes a first projection surface and at least one second projection surface; determining the projection image corresponding to each projection surface based on the rendered image includes: The rendered image is determined to be the projection image corresponding to the first projection surface; Based on the correspondence, a rotation matrix is ​​determined to rotate around the center point of the rendered image; The projection image corresponding to the first projection surface is rotated based on the rotation matrix to generate projection images corresponding to each of the second projection surfaces.

8. The holographic video generation method according to claim 6, characterized in that, Determining the target rotation angle includes: Determine the different target rotation angles corresponding to each of the projection planes; Accordingly, the step of performing three-dimensional volume rendering on multiple three-dimensional ultrasound volume data based on the target rotation angle to generate an ultrasound image containing a rendered image corresponding to each of the three-dimensional ultrasound volume data includes: Based on the target rotation angle corresponding to each of the projection planes, three-dimensional volume data of multiple three-dimensional ultrasound volumes are rendered to generate an ultrasound image containing multiple rendered images corresponding to each of the projection planes. Accordingly, the projection surface of the holographic projection device includes a first projection surface and at least one second projection surface; determining the projection image corresponding to each projection surface based on the rendered image includes: The rendered image in the ultrasound image corresponding to the first projection surface is determined as the projection image corresponding to the first projection surface. Based on the correspondence, a rotation matrix is ​​determined to rotate around the center point of the rendered image in the ultrasound image corresponding to each second projection plane. The rendered images in the ultrasound images corresponding to each second projection plane are rotated based on the rotation matrix to generate the projected images corresponding to each second projection plane.

9. The holographic video generation method according to claim 2, characterized in that, Performing three-dimensional volume rendering on the three-dimensional ultrasound volume data to generate an ultrasound image containing the rendered image includes: A transformation matrix is ​​constructed, and the three-dimensional ultrasound volume data is rendered in three dimensions based on the transformation matrix to generate an ultrasound image containing the rendered image. The rendered image in the ultrasound image is preprocessed to fit the size of the projection screen of the holographic projection device.

10. The holographic video generation method according to claim 9, characterized in that, The construction of the transformation matrix includes: A transformation matrix is ​​constructed based on the display height of the three-dimensional ultrasonic volume data; Accordingly, the rendered image in the ultrasound image is preprocessed to fit the size of the projection screen of the holographic projection device, including: Calculate the image scaling factor based on the dimensions of the three-dimensional ultrasonic volume data and the dimensions of the projection screen; The rendered image is scaled based on the image scaling factor to fit the size of the projection screen of the holographic projection device.

11. The holographic video generation method according to claim 9, characterized in that, The construction of the transformation matrix includes: The target rendering image size is calculated based on the resolution of the holographic projection device and the display height and display width of the three-dimensional ultrasonic volume data, and a transformation matrix is ​​constructed based on the target rendering image size; Accordingly, the rendered image in the ultrasound image is preprocessed to fit the size of the projection screen of the holographic projection device, including: The rendered image in the ultrasound image is cropped to fit the size of the projection screen of the holographic projection device.

12. The holographic video generation method according to claim 1, characterized in that, After acquiring the ultrasound image, the process also includes: Obtain the video playback configuration of the holographic projection device; When the video playback is configured for inverted playback, the images in the ultrasound images are flipped vertically.

13. A holographic video generation device, characterized in that, include: The first acquisition module is used to acquire ultrasound images; The first determining module is used to determine the correspondence between each projection surface of the holographic projection device and each projection area in the holographic projection plane area; The second determining module is used to determine the projection image corresponding to each projection plane based on the ultrasound image; The splicing module is used to place the projected image corresponding to each of the projection surfaces into the projection area corresponding to the holographic projection plane area according to the correspondence, so as to splice the holographic projection image. The generation module is used to generate a holographic video based on the holographic projection image; Specifically, the first determining module is used to: calculate a screen scaling factor based on the resolution of the holographic projection device or the resolution of the image in the ultrasound image; calculate the size of the projection screen based on the resolution of the holographic projection device and the screen scaling factor; calculate the position coordinates of each projection area in the holographic projection plane area based on the resolution of the holographic projection device and the size of the projection screen; and determine the correspondence between each projection surface of the holographic projection device and the position coordinates of each projection area in the holographic projection plane area. Wherein, if the ultrasound image includes at least one two-dimensional ultrasound image, the second determining module is specifically used to: determine the two-dimensional ultrasound image in the ultrasound image as the projection image corresponding to the first projection surface; the projection surface of the holographic projection device includes a first projection surface and at least one second projection surface; determine a rotation matrix around the center point of the two-dimensional ultrasound image based on the correspondence; and perform rotation processing on the projection image corresponding to the first projection surface based on the rotation matrix to generate projection images corresponding to each of the second projection surfaces; If the ultrasound image includes a rendered image obtained by rendering three-dimensional ultrasound volume data in three dimensions, then the second determining module is specifically used to: determine the projection image corresponding to each projection surface based on the rendered image.

14. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the holographic video generation method as described in any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the holographic video generation method as described in any one of claims 1 to 12.

Images