An image processing method

Abstract

The present application discloses an image processing method. The image processing method is used to display an image on an ambient display screen, including: obtaining a preset character in the image to be displayed to form a character layer; obtaining a background layer of the image to be displayed, the background layer includes objects other than the preset character in the image to be displayed, and the background layer has the same aspect ratio as the ambient display screen; setting the preset features of the selected preset character in response to a preset operation to form an interactive feedback layer; stacking and integrating the background layer, the character layer, and the interactive feedback layer in sequence to form an image and display it on the ambient display screen. The present application can improve the immersive experience and interactive realism of teaching modes such as online education platforms and VR education technologies.

Description

Technical Field

[0001] The present application relates to the field of images and human-computer interaction, and specifically to an image processing method. Background Art

[0002] The traditional teaching method of teachers explaining and writing on the blackboard is gradually no longer suitable for the current teaching environment and teaching needs. In order to adapt to this, new teaching methods that use tablets, touch-screen all-in-ones and other devices and carry various teaching apps are gradually becoming popular according to different teaching scenarios. There are two main types of new teaching methods: one is a quasi-online education platform; the other is VR education technology. The so-called quasi-online education platform can be understood as: through the servers, tablets or touch-screen all-in-ones set up on campus, and relying on the campus network and various teaching apps and background data, to meet the needs of teachers and students for lesson preparation, explanation and homework arrangement. Based on VR education technology, wearable devices or some special display devices (such as cockpit-style devices with screens) are used to allow students to explore and interact in a simulated digital space.

[0003] However, the interactive capabilities of online education platforms are limited, making it difficult to provide participants (especially students) with an immersive experience. VR education technology relies on wearable devices, which are not only bulky and lead to a limited user experience, but the scenes are completely virtualized, making it difficult for participants to gain a sense of realism in the interaction. Summary of the invention

[0004] The embodiment of the present application provides an image processing method to improve the immersive experience and interactive realism of teaching modes such as online education platforms and VR education technologies.

[0005] The embodiment of the present application provides an image processing method for displaying an image on an ambient display screen, comprising:

[0006] Acquire preset characters in the image to be displayed to form a character layer;

[0007] Acquire a background layer of the image to be displayed, the background layer including objects other than the preset characters in the image to be displayed, and the background layer has the same aspect ratio as the surround display screen;

[0008] In response to the preset operation, the preset characteristics of the selected preset role are set to form an interactive feedback layer;

[0009] The background layer, the character layer and the interactive feedback layer are stacked and integrated in sequence to form an image and displayed on the surround display screen.

[0010] Optionally, before displaying the image on the surround display screen, the image processing method further includes:

[0011] Rendering the image to be displayed to form a foreground layer;

[0012] The background layer, the character layer, the interactive feedback layer and the foreground layer are stacked and integrated in sequence to form an image.

[0013] Optionally, rendering the image to be displayed to form a foreground layer includes:

[0014] Perform a depth of field operation on the objects in the background layer to form the foreground layer.

[0015] Optionally, before displaying the image on the surround display screen, the image processing method further includes:

[0016] An information display layer is formed according to the associated information of the selected preset role;

[0017] The background layer, the character layer, the interactive feedback layer, the information display layer and the foreground layer are stacked and integrated in sequence to form an image.

[0018] Optionally, the preset features include at least one of the following: brightness, background color, outline color, and border color of the area.

[0019] Optionally, the preset operation is to rotate the area with preset characteristics in a clockwise or counterclockwise direction; the preset characteristics of the selected preset character are set in response to the preset operation, including: changing the relative position between the area with preset characteristics and the preset character along the rotation direction; detecting that the rotation stops, taking the preset character at the stop position of the area as the selected preset character, and integrating the preset characteristics with the preset character.

[0020] Optionally, before displaying the image on the surround display screen, the image processing method further includes:

[0021] According to the aspect ratio of the image to be displayed, the distance between at least one of the character layer, the interactive feedback layer and the information display layer and the frame of the ambient display screen is adjusted.

[0022] Optionally, the surround display screen includes a curved screen and / or a curved wall, and the surround display screen also includes a plurality of projectors, and the projectors are used to project the content to be displayed onto the curved screen and / or the curved wall;

[0023] Before displaying the image on the ring display screen, the image processing method further includes:

[0024] Obtain the image resolution of several projectors after fusion, and the aspect ratio of the surround display screen;

[0025] The image to be displayed is stretched and full-screen filled to obtain an intermediate image, and the intermediate image has the same resolution as the image after fusion of the projectors;

[0026] The intermediate image is compressed according to the aspect ratio of the surround display screen so that the compressed intermediate image corresponds to the aspect ratio of the surround display screen.

[0027] Optionally, the ring display screen includes a curved display screen that can emit light independently and display images;

[0028] Displaying an image on the surround display screen includes: controlling the curved display screen to display an image with a resolution that is the same as the resolution of the surround display screen.

[0029] Optionally, before displaying the image on the surround display screen, the image processing method further includes:

[0030] If the pixel length of the long side of the image is greater than or equal to 7560, the image is encoded as mov format; if the pixel length of the long side of the image is less than 7560, the image is encoded as mp4 format.

[0031] As described above, the embodiment of the present application forms an interactive feedback layer by setting the preset features of the selected preset character in response to the preset operation, and stacking and integrating the interactive feedback layer to form the image to be displayed in the surround display. It can be regarded as synchronously feeding back the operation information of users such as students in the surround display image, thereby realizing the interaction between the user and the preset characters in the content, and improving the immersive experience and the realism of the interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 and Figure 2 A schematic diagram of the layout of an ambient display scene in an embodiment of the present application;

[0033] Figure 3 A schematic diagram of a flow chart of an image processing method according to an embodiment of the present application;

[0034] Figure 4 A schematic diagram of a flow chart of an image processing method according to another embodiment of the present application;

[0035] Figure 5 A schematic diagram of various layers of an image according to an embodiment of the present application. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be clearly described below in conjunction with specific embodiments and corresponding drawings. Obviously, the embodiments described below are only part of the embodiments of the present application, not all of the embodiments. In the absence of conflict, the following embodiments and their technical features can be combined with each other and also belong to the technical solutions of the present application.

[0037] It should be understood that in the description of the embodiments of the present application, the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", etc. to indicate orientations or positional relationships are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the technical solutions and simplified descriptions of the corresponding embodiments of the present application, and do not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0038] The present application embodiment provides an image processing method, which can be regarded as a specification of the output format of a resource package or a course package, and at least needs to be applicable to Figure 1 and Figure 2 The surround display scene shown. Figure 1 and Figure 2 The illustrated surround display scene includes a surround display screen 21, which is provided with a ring-shaped display area (also referred to as a ring-shaped display area). In some scenes, the surround display screen 21 includes a curved screen (and / or a curved wall) 211, and the surround display screen 21 also includes a plurality of projectors 212, which are used to project the content to be displayed onto the curved screen 211, and the projection area of ​​the plurality of projectors 212 at least covers the ring-shaped display area.

[0039] In the ambient display scene, the gravity direction y can be regarded as the width direction of the ambient display screen 21, the length direction of the ambient display screen 21 is the perimeter on the xz section, the first direction x can be regarded as the length direction of the rectangular room 2 where the ambient display screen 21 is located, the third direction z can be regarded as the width direction of the rectangular room 2 where the ambient display screen 21 is located, and the gravity direction y can be regarded as the height direction of the rectangular room 2, which can also be called the second direction y. Any two of the first direction x, the second direction y and the third direction z are perpendicular. It should be noted that, due to the errors in actual processing or measurement, the so-called perpendicularity in the full text of this application does not require that the angle between the two must be 90°, but allows a deviation within a predetermined angle range (e.g., ±10°). For example, the so-called perpendicularity can be understood as the angle between any two directions is 80° to 100°.

[0040] like Figures 3 to 5 As shown, the image processing method includes the following steps S11 to S14.

[0041] S11: Acquire a preset character in the image to be displayed to form a character layer.

[0042] The preset characters can be regarded as specific objects for the teaching scene. For example, in the teaching scene of the bird course, the preset characters can be various birds. In this character layer, the image content can include character animation, that is, it includes both the creation of the preset characters and the production of the actions of the preset characters, such as the conventional bird behavior actions - at least one of walking, flying, swimming and hunting.

[0043] S12: Obtain a background layer of the image to be displayed, where the background layer includes objects other than the preset characters in the image to be displayed, and the background layer has the same aspect ratio as the surround display screen.

[0044] The background layer can be regarded as the background environment animation of the preset characters, which is mainly used to enhance the ambient display experience. In this background layer, the image content can include other objects besides the preset characters, such as the living environment of the birds and other objects in the living environment in the teaching scene of the bird course.

[0045] S13: In response to the preset operation, the preset characteristics of the selected preset role are set to form an interactive feedback layer.

[0046] The interactive feedback layer is used to provide feedback on the user's preset operation, which is specifically reflected by changing the preset characteristics of the preset character displayed in the image. Optionally, the preset characteristics include at least one of the following: brightness, background color, outline color, and border color of the area. For example, the selected preset character can be highlighted, which can be expressed as background animations such as aperture, highlight stroke, etc.

[0047] The preset operation can be specifically implemented by means of a turntable operation button, etc., and is represented in the image as rotating the area with preset features in a clockwise or counterclockwise direction. In response to the preset operation, the preset features of the selected preset character are set, including: changing the relative position between the area with the preset features and the preset character along the rotation direction; detecting that the rotation stops, taking the preset character at the position where the area stops as the selected preset character, and integrating the preset features with the preset character. For example, in the teaching scene of the bird course, the highlighted area (such as Figure 5 If bird A is selected when the rotation stops, bird A will be integrated with the highlighted area, i.e., highlighted.

[0048] S14: stacking and integrating the background layer, the character layer and the interactive feedback layer in sequence to form an image and display it on the surround display screen.

[0049] Optionally, before displaying the image on the surround display screen, the image processing method further includes step S131.

[0050] S131: forming an information display layer according to the associated information of the selected preset character. Here, the aforementioned step S14 should be understood as: stacking and integrating the background layer, the character layer, the interactive feedback layer and the information display layer in sequence to form an image and display it on the surround display screen.

[0051] The information display layer can be displayed in an animated form to expand detailed information for the selected preset role and hide the detailed information after being unselected. The detailed information includes but is not limited to a combination of one or more types of text, enlarged details, and charts.

[0052] Optionally, before displaying the image on the surround display screen, the image processing method further includes step S132.

[0053] S132: Render the image to be displayed to form a foreground layer.

[0054] Here, if Figure 4 As shown, the aforementioned step S14 can be understood as S141: stacking and integrating the background layer, the character layer, the interactive feedback layer, the information display layer and the foreground layer in sequence to form an image and display it on the surround display screen.

[0055] The foreground layer is mainly used to enhance the surround display experience, for example, to increase some dynamic details to make the picture more vivid, to create a depth of field effect, and to create animations that match preset characters. For creating a depth of field effect, depth of field operations are performed on objects in the background layer to form a foreground layer.

[0056] Considering the use of different classrooms, all images of resource packages or course packages that have been edited (hereinafter referred to as images) need to adapt to the requirements of the ring display screen 21. Based on this requirement, the edited images are divided into 5 layers of content. Taking the bird course actually used in the classroom as an example, the live viewing screen contains 5 layers of content. Among them, the middle 3 layers, namely the character layer, the interactive feedback layer, and the information display layer, will dynamically adjust the distance from the screen border based on the picture ratio to adapt to the ring display screen 21 with different aspect ratios. The background layer and the foreground layer will be captured according to the actual aspect ratio. Finally, the program end carried by the host 30 integrates these 5 layers of content and displays them on the ring display screen 21.

[0057] For the aforementioned image processing method, before displaying the image on the ambient display screen, optionally, it may also include: adjusting the distance between at least one of the character layer, the interactive feedback layer, and the information display layer and the ambient display screen frame according to the aspect ratio of the image to be displayed. In other words, the character layer, the interactive feedback layer, and the information display layer can be adaptively adjusted, for example, adjusting the display position of the preset character, feedback information, etc.

[0058] Based on the device used by the ring screen, the ring screen can be divided into two categories: projection ring screen and self-display ring screen. All exported content needs to be based on the characteristics of the ring screen, so that the left and right ends of the output content can be seamlessly spliced.

[0059] In some scenarios, the surround display screen includes a curved screen and / or a curved wall, and the surround display screen also includes a plurality of projectors, and the projectors are used to project the content to be displayed onto the curved screen and / or the curved wall;

[0060] Before displaying the image on the ring display screen, the image processing method further includes:

[0061] First, the image resolution after fusion of the projectors 212 and the aspect ratio of the surround display screen are obtained; then, the image to be displayed is stretched and full-screen filled to obtain an intermediate image, and the intermediate image has the same resolution as the image after fusion of the projectors; then, the intermediate image is compressed according to the aspect ratio of the surround display screen, so that the compressed intermediate image corresponds to the aspect ratio of the surround display screen.

[0062] That is, the projection ring screen uses multiple projectors to merge and splice to form a large desktop that is connected at the end. For example, the resolution of the fused picture is obtained by multiplying the actual resolution of the projector 212 (1920*1200 is taken as an example below) by the number of projectors 212. For example, the resolution of the fusion formed by 6 projectors 212 is 1920*6 (11520): 1200. However, due to the loss of pixels caused by actual fusion and compression of the projected picture, the actual output picture ratio needs to be based on the physical projection ratio. Taking the ring display screen 21 with a diameter of 6m and a height of 2.2m as an example, the aspect ratio of the ring display screen 21 is: ring circumference: projection height (6*3.14:2.2), that is, an aspect ratio of 8.56:1. When actually outputting the picture, it is necessary to consider the physical size as a reference, and output a picture with a height of not less than 1000 and a length greater than 8000. On the host side, the control program will process the playback content according to the above, and the picture matching the ring display screen 21 can be output.

[0063] In other scenarios, the surround screen includes a curved display screen that can emit light independently and display images; displaying images on the surround screen includes: controlling the curved display screen to display images with the same resolution as the surround screen. The self-display surround can be realized as a large screen that is seamlessly spliced ​​or point-to-point end-to-end.

[0064] Output video format

[0065] Before displaying the image on the surround display screen, the image processing method also includes: detecting the length and width pixels of the image (i.e., the content to be displayed in the resource package or course package); if the long side pixels of the image are greater than or equal to 7560, the image is encoded into a mov format using, for example, a hap pre-encoding method; if the long side pixels of the image are less than 7560, the image is encoded into an mp4 format using, for example, an h264 pre-encoding method.

[0066] When the aforementioned surround display scenario is applied to a teaching scenario, before the course starts, the teacher can connect to the host through the application of the control terminal (such as iPad) to turn on the computer, projector 212, fresh air system, etc., and control the brightness of the lights, such as setting the lighting illumination and fresh air temperature according to the requirements of different courses, and running the resource package of the corresponding course. After deploying the corresponding course settings, the teacher can quickly arrange the teaching environment according to the teaching aids used in the actual course, so that the hardware equipment can be connected to the corresponding resource package for use.

[0067] For scene display

[0068] The resource package or course package can be a scenario-based background material to meet the teacher's need to simulate and restore some scenes during the teaching process in order to achieve student immersion. The background material can be made into a static picture or a repeatable dynamic image in accordance with the 360° shooting specification (such as 10:1 cylindrical stitching). After the teacher sets up this type of course mode, the distribution of specific knowledge on the ring display screen 21 can be adjusted according to the actual environment needs, and specific key scenes can be inserted as materials into the normally played courseware. On the non-scenario-based courseware page, you can use split screen (such as three-quarter screen) to synchronously play the corresponding courseware content.

[0069] Teachers can control the playback and pause of courseware through mobile applications.

[0070] For exploratory learning courses

[0071] The resource package or course package can be developed and rendered using, for example, the unity system. For example, in the teaching scene of the natural science class, the bird classification course in the elementary school science course is first made into 8 different teaching cards (ie, wireless cards). After loading the course package, the interface displays a total of 8 different bird cards, and reminds that cards with different learning contents can be selected. Students can select different cards according to their interests, and then activate the corresponding bird's habitat scene and display it on the ring display screen 21. The image displayed on the ring display screen 21 can set 5-8 hidden knowledge points of different characters, and remind students to turn the dial to explore. Students can perform preset operations, select different birds on the wall (ie, the aforementioned preset characters), expand hidden information (which can be expressed as a combination of text, pictures, videos, etc.), and understand the knowledge points such as different body characteristics of this type of bird species due to environmental selection.

[0072] For 360° video playback

[0073] Course content can be selected through the cloud connected to the host 30 and loaded into the course package. Teachers can freely replace different video resources and play teaching videos by seeking materials. Through a 5-10 minute recorded teaching video or CG rendered video content, users can be put into an immersive theater for learning and exploration. 360 video playback can be regarded as a theater mode, which can provide students with a 360° VR theater experience.

[0074] Of course, the teaching content can also be presented in the form of interactive games. For example, based on learning needs, interactive props can be connected and courseware can be produced in combination with different scenarios, and loaded into the course package. In the aforementioned surround display environment, the connected interactive props can be used to achieve an individual or multiple people's wide-screen VR game experience.

[0075] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. For ordinary technicians in this field, all equivalent structural changes made using the contents of this specification and drawings are also included in the patent protection scope of the present application.

[0076] Although the terms "first, second", etc. are used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. In addition, the singular forms "one", "an", and "the" are intended to include plural forms as well. The terms "or" and "and / or" are interpreted as inclusive, or mean any one or any combination. Only when the combination of elements, functions, steps, or operations is inherently mutually exclusive in some way, an exception to this definition will occur.

[0077] In this article, although step codes such as S1 and S2 are used, their purpose is to express the corresponding content more clearly and concisely, and do not constitute a substantial limitation on the order. In specific implementation, S2 may be executed first and then S1, etc., which all fall within the scope of protection of this application.

Claims

1. An image processing method for displaying an image on a surround display screen, characterized in that: include: Acquire preset characters in the image to be displayed to form a character layer; Acquire a background layer of the image to be displayed, the background layer including objects other than the preset characters in the image to be displayed, and the background layer has the same aspect ratio as the ambient display screen; Performing a depth of field operation on the objects in the background layer to form a foreground layer; In response to a preset operation, a preset feature of the selected preset character is set to form an interactive feedback layer, wherein the preset feature is a highlight stroke, and the area with the preset feature is a highlight area; the preset operation is to rotate the area with the preset feature in a clockwise or counterclockwise direction through the turntable operation button; The setting of the preset feature of the selected preset character in response to the preset operation includes: changing the relative position between the area with the preset feature and the preset character along the rotation direction, and, upon detecting that the rotation stops, taking the preset character at the stop position of the area as the selected preset character, and integrating the preset feature with the preset character; An information display layer is formed according to the associated information of the selected preset role; The background layer, the character layer, the interactive feedback layer, the information display layer and the foreground layer are stacked and integrated in sequence to form an image with five layers of content and displayed on the surround display screen; wherein the information display layer hides the associated information after the preset character is unselected.

2. The image processing method according to claim 1, characterized in that: The preset features include at least one of the following: brightness, background color, outline color, and border color of the area.

3. The image processing method according to claim 1, characterized in that: Before displaying the image on the ring display screen, the image processing method further includes: According to the aspect ratio of the image to be displayed, the distance between at least one of the character layer, the interactive feedback layer and the information display layer and the frame of the ambient display screen is adjusted.

4. The image processing method according to claim 1, characterized in that: The surround display screen includes a curved screen and / or a curved wall, and the surround display screen also includes a plurality of projectors, and the projectors are used to project the content to be displayed onto the curved screen and / or the curved wall; Before displaying the image on the ring display screen, the image processing method further includes: Obtain the image resolution of several projectors after fusion, and the aspect ratio of the surround display screen; The image to be displayed is stretched and full-screen filled to obtain an intermediate image, wherein the intermediate image has the same resolution as the image after fusion of the plurality of projectors; The intermediate image is compressed according to the aspect ratio of the surround display screen, so that the compressed intermediate image corresponds to the aspect ratio of the surround display screen.

5. The image processing method according to claim 1, characterized in that: The ring display screen includes a curved display screen that can emit light independently and display images; Displaying an image on the surround display screen includes: controlling the curved display screen to display an image with a resolution that is the same as the resolution of the surround display screen.

6. The image processing method according to claim 1, characterized in that: Before displaying the image on the ring display screen, the image processing method further includes: If it is determined that the pixels of the long side of the image are greater than or equal to 7560, the image is encoded into the mov format; if it is determined that the pixels of the long side of the image are less than 7560, the image is encoded into the mp4 format.

Images