Display device, method, electronic device, and program product

Abstract

The invention provides a display device and method, electronic equipment and a program product, the display device is applied to a ship, the display device comprises an image acquisition system, a human eye recognition system, a processing system and a display, and the display is arranged between two pieces of adjacent window glass of the ship; the image acquisition system is used for acquiring an environment image around the ship in the ship running process and sending the environment image to the processing system; the human eye recognition system is used for capturing position information of a human eye focusing display and sending the position information to the processing system; the processing system is used for determining a blind area of the area between the two adjacent window glass according to the position information, and projecting the blind area to the environment image so as to segment a target area image matched with the blind area from the environment image; the display is used for displaying the target area image. According to the invention, a user can more truly and conveniently observe a blind area picture at a ship window frame, so that a ship cab can obtain a better visual field, and the ship navigation safety level is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of marine vessels, and in particular to a display device, method, electronic equipment and program product. Background Art

[0002] The International Maritime Organization has set regulations for the blind spots of ship cabs. On some ships that require a higher level of automation in the cab, the window frames of the cab will also be defined as the blind spots of the cab. When the blind spots are too large, on the one hand, it is not conducive to the cab driver's observation, and on the other hand, it also poses a threat to navigation safety.

[0003] In order to reduce the blind spots caused by window frames, ship designers will consider compressing the size of window frames and reducing the number of window frames. However, due to the consideration of the strength of window frames and the supporting strength of window glass, the size of window frames cannot be reduced indefinitely, so blind spots will inevitably exist.

[0004] Currently, some ships are using CCTV to transmit images outside the ship to the cab to improve the cab's field of view and enhance the visual monitoring capability of the sea surface. However, the images transmitted by this technical route are presented on traditional displays, which cannot be directly arranged on the window glass of the cab and can only be placed on the console, chart area or above the window, making it inconvenient to observe. Summary of the invention

[0005] The technical problem to be solved by the present disclosure is to overcome the defect that the viewing angle of the traditional display in the cab in the prior art is relatively fixed and it is inconvenient to observe the blind area, and to provide a display device, method, electronic equipment and program product.

[0006] The present invention solves the above technical problems through the following technical solutions:

[0007] In a first aspect, a display device is provided, which is applied to a ship, and comprises an image acquisition system, a human eye recognition system, a processing system and a display, wherein the display, the image acquisition system and the human eye recognition system are all connected to the processing system; the display is arranged between two adjacent window glasses of the ship;

[0008] The image acquisition system is used to acquire environmental images around the ship during the travel of the ship, and send the environmental images to the processing system;

[0009] The human eye recognition system is used to capture the position information of the human eye focusing on the display, and send the position information to the processing system;

[0010] The processing system is used to determine the blind area of ​​the area between the two adjacent window glasses viewed by the human eye according to the position information, and project the blind area onto the environment image to segment a target area image matching the blind area from the environment image;

[0011] The display is used to display the target area image.

[0012] Optionally, the display device further comprises shutter glasses for viewing the target area image through the display;

[0013] When the processing system determines that there are at least two eye focus points on the display according to the position information, the processing system sends the target area images matching the position information of each eye to the display at intervals, and synchronously sends a switch signal to the shutter glasses at the positions corresponding to the position information of each eye;

[0014] The shutter glasses are used to open and close according to the switch signal to filter the target area image corresponding to the position.

[0015] Optionally, the position information includes a distance relationship between a human eye and a display, and the processing system is further used to adjust the clarity of the display according to the distance relationship between the human eye and the display, and the clarity of the display is positively correlated with the distance relationship between the human eye and the display.

[0016] Optionally, the display is a flexible display, which is arranged at a window frame between adjacent window glasses of a ship, and a shape of the flexible display is consistent with a shape of the window frame, and the flexible display covers the window frame; the processing system corrects the target area image according to the shape of the flexible display.

[0017] Optionally, light sensors are arranged around the display, and the light sensors are connected to the processing system. The light sensors monitor the indoor light intensity in real time and send the monitoring results to the processing system. The processing system adjusts the display brightness according to the monitoring results.

[0018] A second aspect provides a display method, which is implemented by using any of the display devices described above, and includes:

[0019] Collecting environmental images around the ship during the ship's travel;

[0020] Capture the position information between the human eye and the display;

[0021] Determine a blind area of ​​the area between the two adjacent window glasses when the human eye is looking at the area according to the position information, and project the blind area onto the environment image to segment a target area image matching the blind area from the environment image;

[0022] The target area image is displayed.

[0023] Optionally, the display method also includes sending target area images matching the position information of different human eyes to the display at intervals and synchronously sending a switch signal when it is determined that there are at least two human eye focusing points on the display based on the position information; and turning on and off according to the switch signal to filter the target area images corresponding to the position information.

[0024] Optionally, the position information includes a distance relationship between a human eye and a display, and the display method further includes adjusting the clarity of the display according to the distance relationship between the human eye and the display, and the clarity of the display is positively correlated with the distance relationship between the human eye and the display.

[0025] A third aspect provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and used to run on the processor, wherein the processor implements the above-mentioned display method when executing the computer program.

[0026] A fourth aspect provides a computer program product, including a computer program, wherein when the computer program is executed by a processor, the display method described above is implemented.

[0027] On the basis of being in accordance with the common sense in the art, the above-mentioned preferred conditions can be arbitrarily combined to obtain the preferred embodiments of the present disclosure.

[0028] The positive progressive effect of the present disclosure is that the display is set at the window frame between adjacent window glasses of the ship, the blind spot of the area between the two adjacent window glasses when the human eye is looking at it is determined according to the position information of the human eye recognition system, and the blind spot is projected onto the environmental image to segment the target area image matching the blind spot from the environmental image, and the image is compensated for the blind spot of the window frame, so that the user can observe the blind spot image at the window frame of the ship more realistically and conveniently, so that the ship's cab has a better field of view, and the ship's navigation safety level is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A schematic diagram of a display device according to an exemplary embodiment of the present disclosure is shown in FIG.

[0030] Figure 2 A schematic diagram of an installation position of a display provided by an exemplary embodiment of the present disclosure,

[0031] Figure 3A schematic diagram of an application scenario of a human eye recognition system provided by an exemplary embodiment of the present disclosure is shown in FIG.

[0032] Figure 4 An exemplary embodiment of the present disclosure provides a flowchart of a display method,

[0033] Figure 5 The present invention is a schematic structural diagram of an electronic device according to an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0034] The present disclosure is further described below by way of examples, but the present disclosure is not limited to the scope of the examples.

[0035] Prefixes such as "first" and "second" are used in the embodiments of the present disclosure only to distinguish different description objects, and have no limiting effect on the position, order, priority, quantity or content of the described objects. The use of prefixes such as ordinal numbers to distinguish description objects in the embodiments of the present disclosure does not constitute a limitation on the described objects. For the statement of the described objects, please refer to the description in the context of the claims or embodiments, and no unnecessary limitation should be constituted due to the use of such prefixes. In addition, in the description of the present embodiment, unless otherwise specified, the meaning of "plurality" is two or more.

[0036] Figure 1 Schematic diagram of a display device provided by an exemplary embodiment of the present disclosure, the display device is applied to a ship, the display device comprises an image acquisition system 1, a human eye recognition system 2, a processing system 3 and a display 4, the display 4, the image acquisition system 1 and the human eye recognition system 2 are all connected to the processing system 3; Figure 2 As described above, the display 4 is disposed between two adjacent window glasses of the ship's cockpit (gray filled area).

[0037] The image acquisition system 1 is used to acquire environmental images around the ship during the ship's travel and send the environmental images to the processing system. The image acquisition system includes a plurality of high-definition wide-angle cameras arranged outside the ship's wheelhouse, and optionally, cameras are arranged at the edges of the left and right wing bridges of the wheelhouse and on both sides of the top of the wheelhouse, respectively, for acquiring environmental images around the ship.

[0038] The human eye recognition system 2 is used to capture the position information of the human eye focusing on the display and send the position information to the processing system. Specifically, a camera with human eye tracking function is set inside the cab to track the position of the driver in the driving position. Figure 3As shown, when the human eye recognition system recognizes the position information of the driver's human eye focusing on the display 4, the obtained position information is sent to the processing system. The position information includes but is not limited to the distance between the person and the display, the position of the human eye focus point, the relationship between the focus point position and the center position of the display, the human eye observation angle, the depth of field information, etc.

[0039] The processing system 3 is used to determine the blind area of ​​the area between two adjacent window glasses when the human eye is looking at it according to the position information, and project the blind area onto the environment image to segment a target area image matching the blind area from the environment image.

[0040] Specifically, the entire cab is first modeled in 1:1 3D. After the 3D modeling is completed, the overall coordinates of the windows and edges in the entire cab are generated, and the environmental image collected by the image acquisition system is spliced ​​with the above coordinate information. Then, according to the position information provided by the human eye recognition system, the blind area (window frame) of the area between two adjacent window glasses is determined, and the blind area is projected onto the environmental image, the environmental image is segmented, and the target area images matching the blind area are generated respectively.

[0041] The display 4 then performs corresponding display according to the target area image sent by the processing system.

[0042] In this embodiment, the display is set at the window frame between adjacent window glasses of the ship, and the blind area of ​​the area between the two adjacent window glasses when the human eye is looking at it is determined according to the position information of the human eye recognition system, and the blind area is projected onto the environmental image to segment the target area image matching the blind area from the environmental image, and the image compensation is performed on the blind area of ​​the window frame, so that the user can observe the blind area image at the window frame of the ship more realistically and conveniently, so that the ship's cab has a better field of view, thereby improving the safety level of ship navigation.

[0043] In one embodiment, the display device further comprises shutter glasses for viewing the target area image through the display;

[0044] When the processing system determines that there are at least two eye focus points on the display according to the position information, the processing system sends the target area images matching the position information of each eye to the display at intervals, and synchronously sends a switch signal to the shutter glasses at the positions corresponding to the position information of each eye;

[0045] The shutter glasses are used to open and close according to the switch signal to filter the image of the target area corresponding to the position.

[0046] Specifically, the indoor human eye capture camera collects the position information of multiple people in real time, such as person A and person B. The processing system simultaneously calculates the target area images corresponding to person A and person B respectively. The cab personnel wear special glasses, which are shutter-type. When two people look at the same display at the same time, the processing system displays the target area images of person A and person B at intervals frame by frame, and simultaneously sends a switch signal to the shutter glasses corresponding to the position information of each person's eyes. Person A's glasses only receive the A frame image, and when the B frame image is displayed, the glasses are instantly closed. Similarly, when the B frame image is displayed, the A glasses are instantly closed. A display with a high refresh rate can be used. Due to the high refresh rate, the human eye will not feel that the glasses are closed, thereby solving the problem of single or multiple use. When two people look at displays at different display positions, the shutter glasses can remain open to display the target area image corresponding to the person.

[0047] In this embodiment, through the shutter glasses, when there are multiple people in the cab looking at the same display, the target area image corresponding to the eye position information of the corresponding person can be displayed according to the different eye position information, thereby improving the experience of multiple people observing the blind spot at the same time.

[0048] In one embodiment, the position information includes the distance relationship between the human eye and the display. The processing system is further used to adjust the clarity of the display according to the distance relationship between the human eye and the display. The clarity of the display is positively correlated with the distance relationship between the human eye and the display.

[0049] Specifically, when a person moves in the cab, the human eye recognition system will collect the distance between the human eye and the display in real time. The processing system will control the equipment of the image acquisition system according to the distance between the human eye and the display, such as the image acquisition direction of each camera in the image acquisition system, zoom the focal length of the camera, and reassemble the environmental image. When a person is close to the display, the image in the direction of the display can be enlarged, and when the person is far away from the display, the image in the direction of the display can be reduced, thereby adjusting the clarity of the display in real time.

[0050] In this embodiment, the human eye recognition system is used for real-time monitoring and the position information is sent to the processing system. The processing system can adjust the image acquisition system according to the position information, thereby achieving different display effects when observing from different positions, which is more in line with the observation habits of the human eye and improves the user experience.

[0051] In one embodiment, the display is a flexible display, which is arranged at the window frame between adjacent window glasses of the ship. The shape of the flexible display is consistent with the shape of the window frame, and the flexible display covers the window frame. The processing system corrects the target area image according to the shape of the flexible display.

[0052] Specifically, a flexible high-definition special-shaped display screen is set at the window frame position of the cab. The shape of the display screen is consistent with the shape of the window frame, and the two sides extend to the edge of the window glass to ensure that the glass and the glass can be covered by the flexible screen lock. The reason for using a flexible screen is that the window frame of the cab and the window glass of the cab are not in the same plane, and the traditional hard screen cannot achieve a good fit. The shape of the flexible screen of each window is three-dimensionally modeled, and the processing system corrects the image of the target area according to the above three-dimensional modeling data, so that the image displayed on the flexible screen is from the perspective of the driver's position looking outdoors, thereby forming a complete fit with the field of view of the window glass, and visually achieving the effect of invisible window frames.

[0053] In this embodiment, by modeling the flexible anisotropic display screen and correcting the target area image according to the modeling data, the driver's perspective of looking outdoors is perfectly aligned with the field of view of the window glass, visually achieving the effect of invisible window frame and improving the driver's observation experience.

[0054] In one embodiment, light sensors are arranged around the display, and the light sensors are connected to the processing system. The light sensors monitor the indoor light intensity in real time and send the monitoring results to the processing system, and the processing system adjusts the display brightness according to the monitoring results.

[0055] Specifically, light sensors are installed around the cab or display to collect light intensity information around the cab or display, and send this information to the processing system, which controls the brightness of the display according to the above information. In this way, the corresponding display brightness can be provided regardless of day or night, so that the cab personnel can observe the corresponding blind spot information more clearly.

[0056] Corresponding to the aforementioned display device embodiment, the present disclosure also provides an embodiment of a display method.

[0057] like Figure 4 The figure shows a flow chart of a display method provided by the present disclosure. The display method is implemented by any of the display devices described above. The display method includes:

[0058] Step 101: Collect environmental images around the ship while the ship is traveling.

[0059] Step 102: Capture the position information between the human eye and the display.

[0060] Step 103 : determining a blind area of ​​the area between two adjacent window glasses when the human eye is looking at it according to the position information, and projecting the blind area onto the environment image to segment a target area image matching the blind area from the environment image.

[0061] Step 104: Display the target area image.

[0062] Specifically, during the ship's travel, the environmental image around the ship is collected, and the position information of the human eye focusing on the display is captured, and a 1:1 3D model of the entire cab is built. After the 3D modeling is completed, the overall coordinates of the entire window and the edge of the entire cab are generated, and the collected environmental image is spliced ​​through the above coordinate information. Then, based on the position information, the blind area (window frame) of the area between two adjacent window glasses is determined, and the blind area is projected onto the environmental image, the environmental image is segmented, and target area images matching the blind area are generated respectively, and then the above target area images are displayed accordingly.

[0063] In this embodiment, the display is set at the window frame between adjacent window glasses of the ship, and the blind area of ​​the area between the two adjacent window glasses when the human eye is looking at it is determined according to the position information of the human eye recognition system, and the blind area is projected onto the environmental image to segment the target area image matching the blind area from the environmental image, and the image compensation is performed on the blind area of ​​the window frame, so that the user can observe the blind area image at the window frame of the ship more realistically and conveniently, so that the ship's cab has a better field of view, thereby improving the safety level of ship navigation.

[0064] In one embodiment, the display method also includes sending target area images matching the position information of different human eyes to the display at intervals and synchronously sending a switch signal when it is determined based on the position information that there are at least two human eye focusing points on the display; turning on and off according to the switch signal to filter the target area images corresponding to the position information.

[0065] Specifically, the position information of multiple persons, such as person A and person B, is collected in real time. At the same time, the target area images corresponding to person A and person B are calculated. When two people look at the same display at the same time, the target area images of person A and person B are displayed frame by frame, and the switch signal is synchronously sent to the shutter glasses corresponding to the position information of each person's eyes. Person A's glasses only receive the A frame image, and the glasses are instantly closed when the B frame image is displayed. Similarly, when the B frame image is displayed, the A glasses are instantly closed.

[0066] In this embodiment, when there are multiple people in the cab looking at the same display, the target area image corresponding to the eye position information of the corresponding person can be displayed according to different eye position information, thereby improving the experience of multiple people observing the blind spot at the same time.

[0067] In one embodiment, the position information includes the distance relationship between the human eye and the display, and the display method further includes adjusting the clarity of the display according to the distance relationship between the human eye and the display, and the clarity of the display is positively correlated with the distance relationship between the human eye and the display.

[0068] Specifically, when a person moves in the cab, the distance between the human eye and the display is collected in real time, and the image acquisition system equipment is controlled according to the distance between the human eye and the display, such as the image acquisition direction of each camera in the image acquisition system, the focal length of the camera is scaled, and the environmental image is reassembled. When a person is close to the display, the image in the direction of the display can be enlarged, and when the person is far away from the display, the image in the direction of the display can be reduced, thereby adjusting the clarity of the display in real time.

[0069] In this embodiment, the distance relationship between the human eye and the display is collected in real time, and the image acquisition system is adjusted according to the distance relationship between the human eye and the display, so as to achieve different display effects when observing from different positions, which is more in line with the observation habits of the human eye and improves the user experience.

[0070] As for the display method embodiment, since it basically corresponds to the display device embodiment, the relevant parts can refer to the partial description of the display device embodiment. The display device embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the disclosed solution.

[0071] Figure 5 This is a structural schematic diagram of an electronic device shown in an exemplary embodiment of the present disclosure, the electronic device includes a memory, a processor, and a computer program stored in the memory and used to run on the processor, and the processor implements the display method described in any of the above embodiments when executing the computer program. Figure 5 The electronic device 50 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0072] like Figure 5 As shown, the electronic device 50 may be in the form of a general-purpose computing device, for example, it may be a server device. The components of the electronic device 50 may include, but are not limited to: at least one processor 51, at least one memory 52, and a bus 53 connecting different system components (including the memory 52 and the processor 51).

[0073] The bus 53 includes a data bus, an address bus, and a control bus.

[0074] The memory 52 may include a volatile memory, such as a random access memory (RAM) 521 and / or a cache memory 522 , and may further include a read-only memory (ROM) 523 .

[0075] The memory 52 may also include a program tool 525 (or utility) having a set (at least one) of program modules 524, such program modules 524 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0076] The processor 51 executes various functional applications and data processing by running the computer program stored in the memory 52, such as the display method provided in any of the above embodiments.

[0077] The electronic device 50 may also communicate with one or more external devices 54 (e.g., keyboards, pointing devices, etc.). Such communication may be performed via an input / output (I / O) interface 55. Furthermore, the electronic device 50 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 56. As shown, the network adapter 56 communicates with other modules of the electronic device 50 via a bus 53. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 50, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems, etc.

[0078] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into multiple units / modules to be embodied.

[0079] The embodiments of the present disclosure further provide a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the display method provided by any of the above embodiments is implemented.

[0080] The readable storage medium may include but is not limited to: a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device or any suitable combination of the above.

[0081] The embodiment of the present disclosure further provides a computer program product, including a computer program, wherein when the computer program is executed by a processor, the display method described in any one of the above items is implemented.

[0082] Among them, the program code for executing the computer program product of the present disclosure can be written in any combination of one or more programming languages, and the program code can be executed completely on the user device, partially on the user device, as an independent software package, partially on the user device and partially on a remote device, or completely on the remote device.

[0083] Although the specific embodiments of the present disclosure are described above, those skilled in the art should understand that this is only an example, and the protection scope of the present disclosure is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present disclosure, but these changes and modifications all fall within the protection scope of the present disclosure.

Claims

1. A display device, characterized in that: The display device is applied to a ship, and comprises an image acquisition system, a human eye recognition system, a processing system and a display, wherein the display, the image acquisition system and the human eye recognition system are all connected to the processing system; the display is arranged between two adjacent window glasses of the ship; The image acquisition system is used to acquire environmental images around the ship during the travel of the ship, and send the environmental images to the processing system; The human eye recognition system is used to capture the position information of the human eye focusing on the display, and send the position information to the processing system; The processing system is used to determine the blind area of ​​the area between the two adjacent window glasses viewed by the human eye according to the position information, and project the blind area onto the environment image to segment a target area image matching the blind area from the environment image; The display is used to display the target area image.

2. The display device according to claim 1, characterized in that The display device further comprises shutter glasses for viewing the target area image through the display; When the processing system determines that there are at least two eye focus points on the display according to the position information, the processing system sends the target area images matching the position information of each eye to the display at intervals, and synchronously sends a switch signal to the shutter glasses at the positions corresponding to the position information of each eye; The shutter glasses are used to open and close according to the switch signal to filter the target area image corresponding to the position.

3. The display device according to claim 1, characterized in that The position information includes the distance relationship between the human eye and the display. The processing system is also used to adjust the clarity of the display according to the distance relationship between the human eye and the display. The clarity of the display is positively correlated with the distance relationship between the human eye and the display.

4. The display device according to claim 1, characterized in that The display is a flexible display, which is arranged at a window frame between adjacent window glasses of a ship, has a shape consistent with that of the window frame, and covers the window frame; the processing system corrects the target area image according to the shape of the flexible display.

5. The display device according to claim 1, characterized in that Light sensors are arranged around the display, and the light sensors are connected to the processing system. The light sensors monitor the indoor light intensity in real time and send the monitoring results to the processing system. The processing system adjusts the display brightness according to the monitoring results.

6. A display method, characterized in that: The display method is implemented by a display device according to any one of claims 1 to 5, and the display method comprises: Collecting environmental images around the ship during the ship's travel; Capture the position information between the human eye and the display; Determine a blind area of ​​the area between the two adjacent window glasses when the human eye is looking at the area according to the position information, and project the blind area onto the environment image to segment a target area image matching the blind area from the environment image; The target area image is displayed.

7. The display method according to claim 6, characterized in that: The display method further includes, when it is determined according to the position information that there are at least two eye focus points on the display, sending target area images matching the position information of different eyes to the display at intervals, and synchronously sending a switch signal; The switch is turned on or off according to the switch signal to filter the target area image corresponding to the position information.

8. The display method according to claim 6, characterized in that: The position information includes the distance relationship between the human eye and the display. The display method also includes adjusting the clarity of the display according to the distance relationship between the human eye and the display. The clarity of the display is positively correlated with the distance relationship between the human eye and the display.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and used to run on the processor, characterized in that: When the processor executes the computer program, the display method according to any one of claims 6 to 8 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the display method according to any one of claims 6 to 8 is implemented.

Images