Display equipment, image processing method and device, electronic equipment and storage medium

Abstract

The invention discloses display equipment, an image processing method and device, electronic equipment and a storage medium. The image processing method comprises the following steps: identifying a viewpoint area of a user on an LCD screen; a backlight source of the LCD screen is provided with infrared light sources in at least one subarea; calling a camera combination corresponding to the viewpoint area to carry out image acquisition; wherein the camera combination comprises a color camera and / or a multispectral infrared camera; the collected image comprises at least one frame of color image collected by each camera; and according to the number of the infrared light source direction types, calling a corresponding method to perform normal vector reconstruction so as to perform three-dimensional reconstruction of the object based on the normal vector.

Description

Display device, image processing method and apparatus, electronic device, and storage medium Technical Field

[0001] The present disclosure relates to the field of three-dimensional imaging technology, and in particular to a display device, an image processing method and apparatus, an electronic device, and a storage medium. Background Art

[0002] Accurately capturing an object's three-dimensional information is crucial for its 3D reconstruction and display. Commonly used visual methods for 3D information capture include time-of-flight (TOF), structured light, and stereo vision. TOF suffers from low precision, while structured light suffers from system complexity and high cost. Binocular vision relies on feature point matching, resulting in incorrect depth in areas lacking texture or features, leading to errors in the resulting 3D reconstructed object.

[0003] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute prior art known to ordinary technicians in the field.

[0004] Summary of the Invention

[0005] According to a first aspect of the present disclosure, a display device is provided, comprising: an LCD screen, the LCD screen comprising a backlight source provided with a plurality of partitions; a visible light source provided in each partition; and an infrared light source arranged in at least one partition.

[0006] In an exemplary embodiment of the present disclosure, an infrared light source is arranged in each partition; the infrared light source of each partition has a different infrared band.

[0007] In an exemplary embodiment of the present disclosure, the infrared light sources of the subareas have different light source directions.

[0008] In an exemplary embodiment of the present disclosure, the partitions of the backlight source are arranged in a polygonal structure.

[0009] In an exemplary embodiment of the present disclosure, a multispectral infrared camera and / or a color camera is provided outside the LCD screen.

[0010] According to one aspect of the present disclosure, there is provided an image processing method, the method comprising:

[0011] Identifying a user's viewpoint area on an LCD screen; wherein the backlight source of the LCD screen is arranged with an infrared light source in at least one partition;

[0012] Calling a camera combination corresponding to the viewpoint area to perform image acquisition; wherein the camera combination includes: a color camera and / or a multispectral infrared camera; the acquired image includes at least one frame of color image acquired by each camera;

[0013] According to the number of types of infrared light source directions, a corresponding method is called to calculate and reconstruct the normal vector, so as to perform three-dimensional reconstruction of the object based on the normal vector.

[0014] In one exemplary embodiment of the present disclosure, the camera assembly includes a plurality of multispectral infrared cameras;

[0015] The method further comprises:

[0016] Extract image channel information from the collected multi-frame color images according to the band information of the infrared light source;

[0017] The normal vector is calculated using the photometric stereo algorithm based on the extracted image channel information.

[0018] In an exemplary embodiment of the present disclosure, the camera combination includes several color cameras and multispectral infrared cameras;

[0019] The method further comprises:

[0020] Extract image channel information from the collected multi-frame color images according to the band information of the infrared light source;

[0021] The normal vector is calculated using the photometric stereo algorithm based on the extracted image channel information.

[0022] In an exemplary embodiment of the present disclosure, the camera assembly includes a plurality of color cameras;

[0023] The method further comprises:

[0024] Calling the camera combination corresponding to the viewpoint area to perform image acquisition to obtain a color image; and acquiring a second image in infrared mode using other cameras;

[0025] extracting image channel information from the collected color image and the second image according to the band information of the infrared light source;

[0026] The normal vector is calculated using the shadow restoration algorithm based on the extracted image channel information.

[0027] According to one aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the image processing method described above is implemented.

[0028] According to one aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform any one of the above-mentioned image processing methods by executing the executable instructions.

[0029] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and those skilled in the art can derive other drawings based on these drawings without inventive effort. In the drawings:

[0031] FIG1 schematically shows a schematic diagram of an LCD screen structure according to an exemplary embodiment of the present disclosure;

[0032] FIG2 schematically shows a schematic diagram of an imaging optical path according to an exemplary embodiment of the present disclosure;

[0033] FIG3 schematically shows a schematic diagram of a backlight partition according to an exemplary embodiment of the present disclosure;

[0034] FIG4 schematically shows a schematic diagram of a camera arrangement in an exemplary embodiment of the present disclosure;

[0035] FIG5 schematically shows another camera arrangement in an exemplary embodiment of the present disclosure;

[0036] FIG6 schematically shows a schematic diagram of yet another camera arrangement in an exemplary embodiment of the present disclosure;

[0037] FIG7 schematically shows a schematic diagram of backlight partitioning and infrared light source distribution in an exemplary embodiment of the present disclosure;

[0038] FIG8 schematically shows a schematic diagram of an image processing method in an exemplary embodiment of the present disclosure;

[0039] FIG9 schematically shows a schematic diagram of an image processing apparatus in an exemplary embodiment of the present disclosure;

[0040] FIG10 schematically shows a schematic diagram of a program product in an exemplary embodiment of the present disclosure;

[0041] FIG11 schematically shows a diagram of an electronic device in an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0042] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0043] In addition, the accompanying drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. Identical reference numerals in the figures denote identical or similar parts, and thus repetitive descriptions thereof will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0044] In response to the shortcomings and deficiencies of the prior art, a display device is provided in this example embodiment, comprising: an LCD screen. Referring to FIG1 , the LCD screen comprises an upper polarizer 1, a glass substrate 2, a color filter 3, a liquid crystal molecule layer 4, a lower polarizer 5, and a backlight panel 6, which are arranged in order from the top layer to the bottom layer. The backlight panel can be divided into multiple areas. For example, it can be divided into at least three areas, and each area can be arranged in a polygonal structure. For example, referring to FIG3 , when setting up four partitions, area A and area C are set in the first direction, and area B and area D are set in the second direction; each partition is connected end to end to form a quadrilateral structure. In each partition, an infrared light source 7 and a visible light source 8 are arranged. The infrared light source uses infrared lamp beads to emit infrared light. The number of visible light sources is greater than the number of infrared light sources; the visible light sources are arranged to surround the infrared light sources. For each partition, the infrared band used in each partition is different. Referring to FIG2 , the light source direction of the infrared light source in each partition is different.

[0045] In this exemplary embodiment, a multispectral infrared camera and / or a color camera are positioned outside the LCD screen. For example, as shown in Figure 4 , five multispectral cameras can be evenly spaced above the screen. Alternatively, as shown in Figure 5 , seven cameras can be evenly spaced above the screen; camera four is a multispectral infrared camera, and the remaining cameras are color cameras. Alternatively, as shown in Figure 6 , seven color cameras can be evenly spaced above the screen.

[0046] In this exemplary embodiment, an image processing method is provided, which can be applied to the above-mentioned display device. Referring to FIG8 , the above-mentioned method may include:

[0047] Step S11, identifying a viewpoint area of ​​a user on an LCD screen; the backlight source of the LCD screen is provided with infrared light sources in at least one partition;

[0048] Step S12: calling a camera combination corresponding to the viewpoint area to perform image acquisition; wherein the camera combination includes: a color camera and / or a multispectral infrared camera; the acquired image includes at least one frame of color image acquired by each camera;

[0049] Step S13 , calling a corresponding method to calculate and reconstruct a normal vector according to the number of types of infrared light source directions, so as to perform three-dimensional reconstruction of the object based on the normal vector.

[0050] The image processing method provided in this example embodiment is achieved by partitioning the backlight source of the LCD screen to set infrared light sources of different bands and directions. When acquiring images, a color camera and / or a multispectral infrared camera can be used to acquire images, thereby obtaining RGB channels and image channel information corresponding to the infrared bands, and then different algorithms can be used to reconstruct normal vectors.

[0051] Below, each step of the light field display method in this exemplary embodiment will be described in more detail with reference to the accompanying drawings and embodiments.

[0052] In step S11, a user's viewpoint area on an LCD screen is identified; the backlight source of the LCD screen is arranged with infrared light sources in at least one partition;

[0053] In this exemplary embodiment, an eye tracking camera may be provided on the LCD screen to identify a viewpoint area where the user's eyes are paying attention in front of the LCD screen.

[0054] In step S12, the camera combination corresponding to the viewpoint area is called to perform image acquisition; wherein the camera combination includes: a color camera and / or a multispectral infrared camera; the acquired image includes at least one frame of color image acquired by each camera;

[0055] In this example implementation, the screen can be divided into two areas, namely Area A and Area B, as shown in Figure 4. Each area can be pre-configured with a corresponding camera combination. For example, the camera combination corresponding to Area A may include Camera 1, Camera 2, and Camera 3; the camera combination corresponding to Area B may include Camera 3, Camera 4, and Camera 5. After identifying the user's viewpoint area, each camera in the corresponding camera combination can be activated to capture an image.

[0056] In step S13, according to the number of types of infrared light source directions, a corresponding method is called to calculate and reconstruct normal vectors, so as to perform three-dimensional reconstruction of the object based on the normal vectors.

[0057] In this example embodiment, the camera combination includes several multispectral infrared cameras;

[0058] The method further comprises: extracting image channel information from the collected multiple frames of color images according to the wavelength information of the infrared light source;

[0059] The normal vector is calculated using the photometric stereo algorithm based on the extracted image channel information.

[0060] Specifically, as shown in FIG4 , five multispectral cameras are used; the partitions of the backlight panel are shown in FIG3 , including four infrared light sources; and the corresponding reconstruction is performed using a photometric stereo normal vector.

[0061] The camera uses a multispectral filter array camera to simultaneously capture infrared light sources and color images in different wavelength bands. A schematic diagram of the viewer is shown in Figure 4. When the viewer's viewpoint is in area A, cameras 1, 2, and 3 can be selected as acquisition cameras; when in area B, cameras 4, 5, and 6 can be selected as acquisition cameras. The color images from the selected cameras and the four-band infrared images are separated based on wavelength information. To synthesize the viewpoint between cameras 1 and 2, stereo vision principles are first used to obtain the disparity images of cameras 1 and 2. Mismatched points in the disparity images are then removed using a stereo vision algorithm. The infrared images of each wavelength obtained by cameras 1 and 2 are then combined using the principle of photometric stereo to determine the surface normal vectors of the object in the coordinate systems of cameras 1 and 2. The normal vectors from cameras 1 and 2 are fused separately to compensate for parallax holes caused by occlusion and weak textures, resulting in an accurate disparity image. The viewpoint is then synthesized using a forward / inverse method. The same process is used for other viewpoints.

[0062] The photometric stereo method can obtain the normal vector of an object based on multi-angle illumination images and illumination vectors. The target can be approximated as a Lambert body, and the normal vector can be obtained by the following formula:

[0063] Among them, N(i,j) is the normal vector of the object at the actual point corresponding to I(i,j), L is the illumination vector corresponding to the actual point corresponding to I(i,j), and ρ is the reflectivity of the object surface.

[0064] Reconstruction based on photometric stereo requires the acquisition of multiple images. A multispectral camera is used to simultaneously capture a color image of the object and infrared images of different wavelengths. Light of different wavelengths is separated by wavelength. The color image is used to capture the object's light field information, while the multiple infrared images are used to reconstruct the object's normal vector using photometric stereo. The multispectral camera can be a pixelated multispectral filter array, which enables single-exposure multi-channel imaging.

[0065] In this example embodiment, the camera combination includes several color cameras and a multi-spectral infrared camera.

[0066] The method further includes: extracting image channel information from the collected multiple frames of color images based on the band information of the infrared light source; the partitioning of the backlight panel is shown in Figure 3, including four infrared light sources; and calculating the normal vector using a photometric stereo algorithm based on the extracted image channel information.

[0067] Specifically, when the screen is operating, all infrared and white light sources are illuminated. Intermediate camera 4 uses a multispectral infrared camera, which, together with the infrared light source, forms a photometric stereo vision system. After acquiring the image, wavelength information is used to separate the infrared images of each band, and the surface normal vector of the object is determined based on the principle of photometric stereo.

[0068] [Corrected 07 / 08 / 2023 according to Rule 91] Cameras 1, 2, 3, 5, 6, and 7 are color cameras used to collect light field information. A schematic diagram of the viewer is shown in Figure 5. When the viewer's viewpoint is in Area A, cameras 1, 2, and 3 can be selected as acquisition cameras. When in Area B, cameras 5, 6, and 7 can be selected as acquisition cameras. To synthesize a viewpoint between cameras 1 and 2, stereo vision principles are used to obtain disparity maps 1 and 2. Mismatched points are removed using a stereo vision algorithm to obtain a point cloud in camera coordinate systems 1 and 2. After the relationship between the cameras is calibrated, the point cloud is transferred to camera 4 and projected to obtain a depth image. The normal vector information from camera 4 is integrated to fill in gaps in stereo matching caused by weak texture and occlusion. The filled depth map is converted to a point cloud, which is then converted to the original coordinate system and projected to obtain a more accurate disparity image. The viewpoint is then synthesized using the forward / inverse method. The same process is applied to other viewpoints.

[0069] In this exemplary embodiment, the camera combination includes several color cameras.

[0070] The method further includes: calling the camera combination corresponding to the viewpoint area to perform image acquisition to obtain a color image; and acquiring a second image in an infrared mode using other cameras;

[0071] extracting image channel information from the collected color image and the second image according to the band information of the infrared light source;

[0072] The normal vector is calculated using the shadow restoration algorithm based on the extracted image channel information.

[0073] Specifically, as shown in Figure 7, seven color cameras are used. As shown in Figure 7, the backlight only contains infrared LEDs in area C below the screen; all other areas are visible light LEDs. When the screen is operating, both the visible light LEDs and the infrared LEDs are constantly on. When the viewer's viewpoint is in area A, cameras 1, 2, 3, and 4 are selected as acquisition cameras. The other cameras operate in infrared mode. When the viewer is in area B, cameras 4, 5, 6, and 7 are selected as acquisition cameras, while the other cameras operate in infrared mode. After the infrared cameras acquire the target image, the object's normal vector is calculated using shape from shading. To synthesize the viewpoint between cameras 1 and 2, stereo vision principles are used to calculate disparity maps 1 and 2. Mismatched points are removed using a stereo vision algorithm, resulting in a point cloud in camera coordinate systems 1 and 2. After the cameras are calibrated, the point cloud is transferred to cameras 5, 6, and 7 and projected to obtain a depth image. This normal vector information is then incorporated to fill in gaps in the stereo matching caused by weak textures and occlusions. The filled depth map is converted into a point cloud, which is then converted to the original coordinate system and projected to obtain a more accurate disparity image. The forward / inverse method is then used to synthesize the viewpoint. The same applies to other viewpoints.

[0074] The shape-from-shadow method can reconstruct an object using a single frame image. If the target reflectivity is known, the infrared mode of a color camera can reconstruct the object normal vector using a single infrared light source and introducing other constraints. The specific principles are as follows:

[0075] Among them, E(x,y) is the pixel grayscale value, I(x,y) is the light intensity in the image, ρ is the surface reflectivity of the object, (p i ,q i ,-1) T is the light vector, (p,q,-1) T is the surface normal vector.

[0076] The image processing method provided in the embodiments of the present disclosure is suitable for three-dimensional reconstruction of LCD screens. The screen uses a backlight that emits visible and infrared light in the wavelength bands. The backlight is divided into multiple zones, each with a different infrared wavelength. The visible light is used for display, while the infrared light sources of different wavelengths, together with a single camera, form a photometric stereo / shape-from-shadow vision system. Multiple multispectral cameras are installed around the screen to capture color images of the object and infrared images of different wavelengths. The color images captured by the multiple cameras are used to achieve three-dimensional reconstruction of the object based on stereo vision. The infrared images of different wavelengths from a single camera are used to reconstruct normal vectors based on photometric stereo / shape-from-shadow. Visual information can be collected simultaneously with the display. By using different algorithms for capturing different images, photometric stereo methods can be used to reflect the three-dimensional topography of the object's surface using the varying intensity of light reflection at different depths, enabling high-detail, pixel-level reconstruction of the object. By fusing the global information of stereo vision with the detailed information of photometric stereo, richly detailed three-dimensional reconstruction of the object is achieved.

[0077] It should be noted that the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention and are not intended to be limiting. It is readily understood that the processes illustrated in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0078] It should be noted that although several modules or units of the device for action execution are mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be concretized in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.

[0079] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0080] The units involved in the embodiments of the present invention may be implemented in software or hardware, and the units described may also be provided in a processor. In some cases, the names of these units do not limit the units themselves.

[0081] In another aspect, the present disclosure further provides an image processing device. Referring to FIG9 , an image processing device 90 according to an exemplary embodiment of the present invention may include: a user viewpoint recognition module 901, an image acquisition module 902, and a normal vector reconstruction module 903.

[0082] The user viewpoint identification module 901 may be used to identify the viewpoint area of ​​the user on the LCD screen; the backlight source of the LCD screen is arranged with infrared light sources in at least one partition.

[0083] The image acquisition module 902 can be used to call the camera combination corresponding to the viewpoint area to perform image acquisition; wherein the camera combination includes: a color camera and / or a multispectral infrared camera; the acquired image includes at least one frame of color image acquired by each camera.

[0084] The normal vector reconstruction module 903 can be used to call a corresponding method to calculate and reconstruct the normal vector according to the number of types of infrared light source directions, so as to perform three-dimensional reconstruction of the object based on the normal vector.

[0085] Since the functional modules of the image processing device in the embodiment of the present disclosure are the same as those in the above-mentioned embodiment of the image processing method invention, they are not described again here.

[0086] It should be noted that, as shown in FIG10 , the present disclosure further provides a storage medium as another aspect. The storage medium may be included in an electronic device, or may exist independently and not be incorporated into the electronic device. The storage medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiments.

[0087] It should be noted that the storage medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), flash memory, optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present invention, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any storage medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on the storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0088] After the storage medium according to the exemplary embodiment of the present invention has been introduced, next, an electronic device according to the exemplary embodiment of the present invention will be described with reference to FIG. 11 .

[0089] The electronic device 800 shown in FIG11 is merely an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0090] As shown in FIG11 , electronic device 800 is implemented as a general-purpose computing device. Components of electronic device 800 may include, but are not limited to, the aforementioned at least one processing unit 810, the aforementioned at least one storage unit 820, a bus 830 connecting various system components (including storage unit 820 and processing unit 810), and a display unit 840.

[0091] The storage unit stores program code that can be executed by the processing unit 810, causing the processing unit 810 to perform the steps according to various exemplary embodiments of the present invention described in the "Exemplary Methods" section above. For example, the processing unit 810 may perform the steps shown in FIG.

[0092] The storage unit 820 may include a volatile storage unit, such as a random access memory unit (RAM) 8201 and / or a cache memory unit 8202 , and may further include a read-only memory unit (ROM) 8203 .

[0093] The storage unit 820 may also include a program / utility 8204 having a set (at least one) of program modules 8205, such program modules 8205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0094] The bus 830 may include a data bus, an address bus, and a control bus.

[0095] The electronic device 800 can also communicate with one or more external devices 900 (e.g., a keyboard, a pointing device, a Bluetooth device, etc.), and such communication can be performed via an input / output (I / O) interface 850. The electronic device 800 also includes a display unit 840, which is connected to the input / output (I / O) interface 850 for display. In addition, the electronic device 800 can also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 860. As shown, the network adapter 860 communicates with other modules of the electronic device 800 via a bus 830. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0096] Furthermore, the above-described figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention and are not intended to be limiting. It is readily understood that the processes illustrated in the above-described figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0097] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow from the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the claims.

Claims

1. A display device, comprising: An LCD screen, characterized in that the LCD screen includes a backlight source provided with a plurality of partitions; A visible light source is provided in each partition; and An infrared light source is arranged in at least one partition.

2. The display device according to claim 1, characterized in that Infrared light sources are arranged in each partition respectively; the infrared light sources in each partition have different infrared bands.

3. The display device according to claim 2, characterized in that The infrared light sources of each partition have different light source directions.

4. The display device according to claim 1 or 2, characterized in that: The partitions of the backlight source are arranged in a polygonal structure.

5. The display device according to claim 1, characterized in that In the partition, the number of visible light sources is greater than the number of infrared light sources; the visible light sources are arranged to surround the infrared light sources.

6. The display device according to claim 1, characterized in that A multi-spectral infrared camera and / or a color camera is arranged outside the LCD screen.

7. An image processing method, characterized in that: The method comprises: Identifying a user's viewpoint area on an LCD screen; a backlight source of the LCD screen is arranged with an infrared light source in at least one partition; Calling a camera combination corresponding to the viewpoint area to perform image acquisition; wherein the camera combination includes: a color camera and / or a multi-spectral infrared camera; the acquired image includes at least one frame of color image acquired by each camera; According to the number of types of infrared light source directions, a corresponding method is called to calculate and perform normal vector reconstruction, so as to perform three-dimensional reconstruction of the object based on the normal vector.

8. The image processing method according to claim 7, characterized in that: The camera combination includes a plurality of multi-spectral infrared cameras; The method further comprises: Extracting image channel information from the collected multi-frame color images according to the band information of the infrared light source; The normal vector is calculated using the photometric stereo algorithm based on the extracted image channel information.

9. The image processing method according to claim 7, characterized in that: The camera combination includes a plurality of color cameras and a multi-spectral infrared camera; The method further comprises: Extracting image channel information from the collected multi-frame color images according to the band information of the infrared light source; The normal vector is calculated using the photometric stereo algorithm based on the extracted image channel information.

10. The image processing method according to claim 7, characterized in that: The camera combination includes a plurality of color cameras; The method further comprises: Calling the camera combination corresponding to the viewpoint area to perform image acquisition to obtain a color image; and acquiring a second image in an infrared mode for other cameras; Extracting image channel information from the collected color image and the second image according to the band information of the infrared light source; The normal vector is calculated using the shadow restoration algorithm based on the extracted image channel information.

11. An image processing device, characterized in that: The device comprises: A user viewpoint recognition module, used to recognize the viewpoint area of ​​the user on the LCD screen; the backlight source of the LCD screen is arranged with an infrared light source in at least one partition; An image acquisition module, used for calling the camera combination corresponding to the viewpoint area to acquire images; wherein the camera combination includes: a color camera and / or a multi-spectral infrared camera; the acquired image includes at least one frame of color image acquired by each camera; The normal vector reconstruction module is used to call the corresponding method to calculate and reconstruct the normal vector according to the number of types of infrared light source directions, so as to perform three-dimensional reconstruction of the object based on the normal vector.

12. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image processing method according to any one of claims 7 to 10 is implemented.

13. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the image processing method according to any one of claims 7 to 10 by executing the executable instructions.

Images