Image Processing Apparatus, Method, and Device for Generating Depth Images

Abstract

The present application provides an image processing apparatus, method and device for generating a depth image. By setting a plurality of image processing units, the receiving end of each image processing unit is matched with the output end of at least one type of image acquisition device. According to the type of the image acquisition device, the original image data or the image data obtained by preprocessing the original image data is written into the memory; the DSP reads the image data from the memory according to the type of the currently connected image acquisition device and generates a corresponding 3D image. It can be compatible with a variety of different types of cameras, and a corresponding 3D image can be generated based on the 2D image acquired by any type of camera. Different types of cameras can be flexibly selected according to the application scenario, improving flexibility; a programmable DSP is used to implement the conversion processing of the 2D image into the 3D image, and the algorithm software on the DSP is programmable and can be continuously iteratively updated, improving the efficiency of iterative evolution and reducing the cost of iterative evolution.

Description

Technical Field

[0001] This application relates to image processing technology, and in particular, to an image processing device, method, and equipment for generating a depth image. Background Art

[0002] With the development of 3D imaging technology, 3D images (i.e., depth images) are widely used in multiple application scenarios. For example, in various face-scanning application scenarios such as access control, attendance, and payment, face recognition is performed based on 3D images, and other object recognition application scenarios. Currently, in the face recognition scenario, there are two methods: face recognition based on 2D images and face recognition based on 3D images (i.e., depth images). Since the performance indicators of face recognition based on 3D images are much higher than those of face recognition based on 2D images, face recognition based on 3D images has currently become the mainstream trend. In face recognition based on 3D images, the accuracy of 3D images seriously affects the accuracy of face recognition.

[0003] 3D imaging technology mainly generates 3D images based on 2D images collected by cameras. Currently, it mainly includes 2D images collected by the following types of cameras: binocular cameras, structured light cameras, Time of Flight (TOF) cameras, etc. Currently, 3D imaging devices in the industry generally support a specific type of camera in a hardened manner, generating 3D images based on 2D images collected by a specific type of camera. They can only support a specific type of camera, with a single support function, and the hardened method results in slow product iteration and high costs. Summary of the Invention

[0004] This application provides an image processing device, method, and equipment for generating a depth image.

[0005] In a first aspect, this application provides an image processing device for generating a depth image, which is compatible with multiple different types of image acquisition devices. The image processing device includes:

[0006] A programmable digital signal processing unit, a memory, and multiple image processing units;

[0007] Each of the image processing units has a receiving end, which is used to connect to the output end of the image acquisition device, and the receiving end is matched with the output end of at least one type of image acquisition device;

[0008] Each of the image processing units receives the original image data collected by the image acquisition device through the receiving end, and according to the type of the image acquisition device, writes the original image data into the memory or writes the image data obtained after preprocessing the original image data into the memory;

[0009] The digital signal processing unit is configured to read the stored image data from the memory and generate a depth image based on the stored image data.

[0010] In a second aspect, the present application provides an image processing method for generating a depth image, which is compatible with multiple different types of image acquisition devices. The method includes:

[0011] Receiving, by a plurality of receivers, the original image data acquired by an image acquisition device, where the plurality of receivers are configured to receive the original image data acquired by multiple different types of image acquisition devices;

[0012] In response to receiving the original image data through at least one of the receivers, writing the original image data into the memory or writing the image data obtained by preprocessing the original image data into the memory according to the type of the image acquisition device corresponding to the at least one receiver;

[0013] Reading the stored image data from the memory and generating a depth image based on the stored image data.

[0014] In a third aspect, the present application provides an image processing apparatus for generating a depth image, which is compatible with multiple different types of image acquisition devices. The image processing apparatus includes:

[0015] A receiving module, configured to receive, by a plurality of receivers, the original image data acquired by an image acquisition device, where the plurality of receivers are configured to receive the original image data acquired by multiple different types of image acquisition devices;

[0016] An image data preprocessing module, configured to, in response to receiving the original image data through at least one of the receivers, write the original image data into the memory or write the image data obtained by preprocessing the original image data into the memory according to the type of the image acquisition device corresponding to the at least one receiver;

[0017] A depth image generation module, configured to read the stored image data from the memory and generate a depth image based on the stored image data.

[0018] In a fourth aspect, the present application provides an image processing device for generating a depth image, which is compatible with multiple different types of image acquisition devices. The image processing device includes:

[0019] An image acquisition device,

[0020] and the image processing apparatus for generating a depth image according to the first aspect above.

[0021] The image processing device, method, and equipment for generating depth images provided by this application can be compatible with multiple different types of cameras. Based on the 2D images collected by any one type of camera, corresponding 3D images can be generated. Different types of cameras can be flexibly selected according to the actual application scenario, improving flexibility. In addition, a programmable digital signal processing unit (DSP) is used to implement the conversion process of 2D images to 3D images. The algorithm software on the digital signal processing unit is programmable and can be continuously iteratively updated, improving the efficiency of the iterative evolution of the image processing device and reducing the cost of product iterative evolution. Description of the Drawings

[0022] The drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with this application and used together with the specification to explain the principles of this application.

[0023] Figure 1 Schematic diagram of the architecture of an image processing device for generating depth images provided by an embodiment of this application;

[0024] Figure 2 Schematic diagram of the architecture of an image processing device for generating depth images provided by another embodiment of this application;

[0025] Figure 3 Schematic diagram of the architecture of an image processing device compatible with a binocular camera provided by an embodiment of this application;

[0026] Figure 4 Example diagram of an image processing device compatible with a binocular camera provided by an embodiment of this application;

[0027] Figure 5 Schematic diagram of the architecture of an image processing device compatible with a monocular camera provided by an embodiment of this application;

[0028] Figure 6 Example diagram of an image processing device compatible with a monocular camera provided by an embodiment of this application;

[0029] Figure 7 Schematic diagram of the architecture of an image processing device compatible with both monocular and binocular cameras provided by an embodiment of this application;

[0030] Figure 8 Schematic diagram of the architecture of an image processing device compatible with both monocular and binocular cameras provided by another embodiment of this application;

[0031] Figure 9 Flowchart of an image processing method for generating depth images provided by an embodiment of this application;

[0032] Figure 10Flowchart of the image processing method for the binocular-compatible image acquisition device provided by an embodiment of the present application;

[0033] Figure 11 Flowchart of the image processing method for the monocular-compatible image acquisition device provided by an embodiment of the present application;

[0034] Figure 12 Schematic diagram of the architecture of the image processing device for generating a depth image provided by an embodiment of the present application.

[0035] Through the above-mentioned drawings, specific embodiments of the present application have been shown, and there will be more detailed descriptions hereinafter. These drawings and textual descriptions are not intended to limit the scope of the concept of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. Detailed implementation manners

[0036] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0037] The terms "first", "second", "third", etc. involved in the present application are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. In the description of the following embodiments, "a plurality" means more than two, unless otherwise specifically defined.

[0038] The present application provides an image processing device that can be compatible with various different types of cameras, including binocular cameras, structured light cameras, TOF cameras, etc. Based on the 2D images collected by any one of these types of cameras, the image processing device can generate corresponding 3D images according to the 2D images to achieve 3D imaging, and different types of cameras can be flexibly selected according to the actual application scenarios.

[0039] In addition, a programmable digital signal processor (DSP) is used as the digital signal processing unit in the image processing device to implement the conversion processing of 2D images to 3D images. The algorithm software on the DSP is programmable and can be continuously iteratively updated. Especially in the current background where various depth algorithms are constantly being optimized, the evolution ability of the DSP is much higher than that of customized hardware, which improves the efficiency of the iterative evolution of the image processing device and reduces the cost of product iterative evolution.

[0040] The nouns involved in this application are explained as follows:

[0041] Depth image: Also known as a distance image, it refers to an image that uses the distance (depth) from an image acquisition device to each point in a scene as pixel values.

[0042] Digital Signal Processor (DSP for short): A processor composed of large-scale or very-large-scale integrated circuit chips used to complete digital signal processing tasks; it is widely used in fields such as telecommunications, audio processing, and digital image processing.

[0043] Binocular camera: Composed of a left and a right camera, which use the two cameras to capture two left and right images of the same scene. Subsequently, the algorithm module can use a stereo matching algorithm to obtain the disparity map of the two left and right images, and then obtain the depth map.

[0044] Structured light camera: Composed of an image sensor and a projector. Structured light is the active structure information projected onto the surface of the object to be measured by the projector, such as laser stripes, Gray codes, sine stripes, etc.; then, the structured light image is obtained by shooting the surface to be measured with a single or multiple cameras; the subsequent algorithm module can compare the offset positions of different tracking light points at the transmitting and receiving ends to obtain the depth map.

[0045] Time-of-Flight camera: Composed of a ToF sensor and a transmitter, which actively emits outward using a modulated light source (such as a laser), reflects after encountering an object, and ends when reflected back to the camera; calculates the time difference or phase difference from emission to reflection back to the camera, and collects the data to obtain a set of distance depth data.

[0046] The technical solution of this application and how the technical solution of this application solves the above technical problems will be described in detail below with specific embodiments. These several specific embodiments below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0047] Figure 1 It is a schematic diagram of the architecture of an image processing device provided by an embodiment of this application. The image processing device provided by this application can be compatible with a variety of different types of image acquisition devices, such as Figure 1 As shown, the image processing device 10 includes: a programmable digital signal processing unit 11, a memory 12, and multiple image processing units 13.

[0048] Among them, each image processing unit 13 has a receiving end 14. The receiving end 14 is used to connect to the output end of the image acquisition device 20, and the receiving end 14 is matched with the output end of at least one type of image acquisition device 20.

[0049] In this embodiment, a plurality of image processing units 13 are provided in the image processing apparatus. Each image processing unit 13 corresponds to one or more types of image acquisition devices 20 and has a receiving end that matches the output end of the corresponding image acquisition device 20.

[0050] Each image processing unit 13 receives the original image data collected by the image acquisition device 20 through the receiving end 14, and writes the original image data into the memory 12 or writes the image data obtained by preprocessing the original image data into the memory 12 according to the type of the image acquisition device 20.

[0051] The digital signal processing unit 11 is used to read the stored image data from the memory 12 and generate a depth image based on the stored image data.

[0052] In practical applications, a suitable image acquisition device is selected according to the actual application scenario. According to the type of the selected image acquisition device, the output end of the image acquisition device is connected to the matching receiving end, and the image processing unit 13 having this receiving end can receive the original image data (2D image) collected by the currently selected image acquisition device.

[0053] Exemplarily, for the two-way original image data collected by a binocular image acquisition device (such as a binocular camera), the image processing unit 13 respectively converts the two-way original image data into standard RGB images and writes the two-way RGB images into the memory. The digital signal processing unit 11 reads the two-way RGB images from the memory and generates a corresponding 3D image based on the two-way RGB images, implementing the 3D imaging technology based on the binocular image acquisition device.

[0054] Exemplarily, for a monocular image acquisition device, such as a structured light camera, a TOF camera, etc., the image processing unit 13 writes the original image data collected by the monocular image acquisition device into the memory. The digital signal processing unit 11 reads the original image data from the memory and generates a corresponding 3D image based on the original image data, implementing the 3D imaging technology based on the monocular image acquisition device.

[0055] The image processing device provided in this embodiment sets multiple image processing units. The receiving end of each image processing unit is matched with the output end of at least one type of image acquisition device, that is, each image processing unit corresponds to at least one type of image acquisition device. If an image processing unit corresponds to multiple types of image acquisition devices, the processing method of the original image data collected by the multiple types of image acquisition devices corresponding to the image processing unit is the same. If the image processing unit corresponds to a monocular image acquisition device, the image processing unit writes the original image data into the memory. If the image processing unit corresponds to a binocular image acquisition device, the image processing unit preprocesses the two-way original image data respectively to obtain two-way RGB images, and writes the obtained two-way RGB images into the memory. The digital signal processing unit performs corresponding processing on the image data read from the memory according to the type of the currently connected image acquisition device to generate a corresponding 3D image. It can be compatible with multiple different types of cameras, including binocular cameras, structured light cameras, TOF cameras, etc. Based on the 2D images collected by any one of these types of cameras, the image processing device can generate corresponding 3D images according to the 2D images to achieve 3D imaging, and different types of cameras can be flexibly selected according to the actual application scenario. In addition, a programmable digital signal processing unit (DSP) is used to implement the conversion processing from 2D images to 3D images. The algorithm software on the digital signal processing unit is programmable and can be continuously iteratively updated. Especially in the current context where various depth algorithms are constantly being optimized, the evolution ability of the digital signal processing unit is much higher than that of customized hardware, which improves the iteration and evolution efficiency of the image processing device and reduces the cost of product iteration and evolution.

[0056] In an optional implementation manner, on the basis of the above embodiment, as Figure 2 shown, the image processing device 10 further includes: a data bus 15. When each image processing unit 13 writes image data into the memory 12, the image data is written into the memory 12 through the data bus 15.

[0057] The image processing unit 13 is configured to write the original image data into the memory 12 through the data bus 15, or write the image data obtained after preprocessing the original image data into the memory 12 through the data bus 15.

[0058] The digital signal processing unit 11 is configured to read the stored image data from the memory 12 through the data bus 15.

[0059] Optionally, the bus protocol such as AXI (Advanced eXtensible Interface) can be used to implement writing image data into the memory and reading image data from the memory.

[0060] Exemplarily, for the original image data of two channels collected by a binocular image acquisition device (such as a binocular camera), the image processing unit 13 preprocesses the original image data of the two channels respectively, converts them into two RGB images, and writes the two RGB images into the memory 12 through the data bus 15. The digital signal processing unit 11 reads the two RGB images from the memory through the data bus 15, generates the corresponding 3D image according to the two RGB images, and realizes the 3D imaging technology based on the binocular image acquisition device.

[0061] Exemplarily, for a monocular image acquisition device, such as a structured light camera, a TOF camera, etc., the image processing unit 13 writes the original image data collected by the monocular image acquisition device into the memory 12 through the data bus 15. The digital signal processing unit 11 reads the original image data from the memory 12 through the data bus 15, generates the corresponding 3D image according to the original image data, and realizes the 3D imaging technology based on the monocular image acquisition device.

[0062] Figure 3 The figure is a schematic architecture diagram of an image processing device provided in another embodiment of the present application. Based on the image processing device provided in any of the above embodiments, in this embodiment, the image processing device can be compatible with a variety of different types of image acquisition devices, including binocular image acquisition devices. The binocular image acquisition device includes a first output end and a second output end, and the first output end and the second output end respectively output a channel of original image data.

[0063] For example, a binocular image acquisition device usually includes a camera arranged on the left side and a camera arranged on the right side. The first output end is used to output the original image (also called the left-eye image) collected by the left camera of the binocular image acquisition device, and the second output end is used to output the original image (also called the right-eye image) collected by the right camera of the binocular image acquisition device; or, the first output end is used to output the original image (also called the right-eye image) collected by the right camera of the binocular image acquisition device, and the second output end is used to output the original image (also called the left-eye image) collected by the left camera of the binocular image acquisition device.

[0064] In this embodiment, as Figure 3 shown, the image processing unit 13 corresponding to the binocular image acquisition device includes a first image signal processing module 131 and a second image signal processing module 132. The first image signal processing module 131 has a first receiving end 141, and the second image signal processing module 132 has a second receiving end 142.

[0065] Among them, the first receiving end 141 is used to connect to the first output end 211 of the binocular image acquisition device 21. The first image signal processing module 131 receives the first path of original image data output from the first output end 211 of the binocular image acquisition device 21 through the first receiving end 141, converts the first path of original image data into an RGB image, obtains the first path of RGB image, and writes the first path of RGB image into the memory 12.

[0066] The second receiving end 142 is used to connect to the second output end 212 of the binocular image acquisition device 21. The second image signal processing module 132 receives the second path of original image data output from the second output end 212 of the binocular image acquisition device 21 through the second receiving end 142, converts the second path of original image data into an RGB image, obtains the second path of RGB image, and writes the second path of RGB image into the memory 12.

[0067] The digital signal processing unit 11 is used to read the first path of RGB image and the second path of RGB image from the memory 12 and generate a corresponding depth image according to the first path of RGB image and the second path of RGB image.

[0068] Optionally, the first image signal processing module and the second image signal processing module can be implemented by an Image Signal Processor (ISP for short).

[0069] Exemplarily, taking an RGB binocular camera as an example, as Figure 4 shown, the RGB binocular camera includes two sensors, namely RGB0 sensor and RGB1 sensor. Among them, the output end of the RGB0 sensor is connected to the first receiving end of the image processing device. The first path of original image data collected by the RGB0 sensor is transmitted to the first image signal processing module ISP0 through the first receiving end. ISP0 converts the first path of original image data into the first path of RGB image (2D image), and writes the first path of RGB image into the memory through the data bus. The output end of the RGB1 sensor is connected to the second receiving end of the image processing device. The second path of original image data collected by the RGB1 sensor is transmitted to the second image signal processing module ISP1 through the second receiving end. ISP1 converts the second path of original image data into the second path of RGB image (2D image), and writes the second path of RGB image into the memory through the data bus. The programmable digital signal processing unit DSP reads the first path of RGB image and the second path of RGB image from the memory through the data bus, and generates a corresponding depth image (3D image) according to the first path of RGB image and the second path of RGB image.

[0070] The image processing device provided in this embodiment can be compatible with a binocular image acquisition device. In practical applications, by connecting the two output ends of the binocular image acquisition device to the first receiving end and the second receiving end of the image acquisition device respectively, the two-way original image data collected by the binocular image acquisition device can be input into the first image signal processing module and the second image signal processing module through the first receiving end and the second receiving end respectively. The first image signal processing module and the second image signal processing module can respectively convert the two-way original image data into RGB images and write them into the memory. The image processing unit can read the two RGB images from the memory and generate the corresponding 3D image, realizing a 3D imaging solution based on the binocular image acquisition device.

[0071] Figure 5 The following is a schematic structural diagram of an image processing device provided in another embodiment of the present application. Based on the image processing device provided in any of the above embodiments, in this embodiment, the image processing device can be compatible with at least one monocular image acquisition device among multiple different types of image acquisition devices. The monocular image acquisition device has one output end for outputting the original image data collected by the monocular image acquisition device.

[0072] In this embodiment, as Figure 5 shown, the image processing unit 13 corresponding to the monocular image acquisition device is a writing module 133. The writing module 133 has a third receiving end 143. The third receiving end 143 is used to connect to the output end of the monocular image acquisition device 22. The writing module 133 receives the original image data collected by the monocular image acquisition device 22 through the third receiving end 143 and writes the original image data into the memory 12.

[0073] The digital signal processing unit 11 is used to read the original image data collected by the monocular image acquisition device 22 from the memory 12 and generate the corresponding depth image according to the original image data collected by the monocular image acquisition device 22.

[0074] Among them, the writing module can be implemented by a DMA (Direct Memory Access) writing module (i.e., DMA_WR) to write image data into the memory, or an image signal processor can be used to write image data into the memory.

[0075] In addition, when the writing module is an image signal processor, the image signal processor only writes the original image data collected by the monocular image acquisition device into the memory and does not perform conversion processing on the original image data collected by the monocular image acquisition device.

[0076] Optionally, in the above Figure 3Based on the corresponding embodiments, if the image processing device includes at least one image signal processing unit, the writing module can use any one of the image signal processing units. The image signal processing unit acting as the writing module is used to write the original image data collected by the monocular image acquisition device into the memory, and will not perform conversion processing on the original image data collected by the monocular image acquisition device.

[0077] Optionally, the monocular image acquisition devices compatible with the image processing device include at least one of the following: a structure-light based image acquisition device (such as a structure-light camera), a TOF-based image acquisition device (such as a TOF camera).

[0078] For example, the original image data collected by the structure-light camera can be speckle pattern data, and the structure-light camera has an output terminal for outputting the collected speckle pattern data.

[0079] For example, the original image data collected by the TOF camera can be multi-phase RAW image data, and the TOF camera has an output terminal for outputting the collected multi-phase RAW image data. Among them, the RAW image data is the original data obtained by converting the light source signal captured by a CMOS (Complementary Metal Oxide Semiconductor) or CCD (charge coupled device camera) image sensor into a digital signal.

[0080] In this embodiment, since the algorithm software on the DSP is programmable, for different types of image acquisition devices, the image processing algorithms implemented by programming in the DSP can be different. Therefore, the architecture of the same image acquisition device can support connecting multiple different types of monocular image acquisition devices to implement a 3D imaging solution.

[0081] Exemplarily, the image acquisition device can support a structure-light camera and a TOF camera. In a specific application scenario, after determining the type of the image acquisition device used, the corresponding image processing algorithm can be implemented by programming in the DSP according to the determined type of the image acquisition device.

[0082] Exemplarily, taking the monocular image acquisition device as a structure-light camera or a TOF camera as an example, as Figure 6 shown, the output terminal of the structure-light camera or the TOF camera is connected to the third receiving terminal of the image processing device, and the original image data collected by the structure-light camera or the TOF camera is transmitted to the writing module through the third receiving terminal (such as Figure 6As shown in ISP2 / DMA_WR). The writing module writes the original image data into the memory through the data bus. The programmable digital signal processing unit DSP reads the monocular image acquisition device from the memory through the data bus and generates a corresponding depth image (3D image) according to the monocular image acquisition device.

[0083] The image processing device provided in this embodiment can be compatible with various types of monocular image acquisition devices, and can be at least compatible with a structured light camera or a TOF camera. After determining the type of the monocular image acquisition device used in actual applications, connect the output end of the monocular image acquisition device to the third receiving end of the image processing device. The original image data collected by the monocular image acquisition device can be input into the writing module through the third receiving end. The writing module writes the original image data collected by the monocular image acquisition device into the memory. The image processing unit can read the original image data collected by the monocular image acquisition device from the memory and generate a corresponding 3D image, realizing a 3D imaging scheme based on the monocular image acquisition device.

[0084] Figure 7 It is a schematic diagram of the architecture of an image processing device compatible with monocular and binocular cameras provided in another embodiment of this application. Combining the above Figure 3 and Figure 5 corresponding embodiments, an image processing device compatible with a binocular image acquisition device and at least one type of monocular image acquisition device can be obtained.

[0085] Exemplarily, as Figure 7 shown, taking the image processing device that can be compatible with a binocular camera, a structured light camera, and a TOF camera as an example, the image processing device 10 includes: a programmable DSP, a memory, a data bus, IPS0, ISP1, and a writing module. Among them, IPS0, ISP1, and the writing module are three image processing units.

[0086] Based on Figure 7 the shown image processing device, in an actual application scenario, if a binocular camera is externally connected, the first output end of the binocular camera is connected to the receiving end (first receiving end) of ISP0, and the second output end of the binocular camera is connected to the receiving end (second receiving end) of ISP1. ISP0 and ISP1 respectively convert the two-way original image data collected by the binocular camera into RGB images and write the two-way RGB images into the memory through the data bus. The DSP reads the two-way RGB data from the memory through the data bus and generates a corresponding depth image according to the two-way RGB data.

[0087] Based on Figure 7The illustrated image processing device, in an actual application scenario, if it is an external structured light camera or a TOF camera, the output end of the structured light camera or the TOF camera is connected to the receiving end (the third receiving end) of the writing module, and the writing module writes the original image data output by the structured light camera or the TOF camera into the memory through the data bus. The DSP reads the original image data from the memory through the data bus and generates a corresponding depth image based on the original image data.

[0088] Optionally, Figure 7 In the illustrated image processing device, the writing module can implement the function of writing to the memory using DMA_WR or ISP. If the writing module uses ISP, the ISP serving as the writing module only processes the original image data collected by the monocular camera by writing it into the memory without performing other image processing. Additionally, the writing module can reuse Figure 7 ISP0 or ISP1 among them. When ISP0 or ISP1 serves as the writing module, the original image data collected by the monocular camera is only processed by writing it into the memory without performing other image processing.

[0089] The image processing device provided in this embodiment can be compatible with binocular cameras, structured light cameras, and TOF cameras to achieve 3D imaging, and can flexibly select a suitable camera according to the actual application scenario. Additionally, if 3D imaging technologies based on binocular, structured light, and TOF are supported by hardware, 3 sets of customized hardware are required for adaptation, and the area is much larger than that of a single DSP. In the image processing device provided in this embodiment, the conversion processing of the 2D image to the depth image is implemented through DSP programming, greatly reducing the hardware area, and the algorithm software on the DSP is programmable and can be continuously iteratively updated. Especially in the current context where various depth algorithms are constantly being optimized, the evolution ability of the DSP is much higher than that of customized hardware, improving flexibility.

[0090] Figure 8 This is a schematic diagram of the architecture of an image processing device that is compatible with monocular and binocular cameras provided in another embodiment of the present application. Based on the image processing device provided in any of the above embodiments, in this embodiment, the image processing device may further include: a processor.

[0091] Exemplarily, based on Figure 7 the illustrated image processing device that is compatible with binocular image acquisition devices and at least one type of monocular image acquisition device, as Figure 8 shown, the image processing device 10 may further include: a processor 16.

[0092] Whenever a frame of image data is written into the memory 12, a first interrupt request is generated.

[0093] The processor 16 is configured to call the digital signal processing unit 11 in response to the first interrupt request.

[0094] When called, the digital signal processing unit 11 reads the frame of image data from the memory and generates a corresponding depth image based on the frame of image data.

[0095] In an alternative implementation, for multiple types of image acquisition devices compatible with the image processing device, corresponding image processing algorithms for each type of image acquisition device are implemented programmatically in the digital signal processing unit.

[0096] The processor is further configured to obtain the type of the currently connected image acquisition device.

[0097] The processor is further configured to, in response to a first interrupt request, call the digital signal processing unit to execute an image processing algorithm corresponding to the type of the currently connected image acquisition device, so as to generate a corresponding depth image based on a frame of image data.

[0098] Exemplarily, taking the image processing device compatible with three types of image acquisition devices, namely a binocular camera, a structured light camera, and a TOF camera, as an example, the following three image processing algorithms can be implemented programmatically in the programmable digital signal processing unit DSP:

[0099] The first image processing algorithm: generating a corresponding depth image based on two RGB images converted from the raw image data collected by the binocular camera.

[0100] The second image processing algorithm: generating a corresponding depth image based on the raw image data (such as speckle pattern data) collected by the structured light camera.

[0101] The third image processing algorithm: generating a corresponding depth image based on the raw image data (such as multi-phase RAW image data) collected by the TOF camera.

[0102] In such an exemplary scenario, GPIO pins can be added to the image processing device to indicate the camera type, and the processor can determine the type of the currently connected camera based on the GPIO pin values.

[0103] Whenever a frame of image data is written into the memory, a first interrupt request is generated. The processor responds to the first interrupt request and, based on the type of the currently connected camera, calls the digital signal processing unit to execute an image processing algorithm corresponding to the type of the currently connected camera, thereby generating a corresponding depth image.

[0104] For example, if the currently connected camera is a binocular camera, after the two-channel RGB images converted from the two-channel original image data collected by the binocular camera are written into the memory, a first interrupt request is generated. The processor responds to the first interrupt request and calls the digital signal processing unit to execute the above-mentioned first image processing algorithm, so as to generate a corresponding depth image according to the two-channel RGB images converted from the original image data collected by the binocular camera.

[0105] For example, the previously connected camera is a structured light camera. After a frame of original image data collected by the structured light camera is written into the memory, a first interrupt request is generated. The processor responds to the first interrupt request and calls the digital signal processing unit to execute the above-mentioned second image processing algorithm, so as to generate a corresponding depth image according to a frame of original image data collected by the structured light camera.

[0106] Further, after the digital signal processing unit generates the depth image, a second interrupt request is generated.

[0107] Optionally, the processor is further configured to: in response to the second interrupt request, perform face recognition processing on the depth image to obtain a face recognition result.

[0108] Optionally, the processor is further configured to: in response to the second interrupt request, call the face recognition unit. When the face recognition unit is called, it performs face recognition processing on the depth image to obtain a face recognition result. Among them, the face recognition unit can be implemented by an embedded neural network processor (Neural-network Processing Unit, abbreviated as NPU).

[0109] The image processing device provided in this embodiment can be compatible with various different types of cameras to implement 3D imaging to obtain a depth image, and perform face recognition based on the depth image, and can be used in various face-swiping application scenarios such as access control, attendance, and payment.

[0110] In addition, the image processing device provided in this application can also be applied to other face recognition scenarios, or other application scenarios that require the use of depth images such as object recognition.

[0111] Figure 9 The flowchart of the image processing method for generating a depth image provided in an embodiment of this application. The image processing method for generating a depth image provided in this application can be applied to the above Figure 1 The image processing device corresponding to the embodiment provided for generating a depth image can be compatible with various different types of image acquisition devices.

[0112] As Figure 9 shown, the specific steps of the method are as follows:

[0113] Step S91: Receive the original image data collected by an image acquisition device through multiple receivers, where the multiple receivers are used to receive the original image data collected by multiple different types of image acquisition devices.

[0114] In this embodiment, the image processing device for generating a depth image has multiple receivers. Each receiver is connected to the output end of a type of image acquisition device. By adapting the output ends of multiple different types of image acquisition devices through the multiple receivers, multiple different types of image acquisition devices can be compatible.

[0115] Exemplarily, the receiver is the receiver of the image processing unit of the image processing device. Multiple image processing units are provided in the image processing device. Each image processing unit corresponds to one or more types of image acquisition devices and has a receiver that matches the output end of the corresponding image acquisition device. Each image processing unit receives the original image data collected by the image acquisition device through the receiver, and writes the original image data into the memory or writes the image data obtained by preprocessing the original image data into the memory according to the type of the image acquisition device.

[0116] Step S92: In response to receiving the original image data through at least one receiver, write the original image data into the memory or write the image data obtained by preprocessing the original image data into the memory according to the type of the image acquisition device corresponding to the at least one receiver.

[0117] Among them, the at least one receiver that receives the original image data is the receiver that is actually connected to the output end of the image acquisition device.

[0118] In this step, after receiving the original image data through at least one receiver, the image processing unit writes the original image data into the memory or writes the image data obtained by preprocessing the original image data into the memory according to the type of the image acquisition device corresponding to the at least one receiver.

[0119] Among them, the types of image acquisition devices corresponding to different image processing units are different, and the processing performed on the original image data may be different.

[0120] Step S93: Read the stored image data from the memory and generate a depth image according to the stored image data.

[0121] In this step, the digital signal processing unit reads the stored image data from the memory and generates a depth image according to the stored image data.

[0122] In practical applications, according to the actual application scenario, an image acquisition device suitable for the current application scenario can be selected. According to the type of the selected image acquisition device, the output end of the image acquisition device is connected to a matching receiving end, and the image processing unit with this receiving end can receive the original image data (2D image) acquired by the currently selected image acquisition device.

[0123] Exemplarily, for the two-way original image data acquired by a binocular image acquisition device (such as a binocular camera), the image processing unit respectively converts the two-way original image data into standard RGB images and writes the two RGB images into the memory. The digital signal processing unit reads the two RGB images from the memory and generates corresponding 3D images based on the two RGB images, thereby implementing the 3D imaging technology based on the binocular image acquisition device.

[0124] Exemplarily, for a monocular image acquisition device, such as a structured light camera, a TOF camera, etc., the image processing unit writes the original image data acquired by the monocular image acquisition device into the memory. The digital signal processing unit reads the original image data from the memory and generates corresponding 3D images based on the original image data, thereby implementing the 3D imaging technology based on the monocular image acquisition device.

[0125] In this embodiment, the original image data acquired by the image acquisition device is received through multiple receiving ends, where the multiple receiving ends are used to receive the original image data acquired by multiple different types of image acquisition devices; in response to receiving the original image data through at least one receiving end, according to the type of the image acquisition device corresponding to at least one receiving end, the original image data is written into the memory or the image data obtained after preprocessing the original image data is written into the memory; the stored image data is read from the memory and a depth image is generated based on the stored image data, which can be compatible with multiple different types of image acquisition devices, and different types of cameras can be flexibly selected according to the actual application scenario.

[0126] The image processing method for generating a depth image provided in this application can be applied to an image processing device for generating a depth image that is compatible with both binocular and monocular image acquisition devices.

[0127] In an optional implementation manner, the image processing device for generating a depth image is capable of being compatible with a binocular image acquisition device. The binocular image acquisition device includes a first output end and a second output end, and each of the first output end and the second output end outputs a path of original image data.

[0128] For example, a binocular image acquisition device usually includes a camera disposed on the left side and a camera disposed on the right side. The first output terminal is used to output the original image (also referred to as the left-eye image) captured by the left camera of the binocular image acquisition device, and the second output terminal is used to output the original image (also referred to as the right-eye image) captured by the right camera of the binocular image acquisition device; alternatively, the first output terminal is used to output the original image (also referred to as the right-eye image) captured by the right camera of the binocular image acquisition device, and the second output terminal is used to output the original image (also referred to as the left-eye image) captured by the left camera of the binocular image acquisition device.

[0129] As Figure 10 shown, the image processing method for generating a depth image includes the following steps:

[0130] Step S101: Receive the first path of original image data and the second path of original image data captured by the binocular image acquisition device through the first receiving terminal and the second receiving terminal respectively.

[0131] In this embodiment, in the image processing device for generating a depth image, the image processing unit corresponding to the binocular image acquisition device includes a first image signal processing module and a second image signal processing module. The first image signal processing module has a first receiving terminal, and the second image signal processing module has a second receiving terminal. The first receiving terminal is used to connect to the first output terminal of the binocular image acquisition device. The second receiving terminal is used to connect to the second output terminal of the binocular image acquisition device.

[0132] The first image signal processing module receives the first path of original image data output from the first output terminal of the binocular image acquisition device through the first receiving terminal.

[0133] The second image signal processing module receives the second path of original image data output from the second output terminal of the binocular image acquisition device through the second receiving terminal.

[0134] Step S102: In response to receiving the first path of original image data and the second path of original image data captured by the binocular image acquisition device through the first receiving terminal and the second receiving terminal respectively, convert the first path of original image data into an RGB image to obtain the first path of RGB image, and write the first path of RGB image into the memory; and convert the second path of original image data into an RGB image to obtain the second path of RGB image, and write the second path of RGB image into the memory;

[0135] For the first original image received through the first receiving terminal, the first image signal processing module converts the first path of original image data into an RGB image to obtain the first path of RGB image, and writes the first path of RGB image into the memory.

[0136] For the second original image received by the second receiving end, the second image signal processing module converts the second path of original image data into an RGB image, obtains the second path of RGB image, and writes the second path of RGB image into the memory.

[0137] Step S103: Read the first path of RGB image and the second path of RGB image from the memory, and generate a corresponding depth image according to the first path of RGB image and the second path of RGB image.

[0138] Exemplarily, the image processing device may further include: a processor. A first interrupt request is generated whenever a frame of image data is written into the memory.

[0139] This step may be specifically implemented in the following manner:

[0140] In response to the first interrupt request generated whenever a frame of image data (including the first path of RGB image and the second path of RGB image) is written into the memory, the processor calls the digital signal processing unit. When the digital signal processing unit is called, it reads a frame of image data (the first path of RGB image and the second path of RGB image) from the memory, and generates a corresponding depth image according to the frame of image data (the first path of RGB image and the second path of RGB image).

[0141] The method provided in this embodiment is applied to an image processing device capable of being compatible with a binocular image acquisition device.

[0142] In an optional implementation manner, the image processing device for generating a depth image is capable of being compatible with at least one monocular image acquisition device. The monocular image acquisition device has an output end for outputting the original image data acquired by the monocular image acquisition device.

[0143] For example, the monocular image acquisition devices that the image processing device for generating a depth image can be compatible with include at least one of the following: a structured light-based image acquisition device (such as a structured light camera), a TOF-based image acquisition device (such as a TOF camera).

[0144] For example, the original image data acquired by the structured light camera may be speckle pattern data, and the structured light camera has an output end for outputting the acquired speckle pattern data.

[0145] For example, the original image data collected by a TOF camera can be multi-phase RAW image data. The TOF camera has an output terminal for outputting the collected multi-phase RAW image data. Among them, the RAW image data is the original data obtained by converting the captured light source signal into a digital signal by a CMOS (Complementary Metal Oxide Semiconductor) or a CCD (charge coupled device camera).

[0146] As Figure 11 shown, the image processing method for generating a depth image includes the following steps:

[0147] Step S111: Receive the original image data collected by a monocular image acquisition device through a third receiving terminal.

[0148] In this embodiment, in the image processing device for generating a depth image, the image processing unit corresponding to the monocular image acquisition device has a third receiving terminal, and the third receiving terminal is used to connect to the output terminal of the monocular image acquisition device.

[0149] The image processing unit corresponding to the monocular image acquisition device receives the original image data collected by the monocular image acquisition device through the third receiving terminal.

[0150] Step S112: In response to receiving the original image data collected by the monocular image acquisition device through the third receiving terminal, write the original image data into the memory.

[0151] For the original image data received through the third receiving terminal, the original image data is written into the memory through the corresponding image processing unit.

[0152] Among them, the image processing unit corresponding to the monocular image acquisition device can be a write module, and a write module using DMA (Direct Memory Access) (i.e., DMA_WR) can be used to write image data into the memory, or an image signal processor can be used to write image data into the memory.

[0153] Step S113: Read the original image data collected by the monocular image acquisition device from the memory, and generate a corresponding depth image according to the type of the monocular image acquisition device and the original image data.

[0154] Exemplarily, the image processing device may further include: a processor. A first interrupt request is generated each time a frame of image data is written into the memory.

[0155] This step can be specifically implemented in the following manner:

[0156] In response to a first interrupt request generated whenever a frame of image data (including the original image data collected by a monocular image acquisition device) is written into memory, the processor invokes the digital signal processing unit. When the digital signal processing unit is invoked, it reads a frame of image data (the original image data collected by the monocular image acquisition device) from memory and generates a corresponding depth image based on the frame of image data (the original image data collected by the monocular image acquisition device).

[0157] The method provided in this embodiment is applied to an image processing device that can be compatible with at least one type of monocular image acquisition device.

[0158] Based on any of the above method embodiments, after generating the depth image, face recognition processing can also be performed.

[0159] After the digital signal processing unit generates the depth image, a second interrupt request is generated.

[0160] Optionally, in response to the second interrupt request generated after the depth image is generated, the processor can perform face recognition processing based on the depth image to obtain a face recognition result.

[0161] Optionally, in response to the second interrupt request generated after the depth image is generated, the processor can invoke the face recognition unit. When the face recognition unit is invoked, it performs face recognition processing based on the depth image to obtain a face recognition result.

[0162] Among them, the face recognition unit can be implemented by an embedded neural network processor (Neural-network Processing Unit, abbreviated as NPU).

[0163] The image processing method provided in this embodiment can be compatible with multiple different types of cameras to achieve 3D imaging to obtain a depth image, and perform face recognition based on the depth image, and can be used in various face-swiping application scenarios such as access control, attendance, and payment.

[0164] In addition, the image processing device provided in this application can also be applied to other face recognition scenarios, or other application scenarios that require the use of depth images such as object recognition.

[0165] Figure 12 This is a schematic architecture diagram of an image processing device for generating a depth image provided in an embodiment of the present application. The image processing device for generating a depth image provided in the embodiment of the present application can execute the processing flow provided in the embodiment of the image processing method for generating a depth image, and can be compatible with multiple different types of image acquisition devices. Such as Figure 12As shown in the figure, the image processing device 120 for generating a depth image includes: a receiving module 1201, an image data preprocessing module 1202, and a depth image generation module 1203.

[0166] Specifically, the receiving module 1201 is configured to receive the original image data collected by the image acquisition device through multiple receiving ends, where the multiple receiving ends are used to receive the original image data collected by multiple different types of image acquisition devices;

[0167] The image data preprocessing module 1202 is configured to, in response to receiving the original image data through at least one receiving end, write the original image data into the memory or write the image data obtained by preprocessing the original image data into the memory according to the type of the image acquisition device corresponding to at least one receiving end;

[0168] The depth image generation module 1203 is configured to read the stored image data from the memory and generate a depth image according to the stored image data.

[0169] In an optional implementation manner, the multiple different types of image acquisition devices include a binocular image acquisition device.

[0170] The receiving module is further configured to: receive the first path of original image data and the second path of original image data collected by the binocular image acquisition device through the first receiving end and the second receiving end respectively.

[0171] The image data preprocessing module is further configured to: in response to receiving the first path of original image data and the second path of original image data collected by the binocular image acquisition device through the first receiving end and the second receiving end respectively, convert the first path of original image data into an RGB image to obtain the first path of RGB image, and write the first path of RGB image into the memory; and convert the second path of original image data into an RGB image to obtain the second path of RGB image, and write the second path of RGB image into the memory.

[0172] The depth image generation module is further configured to: read the first path of RGB image and the second path of RGB image from the memory and generate a corresponding depth image according to the first path of RGB image and the second path of RGB image.

[0173] In an optional implementation manner, the multiple different types of image acquisition devices include at least one monocular image acquisition device.

[0174] The receiving module is further configured to: receive the original image data collected by the monocular image acquisition device through the third receiving end.

[0175] The image data preprocessing module is further configured to: in response to receiving the original image data collected by the monocular image acquisition device through the third receiving end, write the original image data into the memory.

[0176] The depth image generation module is further configured to: read the original image data collected by the monocular image acquisition device from the memory, and generate a corresponding depth image according to the type of the monocular image acquisition device and the original image data.

[0177] In an alternative embodiment, the monocular image acquisition device includes at least one of the following: a structured light-based image acquisition device, a time-of-flight-based image acquisition device.

[0178] In an alternative embodiment, the depth image generation module is further configured to:

[0179] In response to a first interrupt request generated whenever a frame of image data is written into the memory, call the digital signal processing unit. When the digital signal processing unit is called, it reads a frame of image data from the memory and generates a corresponding depth image according to the frame of image data.

[0180] In an alternative embodiment, the image processing device for generating a depth image may further include:

[0181] A face recognition processing module, configured to:

[0182] In response to a second interrupt request generated after the depth image is generated, perform face recognition processing on the depth image to obtain a face recognition result; or, in response to a second interrupt request generated after the depth image is generated, call the face recognition unit. When the face recognition unit is called, it performs face recognition processing on the depth image to obtain a face recognition result.

[0183] The device provided by the embodiments of the present application may be specifically configured to execute the solutions provided in any of the above method embodiments. The specific functions and achievable technical effects are not described in detail herein.

[0184] The present application provides an image processing device, including: an image acquisition device, and the image processing device provided in any of the above embodiments. Among them, the image acquisition device is an externally connected image acquisition device of the image processing device, and may be any type of image acquisition device compatible with the image processing device.

[0185] Those skilled in the art can clearly understand that for the convenience and brevity of description, only the above division of each functional module is used as an example. In actual applications, the above functions may be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the device described above may refer to the corresponding process in the foregoing method embodiments and will not be described in detail here.

[0186] Other embodiments of the present application will be readily apparent to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. The present application is intended to cover any variations, uses, or adaptations of the present application, which follow the general principles of the present application and include known common general knowledge or conventional technical means in the technical field not disclosed in the present application. The specification and examples are only to be considered as exemplary, and the true scope and spirit of the present application are pointed out by the following claims.

[0187] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present application is only limited by the appended claims.

Claims

1. An image processing apparatus for generating a depth image, characterized in that, Compatible with a variety of different types of image acquisition devices, the image processing device includes: A programmable digital signal processing unit, a memory, and multiple image processing units; wherein, the algorithm software on the digital signal processing unit is programmable and can be continuously iteratively updated; Each of the image processing units has a receiving end, and the receiving end is used to connect to the output end of the image acquisition device, and the receiving end is matched with the output end of at least one type of image acquisition device; Each of the image processing units receives the original image data collected by the image acquisition device through the receiving end, and according to the type of the image acquisition device, writes the original image data into the memory or writes the image data obtained after preprocessing the original image data into the memory; The digital signal processing unit is used to read the stored image data from the memory and generate a depth image according to the stored image data.

2. The image processing apparatus according to claim 1, wherein The variety of different types of image acquisition devices includes a binocular image acquisition device, and the binocular image acquisition device includes a first output end and a second output end, and the first output end and the second output end respectively output a path of original image data; The image processing unit corresponding to the binocular image acquisition device includes a first image signal processing module and a second image signal processing module, and the first image signal processing module has a first receiving end, and the second image signal processing module has a second receiving end; The first receiving end is used to connect to the first output end, and the first image signal processing module receives the first path of original image data output by the first output end through the first receiving end, converts the first path of original image data into an RGB image, obtains a first path of RGB image, and writes the first path of RGB image into the memory; The second receiving end is used to connect to the second output end, and the second image signal processing module receives the second path of original image data output by the second output end through the second receiving end, converts the second path of original image data into an RGB image, obtains a second path of RGB image, and writes the second path of RGB image into the memory; The digital signal processing unit is used to read the first path of RGB image and the second path of RGB image from the memory and generate a corresponding depth image according to the first path of RGB image and the second path of RGB image.

3. The image processing apparatus according to claim 1, wherein The variety of different types of image acquisition devices includes at least one monocular image acquisition device, and the image processing unit corresponding to the at least one monocular image acquisition device includes a writing module, The writing module has a third receiving end, and the third receiving end is used to connect to the output end of the monocular image acquisition device. The writing module receives the original image data collected by the monocular image acquisition device through the third receiving end and writes the original image data into the memory; The digital signal processing unit is used to read the original image data collected by the monocular image acquisition device from the memory and generate a corresponding depth image according to the original image data collected by the monocular image acquisition device.

4. An image processing method for generating a depth image, applied to the image processing device for generating a depth image according to any one of claims 1-3, wherein the algorithm software on the digital signal processing unit included in the image processing device is programmable and can be continuously and iteratively updated. Compatible with a variety of different types of image acquisition devices, the method includes: Receiving the original image data collected by an image acquisition device through multiple receivers, wherein the multiple receivers are used to receive the original image data collected by multiple different types of image acquisition devices; In response to receiving the original image data through at least one of the receivers, writing the original image data into the memory or writing the image data obtained by preprocessing the original image data into the memory according to the type of the image acquisition device corresponding to the at least one receiver; Reading the stored image data from the memory and generating a depth image according to the stored image data.

5. The method according to claim 4, wherein The multiple different types of image acquisition devices include binocular image acquisition devices, The receiving the original image data collected by an image acquisition device through multiple receivers includes: Receiving the first path of original image data and the second path of original image data collected by a binocular image acquisition device through a first receiver and a second receiver respectively; The in response to receiving the original image data through at least one of the receivers, writing the original image data into the memory or writing the image data obtained by preprocessing the original image data into the memory according to the type of the image acquisition device corresponding to the at least one receiver includes: In response to receiving the first path of original image data and the second path of original image data collected by a binocular image acquisition device through a first receiver and a second receiver respectively, converting the first path of original image data into an RGB image to obtain a first path of RGB image, and writing the first path of RGB image into the memory; and converting the second path of original image data into an RGB image to obtain a second path of RGB image, and writing the second path of RGB image into the memory; The reading the stored image data from the memory and generating a depth image according to the stored image data includes: Reading the first path of RGB image and the second path of RGB image from the memory and generating a corresponding depth image according to the first path of RGB image and the second path of RGB image.

6. The method according to claim 4, characterized in that The multiple different types of image acquisition devices include at least one monocular image acquisition device, The receiving the original image data collected by an image acquisition device through multiple receivers includes: Receiving the original image data collected by a monocular image acquisition device through a third receiver; The in response to receiving the original image data through at least one of the receivers, writing the original image data into the memory or writing the image data obtained by preprocessing the original image data into the memory according to the type of the image acquisition device corresponding to the at least one receiver includes: In response to receiving the original image data collected by a monocular image acquisition device through a third receiver, writing the original image data into the memory; The reading the stored image data from the memory and generating a depth image according to the stored image data includes: Reading the original image data collected by the monocular image acquisition device from the memory and generating a corresponding depth image according to the type of the monocular image acquisition device and the original image data.

7. The method according to claim 6, wherein The monocular image acquisition device includes at least one of the following: a structure light-based image acquisition device, a time-of-flight-based image acquisition device.

8. The method according to any one of claims 4 to 7, characterized in that Reading the stored image data from the memory and generating a depth image based on the stored image data includes: In response to a first interrupt request generated whenever a frame of image data is written into the memory, calling a digital signal processing unit, which, when called, reads the frame of image data from the memory and generates a corresponding depth image based on the frame of image data.

9. The method according to claim 8, characterized in that, It further includes: In response to a second interrupt request generated after the depth image is generated, performing face recognition processing on the depth image to obtain a face recognition result; Or, In response to a second interrupt request generated after the depth image is generated, calling a face recognition unit, which, when called, performs face recognition processing on the depth image to obtain a face recognition result.

10. An image processing device for generating a depth image, wherein the algorithm software on the digital signal processing unit included in the image processing device is programmable and can be continuously iteratively updated. The characteristics are as follows: Compatible with multiple different types of image acquisition devices, the image processing device includes: A receiving module for receiving the original image data collected by the image acquisition device through multiple receiving ends, where the multiple receiving ends are used to receive the original image data collected by multiple different types of image acquisition devices; An image data preprocessing module for, in response to receiving the original image data through at least one of the receiving ends, writing the original image data into the memory or writing the image data obtained by preprocessing the original image data into the memory according to the type of the image acquisition device corresponding to the at least one receiving end; A depth image generation module for reading the stored image data from the memory and generating a depth image based on the stored image data.

11. An image processing device for generating a depth image, characterized in that, Compatible with multiple different types of image acquisition devices, the image processing equipment includes: An image acquisition device, And the image processing device for generating a depth image according to any one of claims 1-3.

Images