Facial image processing device and method based on USB transmission technology
By setting up the processor and transmission chip before the processing terminal, the delay problem in USB video transmission technology is solved, image recognition efficiency and accuracy are improved, and lossless image transmission is achieved.
Patent Information
- Application Number
- CN202411329198.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-24
- Publication Date
- 2024-12-13
AI Technical Summary
The existing USB video transmission technology has major delay problems, which leads to poor user experience, especially when the image transmission delay requirements are demanding during eye or iris recognition, resulting in misidentification problems.
Before the processing terminal, the processor and the transmission chip are set up. The processor completes the downsampling and framing of the facial image. The transmission chip transmits the processing results to the processing terminal, eliminating the steps of image decompression, downsampling and seizure. The processing terminal is responsible for the processing terminal. Only responsible for feature extraction and comparison.
It effectively reduces the time delay of the processing terminal, improves image recognition efficiency and accuracy, avoids image compression, and thus provides lossless image transmission.
Smart Images
Figure CN119136042A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a facial image processing device and a processing method based on USB transmission technology, belonging to the technical field of data transmission. Background Art
[0002] At present, all USB video transmission solutions have the problem of time delay. This is mainly due to the time it takes for the image sensor to generate the image, the time it takes to compress the image, the time it takes for the USB to transmit the image, the time it takes for the PC to obtain data, the time it takes for the PC to decompress the image, and the time it takes for the algorithm to process the image. This will cause a large time delay from image acquisition to calculation of the result, resulting in a poor user experience. For example, the image display becomes stuck. When recognizing a person, the image is clearly displayed to be suitable, but the result is not displayed for a long time. What's worse, it may cause misrecognition. For example, the previous user has passed the recognition and has left, but the image is still being transmitted in the channel. The next user has not started to be recognized yet, but the system will report the recognition result of the previous user, resulting in false recognition.
[0003] The current general USB transmission technology cannot solve the problem of transmission delay, especially when performing eye recognition and iris recognition, which requires special processing procedures for the two eyes in an image. This is more complicated than general biometric recognition. At this time, the delay requirements for image transmission are more stringent. For example, when processing eye or iris images, image decompression, image downsampling and image subtraction are required, which consumes a lot of platform resources and time, so the delay problem will become more serious. Summary of the invention
[0004] The present invention provides a facial image processing device and a processing method based on USB transmission technology, which can solve the problem of transmission delay that the existing USB transmission technology cannot solve.
[0005] On the one hand, the present invention provides a facial image processing device based on USB transmission technology, the device comprising: an image collector, a processor connected to the image collector, a transmission chip connected to the processor, and a processing terminal connected to the transmission chip; The image collector is used to collect facial images; The processor is used to transmit the facial image to the transmission chip, or pre-process the facial image and transmit the processed image to the transmission chip; The transmission chip is used to transmit the facial image or the processed image to the processing terminal; The processing terminal is used to perform recognition processing on the facial image or the processed image.
[0006] Optionally, the processor is an ARM processor; the ARM processor is connected to the image collector via a DVP interface, and is connected to the transmission chip via a FIFO interface.
[0007] Optionally, the transmission chip is a USB interface chip; the USB interface chip is connected to the processing terminal via a USB2.0 or USB3.0 interface.
[0008] Optionally, the processor is specifically configured to: The processing instruction sent by the processing terminal is received through the transmission chip, and the facial image is pre-processed according to the processing instruction.
[0009] Optionally, the processor is specifically configured to: Receiving the facial image collected by the image collector, and receiving the processing instruction sent by the processing terminal; wherein the processing instruction is a face instruction or a human eye instruction; When the processing instruction is a face instruction, downsampling the facial image to obtain a downsampled face image, and framing the face image to transmit it to the processing terminal through the transmission chip; When the processing instruction is a human eye instruction, two human eye images in the facial image are deducted according to the human eye coordinates in the human eye instruction, and the two human eye images are framed and transmitted to the processing terminal through the transmission chip.
[0010] Optionally, the processing terminal is used to: Acquire the facial image transmitted by the processor; Detecting the position of human eyes in the face image, obtaining human eye coordinates, and sending human eye instructions to the processor through the transmission chip; the human eye instructions include the human eye coordinates; The two human eye images transmitted by the processor are acquired, and human eye recognition is performed based on the two human eye images.
[0011] On the other hand, the present invention provides a facial image processing method based on USB transmission technology, the method comprising: Using an image collector to collect facial images, and transmitting the facial images to a processor; The facial image is transmitted to the transmission chip by using a processor, or the processor pre-processes the facial image and transmits the processed image to the transmission chip; Transmitting the facial image or the processed image to a processing terminal using the transmission chip; The facial image or the processed image is recognized and processed by using the processing terminal.
[0012] Optionally, the processor preprocesses the facial image, specifically: The processor receives the processing instruction from the processing terminal through the transmission chip, and pre-processes the facial image according to the processing instruction.
[0013] Optionally, the processor preprocesses the facial image and transmits the processed image to the transmission chip, specifically including: The processor receives a processing instruction sent by the processing terminal; wherein the processing instruction is a face instruction or a human eye instruction; When the processing instruction is a face instruction, the processor performs downsampling processing on the facial image to obtain a downsampled face image, and transmits the face image to the transmission chip after framing; When the processing instruction is a human eye instruction, the processor deducts two human eye images from the facial image according to the human eye coordinates in the human eye instruction, and transmits the two human eye images to the transmission chip after framing them.
[0014] Optionally, the processing terminal performs recognition processing on the processed image, specifically including: The processing terminal obtains the facial image transmitted by the processor; The processing terminal detects the position of the human eye in the face image, obtains the human eye coordinates, and sends a human eye instruction to the processor through the transmission chip; the human eye instruction includes the human eye coordinates; The processing terminal obtains the two human eye images transmitted by the processor, and performs human eye recognition based on the two human eye images. The beneficial effects that the present invention can produce include: The facial image processing device based on USB transmission technology provided by the present invention, because a processor and a transmission chip are arranged before the processing terminal, the downsampling and framing work of the facial image can be completed on the processor, and the processing result of the processor is transmitted to the processing terminal by using the transmission chip, so that the processing terminal saves several steps that consume a lot of resources, such as image decompression, image downsampling, and human eye image deduction. The algorithm on the processing terminal only needs to be responsible for feature extraction and comparison, and does not need to manage time-consuming and labor-intensive image processing work, so that the efficiency of the processing terminal will be greatly improved, and the delay will be effectively reduced. At the same time, because this set of transmission schemes will not compress the image, the image transmitted to the processing terminal is lossless, which is more conducive to image recognition and improves the accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 A schematic diagram of the structure of a facial image processing device based on USB transmission technology provided by an embodiment of the present invention; Figure 2 A schematic diagram of the ARM processor processing logic provided by an embodiment of the present invention; Figure 3 A schematic diagram of processing logic of a processing terminal provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0016] The present invention is described in detail below in conjunction with embodiments, but the present invention is not limited to these embodiments.
[0017] The embodiment of the present invention provides a facial image processing device based on USB transmission technology, such as Figure 1 As shown, the device includes: an image collector, a processor connected to the image collector, a transmission chip connected to the processor, and a processing terminal connected to the transmission chip.
[0018] The image collector is used to collect facial images, wherein the image collector can be an image sensor.
[0019] The processor is used to transmit the facial image to the transmission chip, or pre-process the facial image and transmit the processed image to the transmission chip; the processor can be an ARM processor; the ARM processor is connected to the image collector through a DVP (Digital Video Port) interface to obtain facial image data, and is connected to the transmission chip through a FIFO (First Input First Output) interface to transmit the image data to the transmission chip.
[0020] The transmission chip is used to transmit the facial image or the processed image to the processing terminal. The transmission chip can be a USB (Universal Serial Bus) interface chip; the USB interface chip supports high-speed USB2.0 protocol or USB3.0 protocol, and the USB interface chip is connected to the processing terminal through the USB2.0 or USB3.0 interface to send data to the processing terminal.
[0021] The processing terminal is used to perform recognition processing on the facial image or the processed image. In practical applications, the processing terminal may be a PC (personal computer).
[0022] In the embodiment of the present invention, the processor may receive the processing instruction sent by the processing terminal through the transmission chip, and pre-process the facial image according to the processing instruction.
[0023] Specifically: the processor receives the facial image collected by the image collector, and receives the processing instruction sent by the processing terminal; wherein the processing instruction is a face instruction or a human eye instruction; When the processing instruction is a face instruction, the facial image is downsampled to obtain a downsampled face image, and the face image is framed and transmitted to the processing terminal through a transmission chip; When the processing instruction is a human eye instruction, two human eye images in the facial image are deducted according to the human eye coordinates in the human eye instruction, and the two human eye images are framed and transmitted to the processing terminal through the transmission chip.
[0024] ARM processor design logic such as Figure 2 As shown, firstly, it is detected whether a frame of facial image is full; If not, check whether the USB FIFO is not empty. If the USB FIFO is not empty, obtain the processing instructions and related data from the PC, and parse and cache the processing instructions and related data; if the USB FIFO is empty, continue to check whether the USB FIFO is not empty.
[0025] If yes, cache the facial image of this frame; Next, the facial image is processed for data integration according to the processing instructions on the PC side; Specifically, when the received processing instruction is a face instruction, the original facial image is downsampled to obtain a downsampled face image, and the face image is framed and transmitted to the PC through the USB interface chip; When the received processing instruction is a human eye instruction, two human eye images in the facial image are deducted according to the human eye coordinates in the human eye instruction, and the two human eye images are framed and transmitted to the PC through the USB interface chip.
[0026] After completing the preprocessing of a frame of facial image, the processor can continue to obtain the next frame of facial image from the image collector for preprocessing operation.
[0027] The present invention gives full play to the image acquisition and image preprocessing capabilities of the ARM processor, and the ARM processor can quickly complete tasks such as downsampling, framing, and image clipping.
[0028] In an embodiment of the present invention, the processing terminal is used to: obtain a facial image transmitted by a processor; detect the position of human eyes in the facial image, obtain human eye coordinates, and send human eye instructions to the processor through a transmission chip; the human eye instructions include human eye coordinates; obtain two human eye images transmitted by the processor, and perform human eye recognition based on the two human eye images.
[0029] Specifically, the processing flow of the processing terminal is as follows Figure 3As shown, first obtain a downsampled face image. Among them, the image sensor output image size can be 2560x1944, the ARM processor main frequency is 432MHz, and the resolution of the face image after the ARM processor downsamples can be 320x240. Then determine whether the human eye is detected from the face image. If not, reacquire the next downsampled face image; if so, calculate the human eye coordinates in the face image, and send the human eye coordinates to the ARM processor for the ARM processor to use when deducting the human eye image. Next, obtain two human eye images transmitted by the ARM processor, and the resolution of the human eye image can be 640x480. Finally, perform human eye recognition based on the two human eye images. The specific method for performing human eye recognition here can adopt the conventional method in the prior art, and the embodiment of the present invention is not limited to this. For example, after obtaining two human eye images, it can be determined whether the quality of the two human eye images meets the requirements. If not, the next downsampled face image is obtained for processing; if it meets the requirements, the human eyes are precisely located and the image is segmented, and the human eye features are extracted. The human eye features are then sent to the feature database for feature comparison, and the comparison results are obtained and returned to the user.
[0030] In the embodiment of the present invention, since the PC side omits the steps of image decompression, image downsampling, and human eye image deduction, which are very resource-consuming, the algorithm only needs to be responsible for feature extraction and comparison, and does not need to manage the time-consuming and labor-intensive image processing work, so that the efficiency is greatly improved and the delay is effectively reduced. At the same time, since this transmission solution does not compress the image, the image given to the PC side is lossless, which is more conducive to image recognition and improves the accuracy.
[0031] Another embodiment of the present invention provides a facial image processing method based on USB transmission technology, the method comprising: collecting a facial image using an image collector and transmitting the facial image to a processor; The facial image is transmitted to the transmission chip by using the processor, or the processor pre-processes the facial image and transmits the processed image to the transmission chip; The facial image or the processed image is transmitted to the processing terminal using a transmission chip; The facial image or the processed image is recognized and processed using the processing terminal.
[0032] Furthermore, the processor pre-processes the facial image, specifically: The processor receives the processing instruction from the processing terminal through the transmission chip, and pre-processes the facial image according to the processing instruction.
[0033] Furthermore, the processor pre-processes the facial image and transmits the processed image to the transmission chip, specifically including: The processor receives a processing instruction sent by the processing terminal; wherein the processing instruction is a face instruction or a human eye instruction; When the processing instruction is a face instruction, the processor downsamples the facial image to obtain a downsampled face image, and frames the face image and transmits it to the transmission chip; When the processing instruction is a human eye instruction, the processor deducts two human eye images from the facial image according to the human eye coordinates in the human eye instruction, and transmits the two human eye images to the transmission chip after framing them.
[0034] Furthermore, the processing terminal performs recognition processing on the processed image, specifically including: The processing terminal obtains the facial image transmitted by the processor; The processing terminal detects the position of the human eye in the face image, obtains the human eye coordinates, and sends the human eye instruction to the processor through the transmission chip; the human eye instruction includes the human eye coordinates; The processing terminal obtains the two human eye images transmitted by the processor, and performs human eye recognition based on the two human eye images.
[0035] The specific description of each step in the above processing method can refer to the description of each module in the processing device, which will not be repeated here. The above processing method can achieve the same function as the processing device side.
[0036] The facial image processing method based on USB transmission technology provided by the present invention, because a processor and a transmission chip are arranged before the processing terminal, the downsampling and framing work of the facial image can be completed on the processor, and the processing result of the processor is transmitted to the processing terminal by using the transmission chip, so that the processing terminal saves several steps that consume a lot of resources, such as image decompression, image downsampling, and human eye image deduction, and the algorithm on the processing terminal only needs to be responsible for feature extraction and comparison, and does not need to manage time-consuming and labor-intensive image processing work, so that the efficiency of the processing terminal will be greatly improved, and the delay will be effectively reduced. At the same time, because this set of transmission schemes will not compress the image, the image transmitted to the processing terminal is lossless, which is more conducive to image recognition and improves the accuracy.
[0037] The above are only a few embodiments of the present application and do not constitute any form of limitation to the present application. Although the present application is disclosed as above with preferred embodiments, it is not intended to limit the present application. Any technician familiar with the profession, without departing from the scope of the technical solution of the present application, using the technical content disclosed above to make slight changes or modifications are equivalent to equivalent implementation cases and fall within the scope of the technical solution.
Claims
1. A facial image processing device based on USB transmission technology, characterized in that: The device comprises: an image collector, a processor connected to the image collector, a transmission chip connected to the processor, and a processing terminal connected to the transmission chip; The image collector is used to collect facial images; The processor is used to transmit the facial image to the transmission chip, or pre-process the facial image and transmit the processed image to the transmission chip; The transmission chip is used to transmit the facial image or the processed image to the processing terminal; The processing terminal is used to perform recognition processing on the facial image or the processed image.
2. The device according to claim 1, characterized in that The processor is an ARM processor; the ARM processor is connected to the image collector through a DVP interface, and is connected to the transmission chip through a FIFO interface.
3. The device according to claim 2, characterized in that The transmission chip is a USB interface chip; the USB interface chip is connected to the processing terminal via a USB2.0 or USB3.0 interface.
4. The device according to claim 1, characterized in that The processor is specifically used for: The processing instruction sent by the processing terminal is received through the transmission chip, and the facial image is pre-processed according to the processing instruction.
5. The device according to claim 1 or 4, characterized in that: The processor is specifically used for: Receiving the facial image collected by the image collector, and receiving the processing instruction sent by the processing terminal; wherein the processing instruction is a face instruction or a human eye instruction; When the processing instruction is a face instruction, downsampling the facial image to obtain a downsampled face image, and framing the face image to transmit it to the processing terminal through the transmission chip; When the processing instruction is a human eye instruction, two human eye images in the facial image are deducted according to the human eye coordinates in the human eye instruction, and the two human eye images are framed and transmitted to the processing terminal through the transmission chip.
6. The device according to claim 5, characterized in that The processing terminal is used for: Acquire the facial image transmitted by the processor; Detecting the position of human eyes in the face image, obtaining human eye coordinates, and sending human eye instructions to the processor through the transmission chip; the human eye instructions include the human eye coordinates; The two human eye images transmitted by the processor are acquired, and human eye recognition is performed based on the two human eye images.
7. A facial image processing method based on USB transmission technology, characterized in that: The method comprises: Using an image collector to collect facial images, and transmitting the facial images to a processor; The facial image is transmitted to the transmission chip by using a processor, or the processor pre-processes the facial image and transmits the processed image to the transmission chip; Transmitting the facial image or the processed image to a processing terminal using the transmission chip; The facial image or the processed image is recognized and processed by using the processing terminal.
8. The method according to claim 7, characterized in that The processor preprocesses the facial image, specifically: The processor receives the processing instruction from the processing terminal through the transmission chip, and pre-processes the facial image according to the processing instruction.
9. The method according to claim 7, characterized in that: The processor pre-processes the facial image and transmits the processed image to the transmission chip, specifically including: The processor receives a processing instruction sent by the processing terminal; wherein the processing instruction is a face instruction or a human eye instruction; When the processing instruction is a face instruction, the processor performs downsampling processing on the facial image to obtain a downsampled face image, and transmits the face image to the transmission chip after framing; When the processing instruction is a human eye instruction, the processor deducts two human eye images from the facial image according to the human eye coordinates in the human eye instruction, and transmits the two human eye images to the transmission chip after framing them.
10. The method according to claim 9, characterized in that The processing terminal performs recognition processing on the processed image, specifically including: The processing terminal obtains the facial image transmitted by the processor; The processing terminal detects the position of the human eye in the face image, obtains the human eye coordinates, and sends a human eye instruction to the processor through the transmission chip; the human eye instruction includes the human eye coordinates; The processing terminal obtains the two human eye images transmitted by the processor, and performs human eye recognition based on the two human eye images.