Method and system for multi-camera data acquisition based on embedded devices

Abstract

The application discloses a kind of multi-path camera data acquisition method and system based on embedded device, contains two parts, embedded device and PC end, the application supports on the embedded device with USB interface, collect lossless or lossy image, support multiple image formats, support the meta attribute of saving image, for face detection or recognition Intelligent algorithm function processing module, realize statistical analysis or scene reproduction etc..The application also supports simultaneously collecting the data of multiple cameras, saves the correlation between images from different cameras, for collecting the images corresponding to multiple cameras at the same time, supports the data acquisition and data backflow requirements of living body detection algorithm.The application is based on PYTHON tool, realizes the functional requirements for large data acquisition, can assist to improve the level of automated testing, improve development and testing efficiency.

Description

[0001] The present application is a divisional application of the application No. 2023109126131, filed on July 25, 2023, and entitled "Automatic testing method and system for multi-camera data acquisition and data backflow" TECHNICAL FIELD

[0002] The present application relates to the field of computer technology, in particular to a multi-camera data acquisition method and system based on an embedded device. BACKGROUND

[0003] The embedded device, such as a smart door lock, has a smart module as an important component, and the important functional modules in the smart module are face detection, face recognition, and / or live body detection functional modules.

[0004] Among them, the training of the algorithm model is generally completed on a personal computer or a GPU, and an algorithm model is generated, and then the model is applied to the embedded system. In addition, for the live body algorithm, it is required to collect images of two or more cameras at the same time.

[0005] Generally, the face pictures used for training algorithms need to be collected from the cameras of the embedded system, so as to ensure that the algorithm model is consistent with the actual application scenario as much as possible, and the detection and recognition effect is better.

[0006] In addition, in actual application, it is required to backflow face data to the intelligent algorithm for calculating the detection effect or recognition effect of the algorithm model, or for positioning whether the algorithm model has vulnerabilities, etc.

[0007] The current data acquisition method is to save the camera video data into a file. The video format is generally MJEPG or H264, etc. The defects of this acquisition method are: 1. The collected data is distorted to varying degrees after compression. 2. The collected data generally needs to be further classified or re-labeled. 3. It is not possible to save two or more camera data at the same time, which cannot meet the data acquisition requirements of the live body detection algorithm.

[0008] The current backflow data generally uses a special device (such as HDMI to AHD) to convert the collected video into a virtual camera. The defects are: 1. The special device requires additional cost. 2. The video compressed data is distorted. 3. The embedded device (such as a smart door lock module) without a camera interface cannot be used. 4. It cannot meet the data backflow requirements of the live body detection algorithm.

[0009] In addition, there is a reverse flow data mode of transmitting video data or picture data through a network port, but this mode also depends on the existence of a network port or WIFI of the embedded device to be implemented. For embedded devices without a network port (such as a smart door lock module), the reverse flow data mode cannot be used. SUMMARY

[0010] The present application aims to overcome the shortcomings of the prior art and provide a multi-camera data acquisition method based on an embedded device, which has low requirements for the device, does not depend on a network port and a video interface, supports multi-camera data acquisition, and meets the data acquisition requirements of a living body detection algorithm.

[0011] To solve the above problems, the technical scheme adopted by the present application is as follows:

[0012] A multi-camera data acquisition method based on an embedded device, comprising the following steps:

[0013] Step S1: In the camera module of the embedded device, YUV data of cameras 1 to N is captured. The images captured by the multi-cameras at the same time have the same image ID value, which is used to determine the correlation between the images, compare the time stamps, and ensure that the multi-images captured at the same time are used for living body detection. The N images corresponding to the input to the living body algorithm are respectively identified as cam1_idM to camN_idM, wherein cam1 is the image data captured from camera 1, camN is the image data captured from camera N, and idM represents the images captured at the same time point. The meta attributes of the images are saved.

[0014] Step S2: Perform face detection to obtain a face detection result, save the face detection result as image attribute information, and enter the step of performing living body detection if the face detection is successful, otherwise enter the step of capturing YUV data of cameras 1 to N and comparing time stamps to ensure that the multi-images captured at the same time are used for living body detection.

[0015] Step S3: Obtain multi-images at the same time, perform living body detection, obtain a living body detection result, save the score of the living body detection as image attribute information, and enter the step of performing face recognition if the living body detection is successful, otherwise enter the step of capturing YUV data of cameras 1 to N and comparing time stamps to ensure that the multi-images captured at the same time are used for living body detection.

[0016] Step S4: Perform face recognition to obtain a face recognition result, save the face recognition result as image attribute information, and enter the step of saving the image and all image attributes to a file if the face recognition is successful, otherwise enter the step of capturing YUV data of cameras 1 to N and comparing time stamps to ensure that the multi-images captured at the same time are used for living body detection.

[0017] Step S5: save the image and all image attribute information to a file, and the image attribute information is encapsulated as a regular image file name, so that the PC end can parse the corresponding image attribute information from the image file name;

[0018] Step S6: connect the USB interface of the embedded device using a data line, connect the embedded device and the PC end, and the user operates the embedded device on the PC end through ADB;

[0019] Step S7: when collecting image data, the PC end queries whether the embedded device has collected images in a loop, acquires an image list through ADB, and sequentially downloads to the PC end through ADB;

[0020] Step S8: the PC end saves the downloaded image data to a specified folder.

[0021] Further, the process of collecting image data from the embedded device by the PC end specifically includes the following steps:

[0022] Step a, the PC end sends a start recognition command through ADB SHELL;

[0023] Step b, query whether the embedded device successfully collects the recognized images in a loop, and acquire an image list collected successfully;

[0024] Step c, send an image upload command through ADB, upload the collected images, save the images to a specified folder, and if there is no folder, create a new folder;

[0025] Step d, determine whether the number of collected images is greater than a threshold value, if yes, go to step e, otherwise go to step b;

[0026] Step e, end.

[0027] Another object of the present application is to overcome the shortcomings of the prior art, and provide a multi-camera data acquisition system based on an embedded device, which has low requirements on the device, does not rely on a network port and a video interface, supports multi-camera data acquisition, and meets the data acquisition requirements of a living body detection algorithm.

[0028] To achieve the above object, the technical scheme adopted by the present application is as follows:

[0029] A multi-camera data acquisition system based on an embedded device, comprising a plurality of cameras, an embedded device and a PC end;

[0030] The plurality of cameras are used to collect original images;

[0031] The embedded device captures YUV data of the cameras 1 to N, the images collected by the 1 to N cameras at the same time have the same image ID value, which is used to determine the correlation between the images, compare the time stamps, and ensure that the multiple images are captured at the same time, which are used for living body detection; the N images corresponding to the input are respectively identified as cam1_idM to camN_idM, wherein cam1 is the image data captured from the camera 1, camN is the image data captured from the camera N, and idM represents the images collected at the same time point; the meta attributes of the images are saved; the embedded device is used to perform face detection, obtain a face detection result, save the face detection result as image attribute information, obtain multiple images at the same time, perform living body detection, obtain a living body detection result, save the score of the living body detection as image attribute information, perform face recognition, obtain a face recognition result, and save the face recognition result as image attribute information; and save the images and all the image attribute information to a file, the image attribute information is packaged as a regular image file name, so that the PC end parses the corresponding image attribute information from the image file name; the embedded device is also used to execute detection or recognition commands sent by the PC end.

[0032] The PC end uses a data line to connect the USB interface of the embedded device, and uses ADB to operate the embedded device based on PYTHON on the PC end; when collecting image data, the PC end round-robin queries whether the embedded device has collected images, obtains an image list, and sequentially downloads the image list to the PC end; the PC end saves the downloaded image data to a specified folder.

[0033] Further, the specific process of collecting image data from the embedded device by the PC end includes the following steps:

[0034] Step a: the PC end sends a start recognition command through ADB SHELL;

[0035] Step b: round-robin query whether the embedded device successfully collects the recognized images, and obtain an image list collected successfully;

[0036] Step c: send an image upload command through ADB to upload the collected images, save the images to a specified folder, and if there is no folder, create a new folder;

[0037] Step d: determine whether the number of collected images is greater than a threshold value, if yes, go to step e, otherwise go to step b;

[0038] Step e: end.

[0039] After the above scheme, the multi-camera data acquisition method and system of the present application supports collecting lossless or lossy images on an embedded device with a USB interface, supports multiple image formats such as YUV data format (lossless data), MJPEG and JPEG format, supports saving the meta attributes of the image such as image resolution, image format, image source or purpose information, supports collecting image data through the USB interface, and is used for intelligent algorithm function processing modules such as face detection or recognition, realizes statistical analysis or scene reproduction, etc. The present application also supports simultaneous acquisition of multi-camera data, saves the correlation between images from different cameras, is used for collecting images corresponding to multiple cameras at the same time, and supports the data acquisition requirements of living body detection algorithm. In addition, the present application is based on PYTHON tool, realizes the functional requirements of large data acquisition, further improves the automation test level, and improves the development and test efficiency.

[0040] Compared with the prior art, the present application has low requirements for equipment, does not depend on a network port and a video interface, and only needs to have a USB interface; supports various data formats such as lossless and lossy, and more flexible to meet different development and test requirements; supports automatic test, reduces the cost of implementation scheme, saves time cost, and provides more scheme selection flexibility; supports multi-camera data acquisition, and meets the data acquisition requirements of living body detection algorithm. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 The figure is a schematic diagram of the framework of the system of the present application.

[0042] Figure 2 The figure is a schematic diagram of part of the process of the present application. DETAILED DESCRIPTION

[0043] In order to further explain the technical scheme of the present application, the present application will be described in detail below through specific embodiments.

[0044] As shown in Figure 1 and Figure 2 discloses an automatic test method for multi-camera data acquisition and data backflow, the general idea of which is to collect a large number of face images in an embedded device such as a smart door lock, which are used for training intelligent algorithms such as face detection and face recognition algorithms.

[0045] Generally, the face pictures used for training algorithms need to be collected from the camera of the embedded system, which can ensure that the algorithm model and the actual application scene are consistent as much as possible, and the detection and recognition effect is better.

[0046] In a smart device such as a door lock module, a USB interface is generally provided, and ADB function is supported. The USB channel has the characteristics of high bandwidth and fast speed. Therefore, it is considered to download images from a camera through the USB channel, and the raw data or the encoded data can be flexibly selected for downloading, and different formats of data can be collected according to different requirements.

[0047] In a smart door lock, in order to verify the face recognition effect, the detection pass or failure probability of a specific large data test set needs to be statistically tested, and the recognition pass or recognition failure probability of the large data test set also needs to be statistically tested. It is necessary to save which specific image failed or succeeded.

[0048] Therefore, the present application uses the PYHON tool to automatically upload and download images for a large data test set, perform face detection or recognition, and then save the face detection or recognition scores and the corresponding images in a report. For a specific threshold, the failure or success ratio is calculated to determine whether the algorithm meets the requirements or whether the algorithm has defects, and to assist in locating the defects.

[0049] In addition, the live detection algorithm requires multi-channel camera data from the same time as input data. Therefore, the correlation between the multi-channel camera data needs to be saved to meet the synchronization requirements of the image data.

[0050] As shown in Figure 1 and Figure 2 Based on the above idea, the present application discloses an automatic test method for multi-channel camera data acquisition and data backflow, which includes two parts: one part is a device end program, which is an embedded system, and the other part is a PC end device, which executes a PYTHON program. The method includes the following steps:

[0051] In the camera module of the embedded device, the original image is captured; the image format is generally YUV or RGB, and the original format image can be directly collected to ensure that the algorithm model and the actual application scene are consistent, and the detection and recognition effect is better; considering that the original data format will occupy more storage and bandwidth, the image can also be compressed, and the compression format can be MJPEG.

[0052] The meta attributes of the image, such as resolution, timestamp, image ID value, etc. are saved. Among them, the images collected by the multi-channel camera at the same time have the same image ID value, which is used to judge the correlation between the images.

[0053] Face detection is performed, and the face detection result, such as face coordinates and face detection score, is saved as image attribute information.

[0054] The multi-channel images at the same time are obtained, live detection is performed, and the live detection score is saved as image attribute information.

[0055] Perform facial recognition, such as obtaining the highest score and the person's name, and save the facial recognition results as image attribute information.

[0056] The image is saved as a file, and the image attribute information is encapsulated into a regular image file name, so that the PC can parse the corresponding image attribute information from the file name.

[0057] Connect the embedded device to its USB port using a data cable to connect the device to the PC. Users can then operate the embedded device on the PC via ADB, such as running a shell or transferring files.

[0058] During data acquisition, the PC client continuously checks for the existence of images to be acquired, obtains the image list via ADB, and downloads them to the PC client sequentially via ADB.

[0059] The PC client saves the downloaded image data to a specified folder; the folder name includes information such as camera ID, start time of acquisition, and acquisition scene.

[0060] During data backflow, the PC obtains the image list from the collected dataset and uploads the images sequentially to the device via ADB;

[0061] After the data upload is complete, the PC client issues the corresponding detection or recognition command via ADB, waits for completion, and queries the results.

[0062] The device executes detection or identification commands and reports the results;

[0063] After receiving the reported results on the PC, the image and the corresponding results are saved to an Excel report, and the success or failure probability is calculated.

[0064] Sending commands, querying results, and executing image upload and download commands are all implemented using Python, thereby enabling automatic data acquisition and data backflow based on large datasets.

[0065] When performing liveness detection, the PC client uses multiple images with the same image ID value in the same scene as input.

[0066] like Figure 2 As shown, the process of acquiring and recognizing facial data using the embedded device of this invention specifically includes the following steps:

[0067] Step A: Capture YUV data from cameras 1 to N and compare timestamps to ensure that multiple images were captured at the same time for liveness detection. If data compression is required, you can choose to encode the YUV images as MJPEG, which are lossy images.

[0068] Step B: Perform face detection and obtain the face detection result; if the face detection is successful, proceed to step C; otherwise, proceed to step A.

[0069] Step C: Perform a liveness detection and obtain the liveness detection result; if the liveness detection is successful, proceed to step D; otherwise, proceed to step A.

[0070] Step D: Perform face recognition and obtain the face recognition result; if the face recognition is successful, proceed to step E; otherwise, proceed to step A.

[0071] Step E: Save the image and all image attributes to a file. In embedded devices, if there is no SD device, you can also consider using a memory file system (RAMFS). RAMFS is a memory-based file system that works entirely in RAM. It is a virtual file system that can dynamically create temporary file systems in memory at runtime without using space on the hard disk.

[0072] All recognition scores and image attribute information are encapsulated in a file; specific rules can be customized. The main attribute information includes the following:

[0073]

[0074] The specific process of acquiring image data from an embedded device on a PC includes the following steps:

[0075] Step a: The PC sends a start recognition command via ADB SHELL;

[0076] Step b: Loop through the embedded device to check if it has successfully acquired and recognized the image, and obtain a list of successfully acquired images;

[0077] Step c: Send an image upload command via ADB to upload the acquired image and save the image to the specified folder. If the folder does not exist, create it.

[0078] Step d: Determine if the number of images acquired is greater than the threshold. If yes, proceed to step e; otherwise, proceed to step b.

[0079] Step e, End.

[0080] This method for processing correlations between multi-camera data uses the same ID value to determine image correlation. It is simple to implement, flexible, and applicable even to cases where image data is unrelated. More specifically, during data acquisition, the two images input to the liveness detection algorithm are labeled cam1_id100 and cam2_id100, where cam1 is the image data captured from camera 1 and cam2 is the image data captured from camera 2. id100 indicates an image captured at the same time point. During data backfeeding, images captured at the same time point need to be selected from the data collected from the two cameras and uploaded to the device. At this point, Python is used to parse the ID value from the file name; for example, an ID value of 100 is parsed from camera 1, and an image with ID 100 is parsed from camera 2. These two images are considered image pairs captured at the same time. ADB is used to upload the image pairs to the device, and the algorithm is executed for detection, thus achieving scene reconstruction. This method is convenient for applications such as algorithm problem localization and registration recognition.

[0081] like Figure 1 As shown, the present invention also discloses an automatic testing system for multi-channel camera data acquisition and data backflow based on an embedded system, which includes multiple cameras 1, an embedded device 2 and a PC 3;

[0082] Multiple cameras 1 are used to capture raw images;

[0083] Embedded device 2 is used to receive raw images from multiple cameras and save the image metadata. Images acquired simultaneously by multiple cameras have the same image ID value, which is used to determine the correlation between images. Embedded device 2 is also used to perform face detection and save the face detection results; perform liveness detection and save the liveness detection score as image attribute information; perform face recognition and save the face recognition result as image attribute information; and save the images as files, with the image attribute information encapsulated into regular image file names, so that the PC can parse the corresponding image attribute information from the file name. Embedded device 2 is also used to execute detection or recognition commands sent by the PC and report the results.

[0084] The PC connects to the embedded device's USB interface using a data cable. The embedded device is operated via Python on the PC using ADB. During data acquisition, the system polls to check for the existence of images, retrieves the image list via ADB, and downloads them sequentially to the PC. The PC saves the downloaded image data to a designated folder, whose name includes the camera ID, start acquisition time, and acquisition scene. During data backhaul, the PC retrieves the image list from the acquired dataset and uploads the images sequentially to the embedded device via ADB. After data upload is complete, the PC issues corresponding detection or recognition commands via ADB, waits for completion, and queries the results. Upon receiving the reported results from the embedded device, the PC saves the images and corresponding results to an Excel report and calculates the success or failure probability.

[0085] The above embodiments and illustrations are not intended to limit the product form and style of the present invention. Any appropriate changes or modifications made by those skilled in the art should be considered as not departing from the patent scope of the present invention.

Claims

1. A method for acquiring multi-channel camera data based on an embedded device, characterized in that, Includes the following steps: Step S1: In the camera module of the embedded device, capture YUV data from cameras 1 to N. Images captured by multiple cameras at the same time have the same image ID value, which is used to determine the correlation between images. Compare timestamps to ensure that multiple images were captured at the same time, which is used for liveness detection. Identify the N images corresponding to the liveness detection algorithm as cam1_idM to camN_idM, where cam1 is the image data captured from camera 1, camN is the image data captured from camera N, and idM represents the images captured at the same time point. Save the image's metadata. Step S2: Perform face detection, obtain face detection results, and save the face detection results as image attribute information. If face detection is successful, proceed to the step of performing liveness detection and obtaining liveness detection results; otherwise, proceed to the step of capturing YUV data from cameras 1 to N and comparing timestamps to ensure that multiple images captured at the same time are used for liveness detection. Step S3: Obtain multiple images at the same time, perform liveness detection, obtain the liveness detection result, and save the liveness detection score as image attribute information. If the liveness detection is successful, proceed to the step of performing face recognition and obtaining the face recognition result; otherwise, proceed to the step of capturing YUV data from camera 1 to N and comparing the timestamps to ensure that the multiple images are captured at the same time for use in the liveness detection step. Step S4: Perform face recognition, obtain the face recognition result, and save the face recognition result as image attribute information. If the face recognition is successful, proceed to the step of saving the image and all image attributes to a file; otherwise, proceed to the step of capturing YUV data from cameras 1 to N and comparing timestamps to ensure that the images are multiple channels captured at the same time for use in liveness detection. Step S5: Save the image and all image attribute information to a file. The image attribute information is encapsulated into a regular image file name, so that the PC can parse the corresponding image attribute information from the image file name. Step S6: Connect the embedded device to the USB interface using a data cable to connect the embedded device to the PC. The user can then operate the embedded device on the PC via ADB. Step S7: When acquiring image data, the PC client polls the embedded device to check if the acquired images exist, obtains the image list via ADB, and downloads them to the PC client sequentially via ADB. Step S8: The PC client saves the downloaded image data to the specified folder.

2. The multi-channel camera data acquisition method based on embedded devices as described in claim 1, characterized in that: The process of acquiring image data from an embedded device on a PC includes the following steps: Step a: The PC sends a start recognition command via ADB SHELL; Step b: Loop through the embedded device to check if it has successfully acquired and recognized the image, and obtain a list of successfully acquired images; Step c: Send an image upload command via ADB to upload the acquired image and save the image to the specified folder. If the folder does not exist, create it. Step d: Determine if the number of images acquired is greater than the threshold. If it is greater than the threshold, proceed to step e; otherwise, proceed to step b. Step e, End.

3. A multi-channel camera data acquisition system based on embedded devices, characterized in that: Includes multiple cameras, embedded devices, and PCs; Multiple cameras are used to capture raw images; The embedded device captures YUV data from cameras 1 to N. Images captured simultaneously by cameras 1 to N have the same image ID value, which is used to determine the correlation between images. Timestamps are compared to ensure that multiple images were captured at the same time, which is used for liveness detection. The N images input to the liveness detection algorithm are labeled as cam1_idM to camN_idM, where cam1 is the image data captured from camera 1, camN is the image data captured from camera N, and idM indicates the images captured at the same time point. The image's metadata is saved. The embedded device is used to perform face detection, obtain face detection results, save face detection results as image attribute information, obtain multiple images simultaneously, perform liveness detection, obtain liveness detection results, and save the liveness detection score as image attribute information, perform face recognition, obtain face recognition results, and save the face recognition results as image attribute information; and save the image and all image attribute information to a file. The image attribute information is encapsulated into a regular image file name, so that the PC can parse the corresponding image attribute information from the image file name. Embedded devices are also used to execute detection or identification commands sent from the PC. The PC uses a data cable to connect to the USB interface of the embedded device. The PC operates the embedded device via ADB using Python. When acquiring image data, the PC polls the embedded device to check if the acquired image exists, obtains the image list, and downloads it to the PC in sequence. The PC then saves the downloaded image data to a specified folder. Furthermore, the specific process for acquiring image data from embedded devices on a PC includes the following steps: Step a: The PC sends a start recognition command via ADB SHELL; Step b: Loop through the embedded device to check if it has successfully acquired and recognized the image, and obtain a list of successfully acquired images; Step c: Send an image upload command via ADB to upload the acquired image and save the image to the specified folder. If the folder does not exist, create it. Step d: Determine if the number of images acquired is greater than the threshold. If yes, proceed to step e; otherwise, proceed to step b. Step e, End.

Images