A multi-threaded detection method and device for a camera module

Abstract

The present invention relates to the field of image detection technology, and specifically discloses a camera module multi-thread detection method and device, wherein the method comprises the following steps: S1, selecting a corresponding sub-test item according to a test scenario; S2, setting a running time threshold of the sub-test item and its parameters; S3, adaptively extracting a preset number of original images of frames under a preset exposure environment according to the selected sub-test item, and then converting them into image data in a preset format; S4, multi-threadedly calling each sub-test item to detect the sub-test item; S5, terminating the thread when each thread ends running or exceeds the running time threshold of the sub-test item. The technical solution of the present invention can effectively improve the detection efficiency of the camera module.

Description

Technical Field

[0001] The present invention relates to the technical field of image detection, and in particular to a camera module multi-thread detection method and device. Background Art

[0002] At present, the production process and testing methods of camera modules from low-end cameras to high-end cameras are becoming more and more complex. Therefore, camera module factories need to involve many tests in the process of producing camera modules, including OC (optical center), POD (dead pixel), POG (dirty), Shading (brightness Yshading and color ColorShading, including RGShading, BGShading and GGshading), OB (Dark shading), FPN (black field and white field lines), RTSNoise and other tests. In this process, its testing methods often determine its production efficiency. The traditional test is to light up the camera first, expose it to a certain range to capture the picture and then do relevant tests. This method often has the problem of low UPH (unit hour production).

[0003] Therefore, a camera module multi-threaded detection method and device are needed that can effectively improve the detection efficiency. Summary of the invention

[0004] One of the purposes of the present invention is to provide a camera module multi-threaded detection method, which can effectively improve the detection efficiency of the camera module.

[0005] In order to solve the above technical problems, this application provides the following technical solutions:

[0006] A camera module multi-thread detection method comprises the following steps:

[0007] S1. Select the corresponding sub-test item according to the test scenario;

[0008] S2. Set the running time threshold and parameters of the sub-test items;

[0009] S3, adaptively extracting a preset number of frames of original images under a preset exposure environment according to the selected sub-test item, and then converting them into image data in a preset format;

[0010] S4, calling each sub-test item by multiple threads to detect the sub-test items;

[0011] S5. When each thread finishes running or exceeds the running time threshold of the sub-test item, the thread is terminated.

[0012] The basic scheme principles and beneficial effects are as follows:

[0013] The existing camera module detection is to complete one test before proceeding to the next test. This test is very convenient and has low requirements for operation, so it has been used all the time, but the disadvantage of low efficiency has always existed. In this scheme, the corresponding sub-test items are selected according to the test scenario, so that multiple sub-test items can be tested at the same time. The running time threshold and its parameters are set for each sub-test item to ensure that each sub-test item can run normally. Moreover, the running time threshold can avoid a sub-test item from running for a long time and prevent the long-term occupation of test resources. Since the number of frames of the original image and the format of the image data required by each sub-test item are different, the original image of the preset number of frames is adaptively extracted and then converted into image data in a preset format. When multiple sub-test items are tested at the same time, appropriate detection data can be collected for each sub-test item in a targeted manner. Finally, after completing the above preparations, multi-threaded calls are made to each sub-test item to detect the sub-test item, that is, each sub-test item is detected at the same time, which can effectively save time.

[0014] In summary, this solution can adaptively process the general detection of camera modules in a multi-threaded manner, which can greatly improve the running time, effectively improve the detection efficiency, and facilitate camera module manufacturers to conduct quick and convenient detection.

[0015] Furthermore, the step S2 also includes: collecting the parameters of each sub-test item and the actual running time when the corresponding thread ends normal operation, using the parameters and the actual running time of each sub-test item as training samples, inputting the training samples into the BP neural network model for training, and obtaining a trained BP neural network model; inputting the set parameters into the trained BP neural network model to obtain the predicted running time of the corresponding sub-test item, and using the predicted running time as the running time threshold.

[0016] In traditional single-item tests, the time for each test is fixed, that is, the running time of each test is consistent. This preferred solution will terminate the thread when each thread finishes running. In other words, when the test is completed, even if the fixed time has not been reached, the thread will be terminated, which is more flexible. In addition, the BP neural network model is trained according to a certain number of samples, and then the set parameters are input into the trained BP neural network model to obtain the predicted running time of the corresponding sub-test items. In this way, the running time threshold of each sub-test item is dynamically adjusted according to the test situation of each thread. When the test time of a thread is too long, the thread can be terminated to reduce the impact on UPH. Compared with using a fixed running time threshold, it has better adaptability.

[0017] Further, the method further comprises the following steps:

[0018] S6. Obtain the test results of each sub-test item;

[0019] S7. Analyze the test results of each sub-test item and count the number of test failures based on the test results.

[0020] Further, the method further comprises the following steps:

[0021] S8, judging whether the number of sub-test item failures is 0;

[0022] If the number of sub-test item failures is 0, jump to step S9;

[0023] If the number of sub-test item failures is greater than 1, an error result is directly returned and the test is terminated;

[0024] If the number of test failures of the sub-test item is equal to 1, the corresponding image data is re-acquired for separate testing, and the test result of the sub-test item is returned. It is determined again whether the number of test failures is 0. If so, the process jumps to step S9; otherwise, an error result is returned and the test is terminated;

[0025] S9. If the returned test results do not contain any test failures, the test is passed.

[0026] When the number of sub-test item failures is equal to 1, a secondary test is performed to improve the primary yield and reduce the misjudgment rate. If the test result is still a failure, it means that there is indeed a problem with the sub-test item, otherwise the sub-test item is judged to be correct.

[0027] Furthermore, in step S1, sub-test items with the same color temperature, the same illumination and the same exposure are selected.

[0028] A second object of the present invention is to provide a camera module multi-thread detection device, comprising:

[0029] An environment setting unit, used to set the exposure environment according to the selected sub-test item;

[0030] An image extraction unit, used for adaptively acquiring original images of N frames according to the selected sub-test item and converting them into image data in a preset format, where N≥1 and N is a positive integer;

[0031] A parameter configuration unit, used to configure respective image detection parameters according to the selected sub-test items, the parameters are M groups, M ≥ 2 and M is a positive integer;

[0032] An estimation unit, used for determining the running time of each image detection unit according to the selected sub-test items, and setting the running time threshold and thread of the image detection unit;

[0033] A detection thread unit is used to call each image detection unit in multiple threads and run the image detection unit;

[0034] The image detection unit is used to detect the image data after the format conversion according to the selected sub-test items.

[0035] In this solution, the exposure environment is set according to the selected sub-test items by the environment setting unit, so that the selected sub-test items can be ensured to be under the same exposure environment.

[0036] Since the number of frames of the original image and the format of the image data required by each sub-test item are not the same, the image extraction unit adaptively extracts the original image of the number of frames N, and then converts it into image data in a preset format. When multiple sub-test items are tested at the same time, appropriate detection data can be collected for each sub-test item in a targeted manner. The estimation unit sets the running time threshold of its image detection unit; this can avoid a certain test item from running for a long time and prevent long-term occupation of test resources. The detection thread unit multi-threads and calls each image detection unit to detect the sub-test items, that is, to detect each sub-test item at the same time, which can effectively save time. In summary, this solution can adaptively process the general detection of camera modules in multiple threads, which can greatly improve the running time, effectively improve the detection efficiency, and facilitate the camera module factory to conduct quick and convenient detection.

[0037] Furthermore, the system also includes a thread processing unit, which is used to obtain the test result of the image detection unit and close the detection thread unit when the execution time of the image detection unit exceeds its running time threshold or the thread is executed.

[0038] In traditional single-item testing, each test is performed in a fixed time, that is, the running time of each test is consistent. In this preferred solution, the thread processing unit will terminate the thread when each thread ends or exceeds the running time threshold of the sub-test item. In other words, when the test is completed, the thread will be terminated even if the fixed time is not reached. This is more flexible. When the running time exceeds the running time threshold, the thread will be terminated, which can avoid a certain test item from running for a long time and prevent long-term occupation of test resources.

[0039] Furthermore, the system also includes an analysis unit: configured to perform image data analysis according to the test results of the image detection units of each thread, and to count the number of test failures of its sub-test items. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a flow chart of a camera module multi-thread detection method according to Embodiment 1;

[0041] Figure 2 This is a logic block diagram of a camera module multi-thread detection device according to embodiment 1. DETAILED DESCRIPTION

[0042] The following is further described in detail through specific implementation methods:

[0043] Embodiment 1

[0044] like Figure 1 As shown, a camera module multi-thread detection method of this embodiment includes the following steps:

[0045] S1. Select the corresponding sub-test items according to the test scenario. For example, specify the sub-test items according to the project scenario and customer needs. In black field, OB, FPN, POD and even RTS noise need to be tested. In white field, OC, POD, POG, Shading and FPN need to be tested.

[0046] S2. Set the running time threshold of the sub-test items and their corresponding parameters. Since the running time of each sub-test item is related to the camera chip resolution, that is, the image data, the running time threshold of different sub-test items is different. For example, for a 12Meg camera, the OC test optical center and Shading test time should be less than 0.5 seconds, the POD white field card bad pixel and FPN detection line test time should be less than 1 second, and the POG dirt test time should be less than 2 seconds. It will be different for low-end or high-end cameras. For example, for high-end cameras (16Meg and above, 16M\30M\48M\64M\108M, etc.): OC\shading\FPN time is within 1.5 seconds; POD is within 2 seconds; POG is within 3 seconds; for example, for low-end cameras (below 16Meg): OC\shading\FPN time is within 1 second; POD is within 1.5 seconds; POG is about 1.5 seconds; the specific running time threshold in this embodiment can be set on the host computer interface during the project debugging phase.

[0047] There are many related parameter settings for each sub-test item. For example, OC needs to set its chip pixel size, the distance threshold between the optical center and the physical center; Shading needs to set the card frame size, brightness range, RGshading and BGshading range, four-corner brightness difference, etc.; POD needs to set the bad pixel concentration, bad pixel type and number of bad pixels on the white field card; POG needs to set each field of view threshold, edge and four-corner threshold, etc.; FPN lines need to set the horizontal and vertical adjacent line differences; OB needs to set the card frame size, maximum threshold and minimum threshold of each channel, etc.; at the same time, the image data type of each test item is not the same, for example, OC uses bmp data, Shading uses Raw10 data; POG uses the fusion of multiple bmp data, etc.

[0048] S3. According to the selected sub-test items, the original image of the preset number of frames is adaptively extracted under the preset exposure environment, and then converted into image data in a preset format and stored in the corresponding memory unit. Select the exposure environment. For example, in the white field, a suitable light source is required, such as D50, color temperature 5100±100K, brightness 1000±100lux, light up the camera and select the appropriate exposure. Generally, the G channel is set to about 150-200; for another example, in the black field, directly cover the blackboard for testing.

[0049] Specifically, when capturing image data, one or more frames of original images are collected from the test box (how many frames are collected is selected according to the sub-test items); and then converted into a corresponding format according to the selected sub-test items.

[0050] The original image is generally in the format of mipi raw10\YUV422 8bit\YUV422 10bit.

[0051] For example, OC: convert the original image to bmp and then to grayscale;

[0052] shading: convert the original image to raw10;

[0053] POD: Convert the original image to raw10 and then to raw8;

[0054] POG: Convert 2 or 3 frames of original images to bmp;

[0055] FPN: Convert the original image to raw10;

[0056] OB: Convert 2, 3 or 4 frames of raw images to raw10.

[0057] S4. Multiple threads call each sub-test item to test the sub-test items. For example, the burning project generally needs to open the sub-test items of the following threads for testing:

[0058] White field: OC / Shading / POD / POG / FPN / LSC,

[0059] Black field: POD / FPN / OB.

[0060] For another example, if the project is not burned, it is generally necessary to enable the following sub-test items for testing:

[0061] White field: OC / Shading / POD / POG / FPN,

[0062] Black field: POD / FPN / OB.

[0063] S5. When each thread finishes running or exceeds the running time threshold of the sub-test item, the thread is terminated;

[0064] S6. Obtain the test results of each sub-test item;

[0065] S7, analyzing the test results of each sub-test item, and counting the number of test failures according to the test results. Specifically, the judgment criteria for test failures of each sub-test item need to be pre-set, and then the number of test failures is counted by analyzing the error logs of each test item.

[0066] For example, FPN line defects are divided into horizontal lines and vertical lines. There may be 1 line, 2 lines, 4 lines, 8 lines, and 16 lines stuck in the image. The result is obtained by comparing the overall pixel values ​​of the front and back columns and the upper and lower rows to determine whether the pixel values ​​exceed the set threshold, and the row or column information of the existing lines is saved at the same time.

[0067] OC optical center: calculate the actual center coordinates of the image, and then compare them with the physical coordinates. If they are inconsistent, it is bad. For example, for an image with a resolution of 8000*6000, the physical coordinates are the theoretical center (4000, 3000), and the optical center is calculated to be 3800, 4100. Then compare the distance between the optical center and the physical center.

[0068] POD: Calculates the bad pixels in the entire image. If the bad pixels exceed the set threshold, the bad pixel type, coordinates, and image of the bad pixels are recorded and saved and marked.

[0069] S8, determine whether the number of sub-test item failures is 0; if the number of sub-test item failures is 0, jump to step S9; if the number of sub-test item failures is greater than 1, directly return an error result and terminate the test; if the number of sub-test item failures is equal to 1, re-acquire its corresponding image data and perform a separate test, and return the test result of the sub-test item, and determine again whether the number of failures is 0, if so, jump to step S9; otherwise, return an error result and terminate the test. In order to improve the primary yield and reduce the misjudgment rate, a secondary test is performed. If the test result is still a failure, it means that the sub-test item is indeed a failure, otherwise the sub-test item is correctly judged. This improves the efficiency of the entire project, thereby increasing UPH. In other embodiments, the number of repeated tests in which the number of sub-test item failures is equal to 1 and the number of repeated tests after retesting is 0 is counted in each thread running simultaneously, and the number of repeated tests exceeding the first threshold is marked as satisfying the first condition. When the number of threads running the same sub-test item reaches a certain number, such as 100 times, it can be set according to actual conditions to determine whether the ratio of the number of times the first condition occurs to the total number of times exceeds the second threshold. If it exceeds, a reminder to replace the sub-test item is generated. A reminder can be given to reselect a different sub-test item, and then the reselected sub-test item is used for multi-threaded detection, so as to eliminate test failures caused by hidden problems such as interference during detection.

[0070] S9. If there is no test failure in the test results returned in sequence, the test is passed.

[0071] Based on a camera module multi-thread detection method, this embodiment also provides a camera module multi-thread detection device, such as Figure 2 As shown, including:

[0072] The environment setting unit is used to set the exposure environment according to the selected sub-test items. For example, in the white field test, the appropriate color temperature and illumination are set, and then the exposure is to the appropriate brightness. In the black field test, the scene is completely black to ensure that there is no light leakage.

[0073] The image extraction unit is used to adaptively acquire the original image of the number N frames according to the selected sub-test item and convert it into image data in a preset format, wherein N≥1 and N is a positive integer.

[0074] The parameter configuration unit is used to configure respective image detection parameters according to the selected sub-test items. The parameters are M groups, M≥2 and M is a positive integer, and M represents M threads, that is, the number of sub-test items.

[0075] The estimation unit is used to determine the running time of each image detection unit according to the selected sub-test items, and set the running time threshold and thread of its image detection unit.

[0076] The detection thread unit is used to call each image detection unit in a multi-threaded manner and run the image detection unit.

[0077] The image detection unit is used to detect the image data after the format conversion according to the selected sub-test items.

[0078] The thread processing unit is used to obtain the test result of the image detection unit and close the detection thread unit when the execution time of the image detection unit exceeds its running time threshold or the thread is executed;

[0079] Analysis unit: used to analyze image data according to the test results of each thread image detection unit, count the number of test failures of its sub-test items, and provide corresponding feedback based on the number of test failures.

[0080] Specifically, if the number of test failures of a sub-test item is greater than 1, an error result is directly returned and the test is terminated; if the number of test failures of a sub-test item is equal to 1, the corresponding image data is recaptured, and the image detection unit performs detection alone, and returns the test result of the sub-test item, and again determines whether the number of test failures is greater than or equal to 1. If so, the test is terminated; if there are no test failures in the test results returned in sequence, the test passes.

[0081] For example, for a 12.5Meg project with a resolution of 4080*3072, the white field test needs to test OC, Shading, POD, POG, and FPN. Under the same environment, relevant parameters and running time thresholds and other information are set. Compared with the method without multi-threaded adaptive operation, it is found according to statistics that the running time of the solution of this embodiment is about 1.867 seconds, while the running time without this method is about 4.913 seconds, with a time difference of more than 3 seconds. The running rate is significantly improved, thereby greatly improving its UPH.

[0082] For example, for a 12.5Meg project with a resolution of 4080*3072, the black field test needs to test OB, POD, and FPN. Under the same environment, relevant parameters and running time thresholds and other information are set. Compared with the method without multi-threaded adaptive operation, it is found according to statistics that the running time of the solution of this embodiment is about 1.473 seconds, while the running time without this method is about 4.233 seconds, with a time difference of about 2.76 seconds. The running rate is significantly improved, thereby greatly improving its UPH.

[0083] As described above, the multi-threaded adaptive image detection method and device provided in this embodiment can greatly improve the running time by multi-threaded adaptive processing of the general detection mode of high-order and low-order camera modules, which is convenient for camera module factories to perform quick and convenient detection. Moreover, the general detection items that the module factory needs to burn or not burn can be multi-threaded according to this scheme, which has strong iterativeness and can be expanded to the general test items involved in the future.

[0084] In the actual process, one or more sub-test items may be used, and they can be adjusted according to the actual situation. They are applicable to both high-end and low-end camera projects, and can also be used in mobile phones, automotive, medical and other fields, with high flexibility.

[0085] Embodiment 2

[0086] The difference between this embodiment and the first embodiment is that step S2 of the method of this embodiment further includes collecting parameters of each sub-test item and the actual running time when the corresponding thread ends normal operation, using the parameters and the actual running time of each sub-test item as training samples, inputting the training samples into the BP neural network model for training, and obtaining a trained BP neural network model; inputting the set parameters into the trained BP neural network model to obtain the predicted running time of the corresponding sub-test item, and using the predicted running time as the running time threshold.

[0087] On the basis that the running time of the sub-test items is related to the resolution of the camera chip, in this embodiment, the running time threshold of each sub-test item is also dynamically and finely adjusted according to the test conditions of each thread, that is, within a large range determined according to the resolution of the camera chip, the range is further narrowed. This can make the running time threshold more closely aligned with the corresponding sub-test item, and the running time threshold will not be set too short, resulting in the test being unable to proceed normally, nor will the running time threshold be set too long, affecting the test efficiency, thereby improving its UPH.

[0088] The camera module multi-thread detection device of this embodiment uses the above method.

[0089] Embodiment 3

[0090] The difference between this embodiment and the second embodiment is that in step S1 of the method of this embodiment, sub-test items with the same color temperature, illumination and exposure are selected. The multi-threaded test of this embodiment can ensure that multiple sub-test items running at the same time will not conflict with each other and avoid affecting the test efficiency by matching and setting the characteristics of each sub-test item (color temperature, illumination exposure, image format, number of frames, running time threshold, and sub-test item parameters). In other embodiments, the probability of each sub-test item exceeding the running time threshold can also be counted, and the sub-test items with the same color temperature, illumination and exposure with a probability greater than a preset value are selected for multi-threaded detection, which can further improve efficiency.

[0091] The above are only embodiments of the present invention. The invention is not limited to the fields involved in this implementation case. The common sense such as the known specific structures and characteristics in the scheme is not described in detail here. The ordinary technicians in the relevant field know all the common technical knowledge in the technical field to which the invention belongs before the application date or the priority date, can obtain all the existing technologies in the field, and have the ability to apply the conventional experimental means before that date. The ordinary technicians in the relevant field can improve and implement this scheme in combination with their own abilities under the enlightenment given by this application. Some typical known structures or known methods should not become obstacles for ordinary technicians in the relevant field to implement this application. It should be pointed out that for those skilled in the art, without departing from the structure of the present invention, several deformations and improvements can be made, which should also be regarded as the protection scope of the present invention, which will not affect the effect of the implementation of the present invention and the practicality of the patent. The protection scope required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to explain the content of the claims.

Claims

1. A camera module multi-thread detection method, characterized in that: The steps include: S1. Select corresponding sub-test items according to the test scene; wherein sub-test items with the same color temperature, the same illumination and the same exposure are selected; S2. Set the running time threshold and parameters of the sub-test items; Collect the parameters of each sub-test item and the actual running time when the corresponding thread ends normal operation, use the parameters of each sub-test item and the actual running time as training samples, input the training samples into the BP neural network model for training, and obtain the trained BP neural network model; input the set parameters into the trained BP neural network model, obtain the predicted running time of the corresponding sub-test item, and use the predicted running time as the running time threshold; S3, adaptively extracting a preset number of frames of original images under a preset exposure environment according to the selected sub-test item, and then converting them into image data in a preset format; S4, calling each sub-test item by multiple threads to detect the sub-test items; S5. When each thread finishes running or exceeds the running time threshold of the sub-test item, the thread is terminated; S6. Obtain the test results of each sub-test item; S7, analyzing the test results of each sub-test item, and counting the number of test failures according to the test results; S8, judging whether the number of sub-test item failures is 0; If the number of sub-test item failures is 0, jump to step S9; If the number of sub-test item failures is greater than 1, an error result is directly returned and the test is terminated; If the number of test failures of the sub-test item is equal to 1, the corresponding image data is re-acquired for separate testing, and the test result of the sub-test item is returned. It is determined again whether the number of test failures is 0. If so, the process jumps to step S9; Otherwise, an error result is returned and the test is terminated; S9. If the returned test results do not contain any test failures, the test is passed.

2. A camera module multi-thread detection device, characterized in that: Executing the camera module multi-thread detection method as claimed in claim 1, comprising: An environment setting unit, used to set the exposure environment according to the selected sub-test item; The image extraction unit is used to adaptively obtain the original image of the number N frames according to the selected sub-test item and convert it into image data in a preset format, wherein and N is a positive integer; The parameter configuration unit is used to configure the respective image detection parameters according to the selected sub-test items. The parameters are M groups. and M is a positive integer; The estimation unit is used to determine the running time of each image detection unit according to the selected sub-test items, set the running time threshold and thread of its image detection unit; collect the parameters of each sub-test item and the actual running time when the corresponding thread ends normal operation, use the parameters of each sub-test item and the actual running time as training samples, input the training samples into the BP neural network model for training, and obtain the trained BP neural network model; input the set parameters into the trained BP neural network model, obtain the predicted running time of the corresponding sub-test item, and use the predicted running time as the running time threshold; A detection thread unit is used to call each image detection unit in multiple threads and run the image detection unit; The image detection unit is used to detect the image data after the format conversion according to the selected sub-test items.

3. The camera module multi-thread detection device according to claim 2, characterized in that: The system also includes a thread processing unit, which is used to obtain the test result of the image detection unit and close the detection thread unit when the execution time of the image detection unit exceeds its running time threshold or the thread is executed.

4. The camera module multi-thread detection device according to claim 3, characterized in that: The method also includes an analysis unit: used to perform image data analysis according to the test results of each thread image detection unit, and count the number of test failures of its sub-test items.

Images