Axial angle deviation correction system and correction method of dual-optical-domain camera

Abstract

The invention discloses an axial angle deviation correction system and correction method for a dual-optical-domain camera, relates to the field of intelligent image processing, and solves the problems that in an existing dual-optical-domain system, off-axis deviation exists, rotation errors exist in CMOS image sensor assembly, then axial angle deviation exists, and use requirements are further difficult to meet. According to the method, a test fixture is adopted, two cameras are used for shooting the same target screen to obtain two images, then a processor is used for recognizing marked graphs in the centers of the images and calculating the magnification ratio of the two cameras, meanwhile, the center coordinates of the marked graphs are automatically recognized, and correction parameters used for correcting the axial angle deviation of the two cameras are calculated; storing in a memory; in actual use, the processor calculates the images shot by the two cameras by using correction parameters, and finally realizes the superposition of the two images. According to the method, the speed and accuracy of intelligent recognition are improved, and double-object attacks initiated by hackers using two different sceneries on the two cameras are effectively prevented.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image visual recognition technology, and particularly relates to a system and method for correcting axis angle deviation of a dual-light-field camera. BACKGROUND

[0002] Image visual recognition technology has been widely used in security authentication, financial payment, and intelligent control, etc. The method is to first capture an image by a camera, and then automatically identify the object in the image by a processor. A dual-light-field image recognition system is composed of two cameras which capture images from the same perspective in the color visible spectrum and infrared spectrum respectively, using the difference characteristics of the object surface in different wavelengths of light. The system can significantly improve the security and anti-counterfeiting capability of the system and improve the accuracy of image recognition. In particular, the focal length and resolution of the two cameras in the dual-light-field system are set to be different, which can further improve the security and anti-counterfeiting level while fully utilizing the performance of the two cameras. In the actual production and assembly process of the product, because of mechanical errors, the images captured by the two cameras cannot completely coincide. The errors of a single camera mainly include off-axis deviation caused by the fact that the main optical axis of the optical unit does not pass through the center point of the CMOS image sensor, and rotation error caused by the assembly of the CMOS image sensor. In the dual-light-field camera, the angle error between the main optical axes of the optical unit elements of the two cameras is called axis angle deviation, which negatively affects the system and increases with the increase of the distance of the scene. The angle error is the main error and is relatively large in batch production. Even if the manual fine adjustment and repeated correction are performed, it is difficult to meet the use requirements, which affects the production efficiency and product performance.

[0003] In summary, the present application provides a method and system for correcting the axis angle deviation of a dual-light-field camera. SUMMARY

[0004] The present application provides a system and method for correcting the axis angle deviation of a dual-light-field camera to solve the problems in the prior art, such as the existence of off-axis deviation and rotation error of the CMOS image sensor assembly, which further leads to axis angle deviation and makes it difficult to meet the use requirements.

[0005] The system for correcting the axis angle deviation of a dual-light-field camera includes a test fixture and a dual-light-field camera structure arranged in the test fixture.

[0006] The test fixture adopts a dark box, a target screen is fixed on the inner top of the dark box, and a mark pattern is arranged at the center of the target screen. A test table is fixed on the inner bottom of the dark box, and the two cameras in the dual-light-field camera structure are fixed on the test table.

[0007] The two cameras shoot the target screen to obtain two images, a processor is used to identify the mark pattern in the center of the image, and the magnification ratio of the two cameras is automatically calculated according to the size of the side length of the mark pattern, and the center coordinates of the mark pattern are automatically identified;

[0008] The processor calculates the correction parameters for correcting the axial angle deviation of the two cameras, and saves the correction parameters in the memory of the dual-optical field camera structure; according to the correction parameters, the axial angle deviation of the dual-optical field camera is corrected, so that the two images of the same scene shot by the two cameras are overlapped.

[0009] The application also provides a method for correcting the axial angle deviation of a dual-optical field camera, which is realized by the axial angle deviation correction system of the dual-optical field camera, and the specific steps of the method are as follows:

[0010] Step 1. Each pixel in the shot image is calculated as a brightness pixel and saved in image B, and the average value C of all pixels in image B is calculated as the threshold value C of the first binary.

[0011] Step 2. The threshold value C is used to binary image B to obtain image D including bright spots and dark spots; the pixel value greater than the threshold value C in image B is set as a bright spot with a value equal to 1 in image D, and the others are set as dark spots with a value of 0.

[0012] Step 3. An image E is initialized, and all pixel values in image E are set to 0.

[0013] Step 4. Each pixel in image D is analyzed row by row from left to right and from top to bottom, specifically, when a pixel in image D is taken, its coordinate value is first saved in variable P, and then its value is analyzed, if the value is 0, step 5 is executed, otherwise, eight adjacent pixels including the upper left corner, the upper side, the upper right corner, the right side, the lower right corner, the lower side, the lower left corner and the left side of the pixel are obtained, and if there is a bright spot with a pixel value equal to 1 in the eight adjacent pixels, the pixel value in image E with the same coordinate as variable P is set to 1.

[0014] Step 5. Step 4 is executed in a loop until all pixels in image D are analyzed and image E is updated.

[0015] Step 6. Define two variables G and H with initial value equal to zero; set the pixel gradual change difference threshold equal to three percent of the maximum brightness of the image, then scan the image B and image E simultaneously in the direction of traversing the pixels from left to right in each row and from top to bottom in each row, pixel by pixel; if a pixel with value 1 is found in image E, save its coordinates in variable P first, then get the pixel in image B with coordinates equal to variable P, compare it with threshold value C, if the difference is greater than the pixel gradual change difference threshold, add the brightness value of this pixel to variable G and add 1 to variable H, otherwise, consider that this pixel is located at the position of the brightness gradual change boundary and does not need to be processed; perform the above search until all coordinates in image E are analyzed;

[0016] Step 7. Divide the value of variable G by variable H to obtain the second binary threshold J; use threshold J to perform binary processing on image B to obtain image K including bright spots and dark spots;

[0017] Step 8. Define a rectangle T with the upper left corner horizontal coordinate equal to one quarter of the width of image K, the upper left corner vertical coordinate equal to one quarter of the height of image K, the right lower corner horizontal coordinate equal to three quarters of the width of image K, and the right lower corner vertical coordinate equal to three quarters of the height of image K; require that the target screen center mark pattern must be completely located within the area of rectangle T in image K;

[0018] Step 9. Perform erosion on image K first, then perform dilation to remove possible small noise points in image K within rectangle T;

[0019] Step 10. Define R rectangle, that is, define a variable R of the type of rectangle data structure to describe the rectangle, which has two sub-variables, one for describing the upper left corner coordinates of the rectangle and the other for describing the right lower corner coordinates of the rectangle; clear the value of variable H; the initial upper left corner coordinates of R are equal to the right lower corner coordinates of rectangle T, and the initial right lower corner coordinates of R are equal to the upper left corner coordinates of rectangle T;

[0020] Step 11. Perform dilation on image K; when dilation is performed, analyze the relationship between the coordinates of the dark spots encountered and the variable R rectangle, and set the R rectangle according to the situation, specifically;

[0021] If the horizontal coordinates of a dark spot are found to be less than the upper left corner horizontal coordinates of the R rectangle, set the upper left corner horizontal coordinates of the variable R rectangle to the horizontal coordinates of this dark spot;

[0022] If the horizontal coordinates of a dark spot are found to be greater than the right lower corner horizontal coordinates of the R rectangle, set the right lower corner horizontal coordinates of the variable R rectangle to the horizontal coordinates of this dark spot;

[0023] If the vertical coordinate of the dark spot is found to be less than the vertical coordinate of the upper left corner of the R rectangle, the variable vertical coordinate of the upper left corner of the R rectangle is set to the vertical coordinate of the dark spot;

[0024] If the vertical coordinate of the dark spot is found to be greater than the vertical coordinate of the lower right corner of the R rectangle, the variable vertical coordinate of the lower right corner of the R rectangle is set to the vertical coordinate of the dark spot;

[0025] Step 12. After the K image inflation is completed, if the width and height of the R rectangle are both greater than 1 pixel, the value of the variable H is added by 1, and then step 11 is executed, otherwise step 13 is executed.

[0026] Step 13. The average of the horizontal coordinates of the upper left corner and the lower right corner of the R rectangle is calculated as the horizontal coordinate of the center of the black square in the recognized image, and the average of the vertical coordinates of the upper left corner and the lower right corner of the R rectangle is calculated as the vertical coordinate of the center of the black square in the recognized image; then the value of the variable H is multiplied by 2, and if the width or height of the R rectangle is equal to 1 pixel, the variable H is added by 1; at this time, the value of H is equal to the side length of the black square identified in the captured image.

[0027] Advantages of the present application:

[0028] 1. The method of the present application can no longer require careful mechanical correction during production, and the axis angle deviation of the two cameras in the dual optical domain can be automatically corrected by using a jig. In common long-range large-field conditions, the off-axis deviation can also be considered as part of the axis angle deviation and corrected together. This correction work can be combined with testing and inspection on the production line, without the need for additional manual operation steps.

[0029] 2. In the application scenario of biological security identification, although the traditional ordinary dual optical domain camera uses the different spectral characteristics of biological bodies to improve the security and anti-counterfeiting performance, hackers still have the opportunity to make two cameras shoot two different targets to impersonate the same security authentication. Since the axis angle deviation is corrected by using the method of the present application, the processor can limit the coordinates of the objects shot by the two cameras to coincide during the identification process, effectively preventing double-object attacks and having strong security and anti-counterfeiting capabilities.

[0030] For example, in a face recognition application, if an attacker wears a mask, he can impersonate a legitimate user under visible light conditions, attacking and cracking the security system; to enhance security, an additional identification and authentication in infrared light environment can be added to form a dual light domain system; although this system improves security, if the attacker provides a visible light image and an infrared image at the same time in the same field of view, because the existing identification method does not require the two objects to coincide in space, the two cameras will each be authenticated successfully, and this attack method is called dual object attack, which threatens the security of the dual light domain system; in order to prevent the dual object attack of the dual light domain image identification system, the system described in the present application requires that the objects in the visible light image and the infrared image must coincide in space, because the hacker cannot provide two fake objects that overlap in space, therefore, the system provided by the present application can effectively prevent the dual object attack. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 A dual light domain camera system circuit diagram is described in the present application;

[0032] Figure 2 A dark box internal structure diagram is described in the present application;

[0033] Figure 3 A fast installation of a nail diagram is described in the present application;

[0034] Figure 4 A test bench fast installation of a nail diagram is described in the present application.

[0035] In the figure, 1 is a target screen, 2 is a target screen center black square, 3 is a dual light domain camera circuit board, 4 is a first camera, 5 is a second camera, 6 is a test bench, 7 is a nail cap, 8 is a spring, 9 is a nail body, 10 is a nail neck, 11 is a nail head, 12 is a fast installation of a nail, 13 is a support column, 14 is a large circular hole, 15 is an iron pad, 16 is a fast installation of a nail hole, 17 is a wiring slot, 18 is a thimble, and 19 is a connector. DETAILED DESCRIPTION

[0036] Specific implementation one, combined Figures 1 to 4 To illustrate this embodiment, a dual light domain camera axis angle deviation correction system is described, which consists of two parts, namely: dual light domain camera structure and test fixture; the dual light domain camera is a product produced by the factory, and the error performance is different; the test fixture is a set of devices used on the production line, and each set of dual light domain camera product must be calibrated by the test fixture before leaving the factory, and the correction parameters are saved in the read-only memory of the camera; using the corrected system, the images taken by the two cameras in the dual light domain camera are calculated using the correction parameters, and the accurate mapping of any pixel coordinates between the infrared image and the color image can be realized, and the two images are overlapped together to determine that the two cameras are shooting the same scene.

[0037] In this embodiment, the dual-light-field camera structure includes an infrared camera IR, a color camera RGB, a processor CPU, a memory ROM, and an external interface IO.

[0038] In order to ensure that the view angles of the infrared camera IR and the color camera RGB are as close as possible, the two cameras need to be installed as close as possible. If a fill light is needed, the fill light also needs to be as close as possible to the cameras so that the cameras can receive the reflected light of the scene after being filled without shadows. Because the fill light may interfere with the camera work due to side light leakage or front shell inner surface reflection, a light shielding measure needs to be designed between the fill light and the camera. Typically, a closed light guide pipe made of black porous foam is used to directly guide the light emitted by the fill light to the light-emitting window of the shell.

[0039] During the correction process, the dual-light-field camera is placed in the test fixture, and the processor CPU analyzes the images captured by the infrared camera IR and the color camera RGB respectively, calculates a set of correction parameters, and saves them to the memory ROM. The correction of the axial angle deviation of the dual-light-field camera is realized.

[0040] In this embodiment, the test fixture includes a dark box, a target screen 1, and a test table surface 6. The dark box is a cuboid, and the target screen 1 and the test table surface 6 are fixedly installed inside the dark box. The target screen 1 is fixed to the inner top surface of the dark box, and the test table surface 6 is fixed to the inner bottom surface of the dark box. The dual-light-field camera to be tested is temporarily fixed to the test table surface 6 with quick mounting pins, and the image of the target screen opposite the dark box is captured.

[0041] Figure 2 The dark box is a cuboid, and the target screen 1 and the test table surface 6 are fixedly installed inside the dark box. The target screen 1 is fixed to the inner top surface of the dark box, and the test table surface 6 is fixed to the inner bottom surface of the dark box. The dual-light-field camera to be tested is temporarily fixed to the test table surface 6 with quick mounting pins, and the image of the target screen opposite the dark box is captured. Figure 2 In this embodiment, the test table surface 6 is a plane made of a thick insulating plate with a thickness greater than 1 cm. It has quick mounting pin holes corresponding to the fixing hole positions on the circuit board of the camera to be tested, and a spring-loaded pin 18 is electrically connected to an external interface IO connector 19 on the circuit board 3. The spring-loaded pin 18 is connected to an electric wire that is led out from a wiring groove 17 under the thick insulating plate to a power supply and a test computer outside the test fixture.

[0042] In this embodiment, the test table surface 6 is a plane made of a thick insulating plate with a thickness greater than 1 cm. It has quick mounting pin holes corresponding to the fixing hole positions on the circuit board of the camera to be tested, and a spring-loaded pin 18 is electrically connected to an external interface IO connector 19 on the circuit board 3. The spring-loaded pin 18 is connected to an electric wire that is led out from a wiring groove 17 under the thick insulating plate to a power supply and a test computer outside the test fixture.

[0043] As Figure 3 shown in the embodiment, the fast mounting pin 12 is composed of a pin cap 7, a spring 8, a pin body 9, a pin neck 10, and a pin head 11. The cross section of the pin body 9 and the pin neck 10 is circular, and the pin neck 10 is the thinnest part of the entire pin. The cross section of the pin head 11 is oblate, which is slightly smaller than the oblate hole in the fast mounting pin hole, and the fast mounting pin hole is designed to have an interference fit with the fast mounting pin, so that the fast mounting pin can be easily inserted into and pulled out of the fast mounting pin hole.

[0044] As Figure 4 shown, the fast mounting pin hole is a hole drilled in the insulating flat plate, which is composed of a small hole on the upper side and a large hole 14 on the lower side, and an intermediate oblate iron washer 15. The diameter of the small hole is slightly larger than the diameter of the pin body, and the oblate hole of the iron washer is also slightly larger than the pin head. The diameter of the large hole is greater than or equal to the outer diameter of the iron washer. The iron washer is fixed in the large hole and does not move. When the oblate direction of the pin head of the fast mounting pin is the same as the oblate direction of the iron washer, the fast mounting pin can be freely inserted or pulled out, and the pin head 11 can pass through the iron washer. After the pin head 11 passes through the iron washer 15, only a slight rotation of the fast mounting pin is needed to make the oblate direction of the pin head 11 different from the oblate direction of the iron washer 14, so that the iron washer 15 is clamped on the pin neck 10, preventing the pin head from being pulled out. After the hand is released, the spring 8 on the fast mounting pin can press the circuit board 4 tightly against the test bench surface 6. In order to prevent the circuit board 12 from being deformed excessively under pressure and affecting the optical parameters, a support 13 is designed at the edge of the fast mounting pin hole on the test bench surface 6, so that the circuit board 4 is safely and stably clamped between the spring 8 on the fast mounting pin and the support 13 of the test bench surface.

[0045] The test personnel use multiple fast mounting pins to fix the circuit board 3 to the test bench surface 6, which can accurately ensure that the circuit board 12 is horizontal, and the position of the optical train is accurately fixed and does not move. The control error of the optical axis direction in the vertical direction is within 5 degrees, and remains unchanged during the test period.

[0046] In the embodiment, the target screen 1 is directly opposite the camera lens on the circuit board 3. The middle of the target screen 1 is a black square 2, which is made of a porous black foam sheet in the shape of a square, which can ensure that the black square 2 has a very obvious contrast with the background in both the visible spectrum and the infrared spectrum. The other parts of the target screen 1 can have some lines and patterns for observing image clarity when manually performing lens focusing work.

[0047] In this embodiment, the test fixture needs to perform center alignment on the test table 6 and the target screen 1 before use, specifically, a plumb line is stuck at the center of the black square 2 of the target screen 1, the dual-optical domain camera circuit board is fixed on the test table 6, and the position of the test table and the target screen 1 is adjusted so that the plumb line extension line passes through the midpoint of the line connecting the two camera lenses; at the same time, a pair of edges of the black square 2 is parallel to the line connecting the two camera lenses; after alignment, the test table 6 and the target screen 1 are locked in the fixed position and no longer moved, and the plumb line is removed, and then the fast plug pin is used to fix each set of dual-optical domain camera circuit boards to be tested, after power on, the focusing of the optical group is adjusted manually, and the automatic correction operation is realized.

[0048] In this embodiment, the dual-optical domain camera after correction of the axis angle deviation can perform intelligent recognition on actual objects, especially on biological objects, in the visible light and infrared wave bands at the same time, and can effectively prevent illegal intrusion by images, screens, and bionic masks, and significantly improve the security of the intelligent recognition system.

[0049] In actual use, the correction parameter data stored in the memory ROM is read-only data; the processor calculates the images captured by the two cameras using the correction parameters, corrects the axis angle deviation, and can realize coordinate mapping of any pixel between the infrared and color images, and superimpose two images of the same scene.

[0050] In the security authentication application scenario, the recognition algorithm of the processor not only requires that the infrared and color images pass the graphic recognition at the same time, but also limits that the coordinates of the objects captured by the two cameras must be superimposed; that is, the dual-optical domain camera described in the application not only recognizes the material of the scene, but also verifies whether the two cameras are the same scene, which is much safer than the traditional image recognition system; because it is difficult for hackers to simultaneously operate two different illegal intrusion tools to make the coordinates of the objects captured by the two cameras superimposed together, so it can effectively prevent double-object attacks and has strong security anti-counterfeiting capability; through the external interface IO, the result of image recognition or control of the dual-optical domain camera can be output.

[0051] Specific implementation method two, the correction method of the dual-optical domain camera axis angle deviation correction system described in specific implementation method one, the method is realized by the following steps:

[0052] In this embodiment, the axis angle deviation is corrected by a digital calculation method, so that the contents of the two images captured by the first camera 4 and the second camera 5 can be superimposed.

[0053] After the tester puts the circuit board 2 on the test bench 6 and fixes it with the quick-mounting pins, the contact pin spring 18 is coupled with the connector 19 on the circuit board 2; after power-on, the image is clearest after manual mechanical adjustment of the focusing group of the optical device, and then the infrared camera and the color camera are used to take images on the target screen 1, and the processor automatically identifies the black square 2 in the images and calculates a set of correction parameters and saves them in the memory.

[0054] During the correction process, the infrared camera needs to turn on the fill light for shooting, because the infrared fill light will cause the brightness of different areas of the image to be uneven, there is a large area of brightness gradient area, which will cause a large area of black area to be generated in the binarization of the image, so it will affect the processor to automatically identify the black square 2 in the center of the target screen; similar situations also occur in the images taken by the color camera, but not as serious as the infrared image; in order to solve this problem, it is necessary to analyze the boundary pixel value of the image binarization to determine whether each boundary pixel is located at the brightness gradient boundary position; the specific identification process steps are as follows:

[0055] Step A1, calculate each pixel in the taken image as a brightness pixel and save it to an image B;

[0056] Step A2, calculate the average value C of all pixels in the image B as the threshold C of the first binarization;

[0057] Step A3, binarize the image B with the threshold C to obtain an image D containing only bright spots and dark spots; the pixel value in the image B greater than the threshold C is set to a bright spot with a value equal to 1 in the image D, otherwise it is set to a dark spot with a value of 0;

[0058] Step A4, initialize an image E and set all pixel values to 0;

[0059] Step A5, analyze each pixel in the image D from left to right and then from top to bottom, specifically: when a pixel in the image D is taken, first save the coordinate value of the pixel to a variable P, and then analyze its value, if the value is 0, then execute step A6, otherwise take the eight adjacent pixels of the upper left corner, the upper side, the upper right corner, the right side, the lower right corner, the lower side, the lower left corner, and the left side, if there is a bright spot with a pixel value equal to 1 in the eight adjacent pixels, then set the pixel value of the image E with the same coordinate as the variable P to 1; execute step A6.

[0060] Step A6, loop step A5 until all pixels in the image D are analyzed and the image E is updated at the same time;

[0061] Step A7, define two variables G and H with initial value equal to zero; set a pixel gradual change difference threshold equal to three percent of the maximum brightness of the image, then scan the image B and image E simultaneously, in the direction of traversing pixels from left to right in each row and from top to bottom in each row; if a pixel with value 1 is found in image E, save the coordinates of the pixel into variable P, then get the pixel with the same coordinates in image B, compare it with threshold C, if the difference is greater than the pixel gradual change difference threshold, add the brightness value of the pixel to variable G and increase variable H by 1, otherwise, consider that the pixel is located at the position of the brightness gradual change boundary and does not need to be processed; perform the above search until all coordinates in image E are analyzed;

[0062] Step A8, divide the value of variable G by variable H to obtain a second binary threshold J; perform binary processing on image B with threshold J to obtain image K containing only bright spots and dark spots;

[0063] Step A9, define a rectangle T, set the horizontal coordinate of the upper left corner to one quarter of the width of image K, the vertical coordinate of the upper left corner to one quarter of the height of image K, the horizontal coordinate of the lower right corner to three quarters of the width of image K, and the vertical coordinate of the lower right corner to three quarters of the height of image K; the black square in the center of the target screen must be completely located within the rectangular region in image K, if it is outside the region, it is considered that the mechanical error of the camera is too large and unqualified, and cannot be corrected; the subsequent steps perform operations on image K within the rectangular region T, and the pixels outside the rectangular region T are not processed;

[0064] Step A10, perform erosion on K first, and then perform dilation, which can remove small noise points that may exist in image K within the rectangular region T;

[0065] Step A11, define a variable R of a rectangular data structure type, which has two sub-variables, one describing the upper left corner coordinates of the rectangle, and the other describing the lower right corner coordinates of the rectangle; clear the value of variable H; the initial upper left corner coordinates of R are equal to the lower right corner coordinates of rectangular region T, and the initial lower right corner coordinates of R are equal to the upper left corner coordinates of rectangular region T;

[0066] Step A12, perform dilation on image K; when dilation is performed, analyze the relationship between the coordinates of the dark spots encountered and the variable R rectangle, and set variable R according to the situation, specifically;

[0067] If the horizontal coordinates of the dark spots are found to be less than the horizontal coordinates of the upper left corner of the variable R rectangle, set the horizontal coordinates of the upper left corner of the variable R rectangle to the horizontal coordinates of the dark spots;

[0068] If the horizontal coordinate of the dark spot is found to be greater than the horizontal coordinate of the lower right corner of the variable R rectangle, then the horizontal coordinate of the lower right corner of the variable R rectangle is set to be the horizontal coordinate of the dark spot;

[0069] If the vertical coordinate of the dark spot is found to be less than the vertical coordinate of the upper left corner of the variable R rectangle, then the vertical coordinate of the upper left corner of the variable R rectangle is set to be the vertical coordinate of the dark spot;

[0070] If the vertical coordinate of the dark spot is found to be greater than the horizontal coordinate of the lower right corner of the variable R rectangle, then the vertical coordinate of the lower right corner of the variable R rectangle is set to be the vertical coordinate of the dark spot;

[0071] Step A13, after the image K is dilated in step A12, if the width and height of the variable R rectangle are both greater than 1 pixel, then the value of the variable H is added by 1, and then step A11 is executed to dilate the image K again while analyzing the variable R rectangle; otherwise, step A14 is executed;

[0072] Step A14, the average of the horizontal coordinates of the upper left corner and the lower right corner of the variable R rectangle is calculated as the horizontal coordinate of the center of the black square in the identified image, and the average of the vertical coordinates of the upper left corner and the lower right corner of the variable R rectangle is calculated as the vertical coordinate of the center of the black square in the identified image; then the value of the variable H is multiplied by 2, and if the width or height of the variable R rectangle is equal to 1 pixel, then the variable H is added by 1; at this time, the value of the variable H is equal to the side length of the black square identified in the photographed image.

[0073] Two target screen images are photographed by an infrared camera and a color camera respectively, and then steps A1 to A14 are executed on the infrared and color images respectively; the infrared side length L IR and the infrared center coordinates (OX IR , OY IR ) of the black square are automatically identified, and the color side length L RGB and the color center coordinates (OX RGB , OY RGB ) are also automatically identified; the infrared magnification factor Z IR and the color magnification factor Z RGB are saved as correction parameters in the memory ROM of the dual-optical-field camera;

[0074] Z IR = 1024 * L IR / L RGB

[0075] Z RGB = 1024 * L RGB / L IR

[0076] In the actual use, the dual-optical domain camera can perform arbitrary pixel mapping between the images captured by the two cameras by the processor; in order to reduce the cost of the processor and improve the performance of the system, the application does not use floating point operation, but adopts integer calculation, and avoids division operation; the coordinate mapping can also support the case that the resolution of the infrared camera is different from that of the color camera;

[0077] (X IR ,Y IR ) represents a pixel coordinate in the coordinate system of the infrared camera; (X RGB ,Y RGB ) represents a pixel coordinate in the coordinate system of the color camera;

[0078] The calculation method of mapping the infrared coordinate (X IR ,Y IR ) to the color camera coordinate (X RGB ,Y RGB ) is as follows:

[0079] X RGB =((X IR -OX IR )*Z IR )>>10+OX RGB

[0080] Y RGB =((Y IR -OY IR )*Z IR )>>10+OY RGB

[0081] The calculation method of mapping the color coordinate (X RGB ,Y RGB ) to the infrared camera coordinate (X IR ,Y IR ) is as follows:

[0082] X IR =((X RGB -OX RGB )*Z RGB )>>10+OX IR

[0083] Y IR =((Y RGB -OY RGB )*Z RGB )>>10+OY IR

[0084] The processor respectively identifies the scene under the infrared spectrum image and the color spectrum of visible light, if the identification is successful, then according to some characteristic points existing in the infrared spectrum and the color spectrum, respectively calculates the infrared and color two kinds of coordinates of the characteristic points; after that, according to a preset error threshold, calculates whether the two kinds of characteristic point coordinates are located within the error threshold after mutual mapping; if all the errors are less than the threshold, then it can be determined that the same scene is identified under the condition of dual light domain, otherwise even if the scene feature identification is successful, it is not the same scene that can be superimposed in space, this situation may be that the hacker is launching a dual object attack.

[0085] The technical features of the above-described embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described, but it should be considered that any combination of the technical features is within the scope of the present disclosure as long as there is no contradiction.

[0086] The above-described embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. The axial angle deviation correction system for a dual-light range camera is characterized by: The system comprises a test fixture and a dual-optical-field camera structure arranged in the test fixture; The test fixture adopts a dark box, a target screen is fixed on the inner top of the dark box, and a mark pattern is arranged at the center of the target screen; A test table is fixed on the inner bottom of the dark box, and two cameras in the dual-optical-field camera structure are fixed on the test table; The two cameras shoot the target screen to obtain two images, a processor is used to identify the mark pattern at the center of the images, and the magnification ratio of the two cameras is automatically calculated according to the size of the mark pattern, and the center coordinates of the mark pattern are automatically identified; The processor calculates correction parameters for correcting the axial angle deviation of the two cameras, and saves the correction parameters in the memory of the dual-optical-field camera structure; and the correction parameters are used to correct the axial angle deviation of the dual-optical-field camera, so that the two cameras shoot two images of the same scene and the two images coincide.

2. The axial angle misregistration correction system for dual optical band cameras of claim 1, wherein: An external interface IO is further arranged, and the external interface IO is used to output the result of image recognition or control the dual-optical-field camera.

3. The axial angle misregistration correction system for dual-optical- domain cameras of claim 1, wherein: Before use, the test table and the target screen need to be centrally registered, and the specific process is as follows: A plumb line is adhered to the center of the mark pattern of the target screen, the dual-optical-field camera circuit board is fixed on the test table, and the positions of the test table and the target screen are adjusted, so that the plumb line extension line passes through the midpoint of the connecting line of the two camera lenses; meanwhile, a pair of opposite sides of the mark pattern are parallel to the connecting line of the two camera lenses; the central registration of the test table and the target screen is completed, and after the registration, the test table and the target screen are locked in the fixed position, and the plumb line is removed.

4. The axial angle misregistration correction system for dual-optical- domain cameras of claim 1, wherein: The test table is a plane, which is made of an insulating plate with a thickness greater than 1 cm, and the insulating plate is provided with quick mounting pin holes corresponding to the fixed hole positions on the camera circuit board to be tested; and the insulating plate is provided with a top pin spring piece electrically connected with the external interface IO connector on the circuit board, and the top pin spring piece is connected with an electric wire which is led out from a wiring groove under the insulating plate to the power supply and the test computer outside the test fixture.

5. The axial angle misregistration correction system for dual optical band cameras of claim 4, wherein: The quick mounting plug is composed of a plug cap, a spring, a plug body, a plug neck and a plug head, the cross section of the plug body and the plug neck is circular, and the plug neck is the thinnest part of the plug; the cross section of the plug head is oblate and smaller than the oblate hole in the quick mounting pin hole, and the quick mounting plug and the quick mounting pin hole are in interference fit, so that the quick mounting plug can be easily inserted into and pulled out of the quick mounting pin hole.

6. The axial angle misregistration correction system for dual optical band cameras of claim 4, wherein: The quick mounting pin hole is a hole drilled in the insulating plate, which is composed of a small hole on the upper side, a large hole on the lower side and a middle iron pad; the diameter of the small hole is greater than the diameter of the plug body, the oblate hole of the iron pad is greater than the diameter of the plug head, and the diameter of the large hole is greater than or equal to the outer diameter of the iron pad; the iron pad is fixed in the large hole and does not move, when the oblate hole of the plug head of the quick mounting plug is the same as the oblate hole of the iron pad, the quick mounting plug can be freely inserted or pulled out up and down, and the plug head can freely pass through the iron pad; after the plug head passes through the iron pad, only the quick mounting plug needs to be rotated, so that the oblate hole of the plug head is different from the oblate hole of the iron pad, the iron pad is clamped on the plug neck to prevent the plug head from being pulled out, and after the hand is released, the spring on the quick mounting plug can press the circuit board tightly on the test table.

7. The axial angle misregistration correction system for dual optical band cameras of claim 6, wherein: The circuit board is fixed to the test table by a plurality of quick-mounting pins to ensure that the circuit board is horizontal and the position of the optical train is accurately fixed and does not change during the test.

8. A method for correcting an axis angle deviation of a dual optical field camera, characterized by: The method is implemented by the axial angle deviation correction system of the dual-optical-field camera according to any one of claims 1-8, and the specific steps of the method are as follows: Step 1. Calculate each pixel in the captured image as a brightness pixel, save it to image B, and calculate the average value C of all pixels in image B as the threshold value C of the first binaryzation; Step 2. Binaryzate image B using threshold value C to obtain image D including bright spots and dark spots; set the pixels with a value greater than threshold value C in image B to bright spots with a value equal to 1 in image D, and set the other pixels to dark spots with a value of 0; Step 3. Initialize an image E and set all pixel values in image E to 0; Step 4. Analyze each pixel in image D from left to right and then from top to bottom, specifically, when a pixel in image D is obtained, first save its coordinate value to variable P, then analyze its value, if the value is 0, execute step 5, otherwise, obtain eight adjacent pixels including the upper left corner, the upper side, the upper right corner, the right side, the lower right corner, the lower side, the lower left corner and the left side of the pixel respectively, if there is a bright spot with a pixel value equal to 1 in the eight adjacent pixels, set the pixel value in image E with the same coordinate as variable P to 1; Step 5. Loop step 4 until all pixels in image D are analyzed and image E is updated; Step 6. Define two variables G and H with initial values both equal to 0; Set the pixel gradual change difference threshold to 3% of the maximum brightness of the image, then scan and analyze each pixel in image B and image E from left to right in each row and from top to bottom in each row; if a pixel with a value of 1 is scanned in image E, first save its coordinates to variable P, then obtain the pixel with the same coordinates as variable P from image B, compare it with threshold value C, if the difference is greater than the pixel gradual change difference threshold, add the brightness value of the pixel to variable G and increase variable H by 1, otherwise, consider that the pixel is located at the brightness gradual change boundary position and does not need to be processed; execute the above search until all coordinates in image E are analyzed; Step 7. Divide the value of variable G by variable H to obtain the second binaryzation threshold J; binaryzate image B in image B using threshold J to obtain image K including bright spots and dark spots; Step 8. Define a rectangle T, the upper left corner horizontal coordinate is one fourth of the width of image K, the upper left corner vertical coordinate is one fourth of the height of image K, the lower right corner horizontal coordinate is three fourths of the width of image K, and the lower right corner vertical coordinate is three fourths of the height of image K; it is required that the mark pattern at the center of the target screen must be completely located within the area of rectangle T in image K; Step 9. First, perform erosion on image K, then perform dilation to remove possible small noise points in image K within rectangle T. Step 10. Define R rectangle, that is, define a variable R for describing the type of rectangle data structure, the structure has two sub-variables, one for describing the upper left corner coordinate of the rectangle, and the other for describing the right lower corner coordinate of the rectangle; clear the value of variable H; the initial upper left corner coordinate of R is equal to the right lower corner coordinate of rectangle T, and the initial right lower corner coordinate of R is equal to the upper left corner coordinate of rectangle T; Step 11. Perform the dilation operation on image K; when dilating, analyze the relationship between the encountered dark point coordinates and the variable R rectangle, and set the R rectangle according to the situation, specifically: If the horizontal coordinate of the dark point is found to be less than the upper left corner horizontal coordinate of the R rectangle, set the upper left corner horizontal coordinate of the variable R rectangle to this dark point horizontal coordinate; If the horizontal coordinate of the dark point is found to be greater than the right lower corner horizontal coordinate of the R rectangle, set the right lower corner horizontal coordinate of the variable R rectangle to this dark point horizontal coordinate; If the vertical coordinate of the dark point is found to be less than the upper left corner vertical coordinate of the R rectangle, set the upper left corner vertical coordinate of the variable R rectangle to this dark point vertical coordinate; If the vertical coordinate of the dark point is found to be greater than the right lower corner horizontal coordinate of the R rectangle, set the right lower corner vertical coordinate of the variable R rectangle to this dark point vertical coordinate; Step 12. After the dilation of image K is completed, if the width and height of the R rectangle are both greater than 1 pixel, then the value of variable H is added once, and then step 11 is executed, otherwise step 13 is executed; Step 13. Calculate the average of the left upper corner and right lower corner horizontal coordinates of the R rectangle as the horizontal coordinate of the center of the black square in the recognized image, and calculate the average of the left upper corner and right lower corner vertical coordinates of the R rectangle as the vertical coordinate of the center of the black square in the recognized image; then multiply the value of variable H by 2, and if the width or height of the R rectangle is equal to 1 pixel, add 1 to the variable H; at this time, the value of H is equal to the edge length of the black square identified in the captured image.

9. The method of claim 8, wherein: The processor performs arbitrary pixel mapping between the images captured by the two cameras; under the infrared spectrum image and the visible color spectrum, the scene is identified respectively, if the identification is successful, according to the feature points that exist simultaneously in the infrared spectrum and the color spectrum, the infrared and color two types of coordinates of the feature points are calculated respectively; according to a preset error threshold, whether the two types of feature point coordinates are located within the error threshold after mutual mapping is calculated; If all errors are less than the threshold, it is determined that the same scene is identified under the dual light domain condition.

10. The method of claim 9, wherein: (X IR ,Y IR ) denotes a pixel coordinate in the coordinate system of the infrared camera; (X RGB ,Y RGB ) denotes a pixel coordinate in the coordinate system of the color camera; The infrared coordinates (X IR ,Y IR ) are mapped to color camera coordinates (X RGB ,Y RGB ) in the following way: X RGB = ((X IR -OX IR )*Z IR )>>10+OX RGB Y RGB = ((Y IR - OY IR ) * Z IR ) » 10 + OY RGB The color coordinates (X RGB ,Y RGB ) are mapped to the infrared camera coordinates (X IR ,Y IR ) in the following way: X IR =((X RGB -OX RGB )*Z RGB )>>10+OX IR Y IR = ((Y RGB - OY RGB )* Z RGB >> 10 + OY IR wherein (OX IR ,OY IR ) are the infrared center coordinates, i.e. the black square center coordinates; and (OX RGB ,OY RGB ) are the color center coordinates.

Images