Camera system control method and camera system

Abstract

The invention provides a control method of a camera system and the camera system. The camera system comprises a controller, a TOF camera and an AB frame camera. The control method comprises the following steps: respectively sending frame synchronization trigger signals to the control module of the TOF camera and the control module of the AB frame camera; sending a first infrared control signal to an infrared emission device of the TOF camera; sending a second infrared control signal to an infrared emission device of the AB frame camera; and starting from the (N + 1) th effective signal of the frame synchronization trigger signal, synchronously outputting an image. Through the frame synchronization trigger signal, the camera processes the frame synchronization trigger signal, and the A frame and the B frame in the TOF camera and the AB frame camera are controlled to synchronously output images according to the same frame rate; the infrared emission devices of the TOF camera and the AB frame camera are controlled not to interfere with each other through the first infrared control signal and the second infrared control signal, so that the TOF camera and the AB frame camera can stably and smoothly output images without interference.

Description

Technical Field

[0001] The present invention relates to the field of image technology, and particularly to a control method for a camera system and a camera system. Background Art

[0002] With the development of automotive technology, more and more functions are configured in the cockpit of a vehicle. For example, functional modules such as a face recognition device and a gesture recognition device are configured in the cockpit, and these functional modules need to rely on a camera to implement corresponding functions. Currently, the cameras used in the cockpit are mainly TOF (Time of flight) cameras or AB frame cameras (i.e., dual-frame acquisition cameras).

[0003] A TOF camera, also known as a full-resolution depth camera, is widely used in mobile phone face recognition; for some recognition tasks in a vehicle cockpit, such cameras are gradually used by each vehicle manufacturer; the core components of a TOF camera include an infrared irradiation unit, an imaging sensor, and a calculation unit, and these components are respectively used for infrared light emission, generating a depth image, and generating an accurate depth map.

[0004] An AB frame camera can output RGB (Red, Green, Blue) color images and black-and-white images, and alternately output color images and black-and-white images in the form of ABAB; when the AB frame camera outputs a B frame, i.e., a black-and-white frame, it is necessary to turn off the infrared emission device to ensure that the next A frame, i.e., a color frame, is not interfered by infrared light; when outputting an A frame, it is necessary to turn on the infrared emission device, because after the photosensitive element CMOS (Complementary Metal Oxide Semiconductor) completes exposure in the current frame, the next frame will output an image. A traditional camera can only output RGB color images or black-and-white images; an AB frame camera can output color and black-and-white images in the front and back frames; based on actual requirements, the main control end can differentially obtain A and B frame images.

[0005] To prevent the infrared emission device of the TOF camera from interfering with the A frame of the AB frame camera, the TOF camera and the AB frame camera are usually used separately. As Figure 1 shown, if the TOF camera and the AB frame camera are used together, the TOF camera may turn on the infrared emission device when the AB frame camera outputs a B frame, which causes the A frame output by the AB frame camera to be interfered by the infrared rays emitted by the infrared emission device of the TOF camera, and the A frame interfered by infrared rays cannot display a color image normally. Summary of the Invention

[0006] The present invention provides a control method for a camera system and a camera system to solve the technical problem that the TOF camera and the AB frame camera cannot be used together.

[0007] To solve the above technical problems, the present invention provides a control method for a camera system, the camera system includes a controller, a TOF camera, and an AB-frame camera, and both the TOF camera and the AB-frame camera include a control module and an infrared emission device; the control method includes the following steps:

[0008] Send frame synchronization trigger signals to the control module of the TOF camera and the control module of the AB-frame camera respectively, and the frame synchronization trigger signal is a square wave signal with a preset frequency;

[0009] Send a first infrared control signal to the infrared emission device of the TOF camera, and the first infrared control signal is used to control the infrared emission device of the TOF camera to turn on only when the AB-frame camera outputs an A-frame;

[0010] Send a second infrared control signal to the infrared emission device of the AB-frame camera, and the second infrared control signal is used to control the infrared emission device of the AB-frame camera to turn on only when the AB-frame camera outputs an A-frame;

[0011] Starting from the (N + 1)-th valid signal of the frame synchronization trigger signal, the TOF camera and the AB-frame camera synchronously output images, where N is a positive integer greater than or equal to 1.

[0012] Preferably, the step of sending frame synchronization trigger signals to the control module of the TOF camera and the control module of the AB-frame camera respectively specifically includes the following steps: At the same moment, send frame synchronization trigger signals to the control module of the TOF camera and the control module of the AB-frame camera respectively.

[0013] Preferably, the frequency of the frame synchronization trigger signal is 60 Hz - 80 Hz.

[0014] Preferably, the first infrared control signal and the second infrared control signal have the same frequency.

[0015] Preferably, the infrared emission devices of the TOF camera and the AB-frame camera have the same on-time each time.

[0016] The present invention also provides a camera system, including a controller, a TOF camera, and an AB-frame camera, the TOF camera and the AB-frame camera are respectively connected to the controller, and the controller, the control module of the TOF camera, and the control module of the AB-frame camera use the control method of a camera system described in any one of the above to control the TOF camera and the AB-frame camera to output images.

[0017] A control method and a camera system provided by the present invention control the TOF camera, the A frame and the B frame in the AB frame camera to output images synchronously at the same frame rate through a frame synchronization trigger signal, the TOF camera, and the processing of the frame synchronization trigger signal by the AB frame camera; the infrared emission devices of the TOF camera and the AB frame camera are controlled by a first infrared control signal and a second infrared control signal so as not to interfere with each other, so that the TOF camera and the AB frame camera can output images stably, without interference, and smoothly. Since the depth image output by the TOF camera includes depth information such as the relative or absolute distance of the object surface from the observation point, and the images output by the AB frame camera include basic information such as pixels, resolution, size, and color, after fusing the depth information and the basic information of the images, functions based on images such as face recognition and gesture recognition can be realized more accurately. Description of the Drawings

[0018] Figure 1 It is the A-frame image output by the AB frame camera in the prior art when it is interfered by the infrared rays emitted by the infrared emission device of the TOF camera.

[0019] Figure 2 It is the normal A-frame image output by the AB frame camera in a camera system provided by an embodiment of the present invention.

[0020] Figure 3 It is a schematic structural diagram of a camera system provided by an embodiment of the present invention.

[0021] Figure 4 It is a timing diagram between the first infrared control signal and the output of the TOF camera, and a timing diagram between the second infrared control signal and the output of the AB frame camera provided by an embodiment of the present invention.

[0022] Figure 5 It is a timing diagram between the frame synchronization trigger signal, the first infrared control signal, the output of the TOF camera, and the output of the AB frame camera provided by an embodiment of the present invention.

[0023] Figure 6 A flowchart of a control method for a camera system provided by an embodiment of the present invention. Detailed Embodiments

[0024] To make the objectives, advantages, and features of the present invention clearer, the following further describes in detail a control method and a camera system of the present invention with reference to the accompanying drawings. It should be noted that the drawings are all in a very simplified form and use non-precise scales, and are only used to facilitate and clearly assist in explaining the objectives of the embodiments of the present invention.

[0025] In the description of the present invention, the qualifiers such as "first", "second", etc. are added for convenience of description and reference, and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with qualifiers such as "first", "second", etc. may explicitly or implicitly include one or more of such features.

[0026] As Figure 3 and Figure 6 shown, this embodiment provides a control method for a camera system. The camera system includes a controller, a TOF camera, and an AB-frame camera. Both the TOF camera and the AB-frame camera include a control module and an infrared emission device. The control method includes the following steps:

[0027] S1. The controller respectively sends frame synchronization trigger signals to the control module of the TOF camera and the control module of the AB-frame camera. The frame synchronization trigger signal is a square wave signal with a preset frequency; Refer Figure 3 shown, the control method for the camera system provided by this solution can be applied in an automotive cockpit. The controller included in the camera system can be the controller configured in the automotive cockpit; The controller can use the frame synchronization trigger signal to make the TOF camera and the AB-frame camera output images synchronously according to the preset frequency; The frequency of the frame synchronization trigger signal can be 60Hz. The control module of the TOF camera can divide 60Hz by two to 30Hz. The A-frame and B-frame in the AB-frame camera are respectively 30Hz. In this way, the TOF camera and the AB-frame camera can synchronously output images with the same frame rate based on the same frame synchronization trigger signal, preventing color abnormalities of the A-frame in the AB-frame camera due to frame rate deviation.

[0028] S2. The control module of the TOF camera sends a first infrared control signal to the infrared emission device of the TOF camera. The first infrared control signal is used to control the infrared emission device of the TOF camera to turn on only when the AB-frame camera outputs the A-frame; As Figure 3 and Figure 4 shown, after the control module of the TOF camera sends a first infrared control signal to the TOF camera, it controls the infrared emission device of the TOF camera to turn on only when the AB-frame camera outputs the A-frame through the first infrared control signal. In this way, the TOF camera can normally output depth images and does not affect the color of the A-frame image output by the AB-frame camera.

[0029] S3. The control module of the AB-frame camera sends a second infrared control signal to the infrared emission device of the AB-frame camera. The second infrared control signal is used to control the infrared emission device of the AB-frame camera to turn on only when the AB-frame camera outputs the A-frame; As Figure 3 and Figure 4As shown, after the control module of the AB-frame camera sends the second infrared control signal to the AB-frame camera, the infrared emission device of the AB-frame camera is controlled by the second infrared control signal to turn on only when the AB-frame camera outputs an A-frame. This enables the AB-frame camera to normally output A-frame images and B-frame images. Considering interference issues and the characteristics of the CMOS photosensitive element, when the AB-frame camera outputs an A-frame, the TOF camera uses the infrared emission device; otherwise, it will cause color abnormalities in the A-frame. The frame rate of the AB-frame camera is preferably 60 frames per second, and for its infrared emission device, the maximum usage time is preferably 16.66667 ms. Within the performance range that the TOF camera meets, the frame rate of the TOF camera is preferably 30 frames per second, and for the infrared emission device of the TOF camera, the restricted usage time is preferably 16.66667 ms. Moreover, within the color output frame of the AB-frame camera, the infrared rays of the TOF camera are prevented from interfering with the color frame of the AB-frame camera.

[0030] S4. Starting from the (N + 1)-th valid signal of the frame synchronization trigger signal, the TOF camera and the AB-frame camera synchronously output images, where N is a positive integer greater than or equal to 1. As Figure 5 shown, for a square wave signal, the high level is usually defined as valid and the low level as invalid. In this solution, considering that the time taken for different cameras to output the first frame of images after power-on may not be the same, the camera system needs to be initialized and synchronized. Specifically, after the camera system is powered on, the camera system remains without outputting video signals. Using the first valid signal of the frame synchronization trigger signal as a reference, assuming the N-th frame as the reference; starting from the (N + 1)-th valid signal of the frame synchronization trigger signal, that is, starting from the (N + 1)-th frame, the TOF camera and the AB-frame camera synchronously output images and send them to the controller. Specifically, at the (N + 1)-th frame, the AB-frame camera outputs a color A-frame image and the AB-frame camera turns on the infrared emission device for fill light, and the TOF camera turns on the infrared emission device to obtain information and outputs a depth image. At this time, the AB-frame camera outputs a color A-frame image as Figure 2 shown, the A-frame image can normally display a color image; at the (N + 2)-th frame, the infrared emission devices of both the AB-frame camera and the TOF camera are turned off, the AB-frame camera outputs a B-frame, that is, a black-and-white image, and the TOF camera outputs a depth image.

[0031] A control method for a camera system provided in this embodiment controls the TOF camera, the A frame and the B frame in the AB frame camera to synchronously output images at the same frame rate through a frame synchronization trigger signal and the processing of the frame synchronization trigger signal by the TOF camera and the AB frame camera; the first infrared control signal and the second infrared control signal are used to control the infrared emission devices of the TOF camera and the AB frame camera so as not to interfere with each other, so that the TOF camera and the AB frame camera can stably, interference-free and smoothly output images. Since the depth image output by the TOF camera includes depth information such as the relative or absolute distance of the object surface from the observation point, and the images output by the AB frame camera include basic information such as pixels, resolution, size and color, after fusing the depth information and the basic information of the images, functions based on images such as face recognition and gesture recognition can be realized more accurately.

[0032] Preferably, step S1, that is, the step of respectively sending a frame synchronization trigger signal to the control module of the TOF camera and the control module of the AB frame camera, specifically includes the following steps: at the same moment, respectively send a frame synchronization trigger signal to the control module of the TOF camera and the control module of the AB frame camera. Sending the frame synchronization trigger signal to the control module of the TOF camera and the control module of the AB frame camera at the same moment can better control the TOF camera and the AB frame camera to synchronously output images, and prevent the frame synchronization trigger signals received by the TOF camera and the AB frame camera from being out of sync, resulting in interference and smoothness problems in the images output by the TOF camera and the AB frame camera.

[0033] Preferably, the frequency of the frame synchronization trigger signal is 60 Hz - 80 Hz. The frequency of the frame synchronization trigger signal needs to be set within a suitable range. If it is too small, the video signal will not be smooth; if it is too large, the camera will not be able to output normally; when the frequency of the frame synchronization trigger signal is 60 Hz - 80 Hz, the video signal can be clear and smooth.

[0034] Preferably, as Figure 4 shown, the frequencies of the first infrared control signal and the second infrared control signal are the same, which can simplify the control method of the camera system.

[0035] Preferably, the duration of each turn-on of the infrared emission devices of the TOF camera and the AB frame camera is the same, which can simplify the control method of the camera system. In other embodiments, the duration of each turn-on of the infrared emission devices of the TOF camera and the AB frame camera only needs to be less than the time of one frame, for example, 1 / 60 second - 1 / 80 second.

[0036] As Figure 3As shown, based on the same technical concept as the control method of the above-mentioned camera system, this embodiment provides a camera system, including a controller, a TOF camera, and an AB-frame camera. The TOF camera and the AB-frame camera are respectively connected to the controller. The controller, the control module of the TOF camera, and the control module of the AB-frame camera use the control method of the above-mentioned camera system to control the TOF camera and the AB-frame camera to output images.

[0037] In the camera system provided in this embodiment, through the frame synchronization trigger signal and the processing of the frame synchronization trigger signal by the TOF camera and the AB-frame camera, it is controlled that the A-frame and B-frame in the TOF camera and the AB-frame camera output images synchronously at the same frame rate; through the first infrared control signal and the second infrared control signal, it is controlled that the infrared emission devices of the TOF camera and the AB-frame camera do not interfere with each other, so that the TOF camera and the AB-frame camera can output images stably, without interference, and smoothly. Since the depth image output by the TOF camera includes depth information such as the relative or absolute distance of the object surface from the observation point, and the image output by the AB-frame camera includes basic information such as pixels, resolution, size, and color, after fusing the depth information and the basic information of the images, functions based on images such as face recognition and gesture recognition can be realized more accurately.

[0038] In summary, for the control method and camera system provided by the present invention, through the frame synchronization trigger signal and the processing of the frame synchronization trigger signal by the TOF camera and the AB-frame camera, it is controlled that the A-frame and B-frame in the TOF camera and the AB-frame camera output images synchronously at the same frame rate; through the first infrared control signal and the second infrared control signal, it is controlled that the infrared emission devices of the TOF camera and the AB-frame camera do not interfere with each other, so that the TOF camera and the AB-frame camera can output images stably, without interference, and smoothly. Since the depth image output by the TOF camera includes depth information such as the relative or absolute distance of the object surface from the observation point, and the image output by the AB-frame camera includes basic information such as pixels, resolution, size, and color, after fusing the depth information and the basic information of the images, functions based on images such as face recognition and gesture recognition can be realized more accurately.

[0039] The above description is only a description of the preferred embodiments of the present invention, and does not limit the scope of the present invention in any way. Any changes and modifications made by those of ordinary skill in the art according to the above disclosure belong to the protection scope of the present invention.

Claims

1. A method for controlling a camera system, characterized in that: The camera system includes a controller, a TOF camera and an AB frame camera, wherein the TOF camera and the AB frame camera both include a control module and an infrared emitting device; the control method includes the following steps: Sending a frame synchronization trigger signal to the control module of the TOF camera and the control module of the AB frame camera respectively, wherein the frame synchronization trigger signal is a square wave signal of a preset frequency; Sending a first infrared control signal to the infrared emitting device of the TOF camera, wherein the first infrared control signal is used to control the infrared emitting device of the TOF camera to be turned on only when the AB frame camera outputs the A frame; Sending a second infrared control signal to the infrared emitting device of the AB frame camera, wherein the second infrared control signal is used to control the infrared emitting device of the AB frame camera to be turned on only when the AB frame camera outputs an A frame; Starting from the N+1th valid signal of the frame synchronization trigger signal, the TOF camera and the AB frame camera synchronously output images, where N is a positive integer greater than or equal to 1.

2. A camera system control method as claimed in claim 1, characterized in that: The step of sending a frame synchronization trigger signal to the control module of the TOF camera and the control module of the AB frame camera respectively comprises the following steps: at the same time, sending a frame synchronization trigger signal to the control module of the TOF camera and the control module of the AB frame camera respectively.

3. The camera system control method according to claim 1, characterized in that: The frequency of the frame synchronization trigger signal is 60 Hz-80 Hz.

4. The camera system control method according to claim 1, characterized in that: The first infrared control signal and the second infrared control signal have the same frequency.

5. The camera system control method according to claim 1, characterized in that: The infrared emitting devices of the TOF camera and the AB frame camera are turned on for the same duration each time.

6. A camera system, characterized in that: The invention comprises a controller, a TOF camera and an AB frame camera, wherein the TOF camera and the AB frame camera are respectively connected to the controller, and the controller, the control module of the TOF camera and the control module of the AB frame camera use the control method of a camera system according to any one of claims 1 to 5 to control the TOF camera and the AB frame camera to output images.

Images