Camera module, control method and robot

Abstract

The invention relates to a camera module, a control method and a robot, and the camera module comprises a color camera which is used for collecting color image information; a plurality of depth cameras, at least two depth cameras have different visual angles and ranging ranges, and each depth camera is used for collecting depth image information; and the main control module is coupled with the color camera and each depth camera, and the main control module is used for determining target image information according to the color image information and each piece of depth image information. In the plurality of depth cameras, at least two depth cameras have different visual angles and ranging ranges, so that the plurality of pieces of depth image information reflect different breadths and distances of the environment information. As the target image information is determined according to the color image information and the depth image information, the target image information can reflect wide-viewing-angle and long-distance environment information, so that the shooting performance of the camera module is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image acquisition technology, and in particular to a camera module, a control method and a robot. Background Art

[0002] At present, with the development of robot technology, robots can interact with users and assist users in handling daily affairs, and are favored by the majority of users. During the movement of the robot, it is necessary to collect and output image information of the surrounding environment through the internal camera module to avoid collision. However, since the image information output by the camera module cannot reflect the environmental information well, the shooting performance of the camera module is poor, which affects the use of the robot. Summary of the invention

[0003] In order to overcome the problems existing in the related art, the present disclosure provides a camera module, a control method and a robot.

[0004] According to a first aspect of the present disclosure, a camera module is provided, the camera module comprising:

[0005] A color camera, wherein the color camera is used to collect color image information;

[0006] A plurality of depth cameras, at least two of which have different viewing angles and ranging ranges, and each of which is used to collect depth image information;

[0007] A main control module, the main control module is coupled to the color camera and each of the depth cameras, and the main control module is used to determine target image information according to the color image information and each of the depth image information.

[0008] In some embodiments of the present disclosure, the plurality of depth cameras include a binocular camera and a time-of-flight camera;

[0009] Among them, the viewing angle of the binocular camera is greater than the viewing angle of the time-of-flight camera; and the ranging range of the binocular camera is smaller than the ranging range of the time-of-flight camera.

[0010] In some embodiments of the present disclosure, the binocular camera includes two fisheye cameras.

[0011] In some embodiments of the present disclosure, the main control module includes:

[0012] An image signal processing unit, the image signal processing unit is coupled to the color camera, and the image signal processing unit is used to process the color image information;

[0013] A main control processing unit, the main control processing unit is coupled to the image signal processing unit and each of the depth cameras, and the main control processing unit is used to determine the target image information according to the processed color image information and each of the depth image information.

[0014] In some embodiments of the present disclosure, the camera module further includes:

[0015] A positioning module, the positioning module is coupled to the main control module, and the positioning module is used to provide time to the main control module and trigger the color camera and each of the depth cameras to perform exposure.

[0016] In some embodiments of the present disclosure, the camera module further includes:

[0017] An inertial measurement unit is coupled to the main control module.

[0018] In some embodiments of the present disclosure, the camera module further includes:

[0019] A data transmission interface, the data transmission interface is coupled to the main control module, and the data transmission interface is used to transmit the target image information.

[0020] In some embodiments of the present disclosure, the camera module further includes:

[0021] A first circuit board, wherein the main control module is arranged on the first circuit board;

[0022] A second circuit board, one side of the second circuit board is coupled to the color camera and each of the depth cameras, and the other side of the second circuit board is coupled to the first circuit board.

[0023] In some embodiments of the present disclosure, the first circuit board is provided with a first interface and a second interface, and the second circuit board is provided with a third interface and a fourth interface; the first interface is coupled to the third interface, and the second interface is coupled to the fourth interface; the first interface and the third interface are used to transmit single-ended signals, and the second interface and the fourth interface are used to transmit differential signals.

[0024] According to a second aspect of the present disclosure, a method for controlling a camera module is provided, and the method for controlling the camera module includes:

[0025] Acquire the color image information and each of the depth image information;

[0026] The target image information is determined according to the color image information and each of the depth image information.

[0027] In some embodiments of the present disclosure, determining the target image information according to the color image information and each of the depth image information includes:

[0028] The color image information and each of the depth image information are input into a preset model to obtain the target image information output by the preset model.

[0029] In some embodiments of the present disclosure, after acquiring the color image information and the depth image information, the method for controlling the camera module further includes:

[0030] Performing time stamping on the color image information and each of the depth image information; and / or,

[0031] The target image information is time-stamped.

[0032] According to a third aspect of the present disclosure, a robot is provided, comprising the camera module as described above.

[0033] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0034] The camera module includes a color camera, multiple depth cameras and a main control module, and the main control module is coupled to the color camera and each depth camera. The main control module is used to determine the target image information based on the received color image information collected by the color camera and the depth image information collected by each depth camera. Since at least two depth cameras have different viewing angles and ranging ranges among the multiple depth cameras, the multiple depth image information reflects different breadths and distances of environmental information. Since the target image information is determined based on the color image information and each depth image information, the target image information can reflect the environmental information of a wide viewing angle and a long distance, thereby improving the shooting performance of the camera module.

[0035] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0037] Figure 1 is a structural schematic diagram of a camera module provided by an exemplary embodiment of the present disclosure;

[0038] Figure 2 is a structural schematic diagram of a camera module provided by another exemplary embodiment of the present disclosure;

[0039] Figure 3is a structural schematic diagram of a camera module provided by another exemplary embodiment of the present disclosure;

[0040] Figure 4 is a structural schematic diagram of a camera module provided by another exemplary embodiment of the present disclosure;

[0041] Figure 5 is a structural schematic diagram of a camera module provided by another exemplary embodiment of the present disclosure;

[0042] Figure 6 is a schematic structural diagram of a storage unit provided by an exemplary embodiment of the present disclosure;

[0043] Figure 7 is a structural schematic diagram of a camera module provided by another exemplary embodiment of the present disclosure;

[0044] Figure 8 is a flow chart of a control method of a camera module provided by an exemplary embodiment of the present disclosure;

[0045] Fig. 9 It is a system block diagram of a robot provided by an exemplary embodiment of the present disclosure.

[0046] In the figure:

[0047] 10-color camera; 20-depth camera; 21-binocular camera; 22-time-of-flight camera; 30-main control module; 31-image signal processing unit; 32-main control processing unit; 40-positioning module; 50-inertial measurement unit; 60-data transmission interface; 70-storage unit; 80-indicator light; 321-programmable logic unit; 322-processing system; 400-robot; 402-processing component; 404-memory; 406-power supply component; 408-multimedia component; 410-audio component; 412-input and output Output interface; 414-sensor component; 416-communication component; 420-processor; R1-first resistor; R2-second resistor; R3-third resistor; R4-fourth resistor; R5-fifth resistor; R6-sixth resistor; R7-seventh resistor; C1-first capacitor; C2-second capacitor; ZD-unidirectional voltage regulator; ZD1-first bidirectional voltage regulator; ZD2-second bidirectional voltage regulator; ZD3-third bidirectional voltage regulator; ZD4-fourth bidirectional voltage regulator; VCC1-first power supply; VCC2-second power supply; GND-ground terminal. DETAILED DESCRIPTION

[0048] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims. It should also be understood that the term "and / or" as used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0049] At present, with the development of robot technology, robots can interact with users and assist users in handling daily affairs, and are favored by the majority of users. During the movement of the robot, it is necessary to collect and output image information of the surrounding environment through the internal camera module. Since the robot needs to determine the distance between it and the obstacle to avoid collision or fall, the camera module often uses a stereo color (RGBD) camera module. The stereo color camera module includes a color camera and a depth camera. By combining the image information collected by the color camera and the depth camera to enhance the output image information, the robot can move better.

[0050] In the related art, a camera module for a robot is provided, including a binocular camera, a structured light camera or a time of flight (TOF) camera. However, the binocular camera has the problem of short distance measurement, the structured light camera has the problem of short distance measurement and high power consumption, and the time of flight camera has the problem of narrow viewing angle. When a binocular camera, a structured light camera or a time of flight camera is used as the camera module of the robot, since the image information output by the camera module to the robot reflects the environmental information with a narrow viewing angle or a short distance, it is difficult for the robot to avoid obstacles and may cause bumps, falls and other problems. Therefore, the image information output by the camera module cannot reflect the environmental information well, and the shooting performance of the camera module is poor, which affects the use of the robot.

[0051] Based on this, the present disclosure provides a camera module, which includes a color camera and multiple depth cameras. Since the viewing angles and ranging ranges of the multiple depth cameras are different, the breadth and distance of the environmental information reflected by the multiple depth image information are different, and the depth image information can be complementary to overcome the defects of a single depth camera. Since the target image information output by the camera module can combine the advantages of each depth image information to reflect the environmental information of a wide viewing angle and a long distance, the shooting performance of the camera module is improved.

[0052] An exemplary embodiment of the present disclosure provides a camera module, such as Figure 1As shown, the camera module includes a color (RGB) camera 10, multiple depth cameras 20 and a main control module 30. The color camera 10 is used to collect color image information. Among the multiple depth cameras 20, at least two depth cameras 20 have different viewing angles and ranging ranges, and each depth camera 20 is used to collect depth image information. The main control module 30 is coupled to the color camera 10 and each depth camera 20, and is used to determine the target image information according to the color image information and each depth image information.

[0053] In this embodiment, the camera module includes a color camera, multiple depth cameras and a main control module, and the main control module is coupled to the color camera and each depth camera. The main control module is used to determine the target image information based on the received color image information collected by the color camera and the depth image information collected by each depth camera. Since at least two depth cameras have different viewing angles and ranging ranges among the multiple depth cameras, the multiple depth image information reflects different breadths and distances of environmental information. Since the target image information is determined based on the color image information and the depth image information, the target image information can reflect the environmental information with a wide viewing angle and a long distance, thereby improving the shooting performance of the camera module.

[0054] In one embodiment, if Figure 2 As shown, the multiple depth cameras 20 include a binocular camera 21 and a time-of-flight (ToF) camera 22. The viewing angle of the binocular camera 21 is greater than that of the time-of-flight camera 22. The ranging range of the binocular camera 21 is smaller than that of the time-of-flight camera 22.

[0055] In this embodiment, due to the large viewing angle of the binocular camera, the depth image information collected by the binocular camera can reflect the environmental information of a wide viewing angle. Due to the large ranging range of the time-of-flight camera, the depth image information collected by the time-of-flight camera can reflect the environmental information of a long distance. Since the multiple depth cameras include a binocular camera and a time-of-flight camera, by combining the advantages of the binocular camera and the time-of-flight camera, the target image information can reflect the environmental information of a wide viewing angle and a long distance, thereby improving the shooting performance of the camera module.

[0056] Exemplarily, the binocular camera 21 may include two mono cameras, which can increase the robustness of the visual algorithm of the main control module 30 outputting the target image information so that the target image information can reflect the environmental information with a wide viewing angle and a long distance, and reduce the volume of the camera module.

[0057] In one embodiment, the binocular camera 21 includes two fisheye cameras, and each micro camera is a fisheye camera.

[0058] In this embodiment, since the fisheye camera has a wide viewing angle, the depth image information collected by the two fisheye cameras can reflect the environmental information of the omnidirectional viewing angle. When the target image information is determined by the color image information collected by the color camera, the depth image information collected by the fisheye camera, and the depth image information collected by the time-of-flight camera, the target image information can reflect the environmental information of the omnidirectional viewing angle and the long distance, thereby improving the shooting performance of the camera module.

[0059] Exemplarily, the time-of-flight camera 22 may include an indirect time-of-flight (iToF) camera and / or a direct time-of-flight (dToF) camera.

[0060] It is understandable that the multiple depth cameras 20 may include, in addition to the binocular camera 21 and the time-of-flight camera 22, further include a structured light camera or other camera for collecting depth image information.

[0061] In one embodiment, the main control module 30 includes an image signal processing (ISP) unit 31 and a main control processing unit 32. The image signal processing unit 31 is coupled to the color camera 10 and is used to process the color image information. The main control processing unit 32 is coupled to the image signal processing unit 31 and each depth camera 20 and is used to determine the target image information according to the processed color image information and each depth image information.

[0062] In this embodiment, the image information collected by the color camera is processed by the image signal processing unit, which can improve the quality of the color image information, so that the main control processing unit receives high-quality color image information and reduces the complexity of the main control processing unit in processing the image information. Since the target image information is determined based on the processed color image information and each depth image information, the target image information can more clearly reflect the environmental information, thereby improving the shooting performance of the camera module.

[0063] Exemplarily, the main control processing unit 32 may include a field programmable gate array (FPGA) circuit. The field programmable gate array circuit includes a programmable logic unit (PL) and a processing system (PS). The programmable logic unit and the processing system are respectively coupled to corresponding random access memories.

[0064] In one embodiment, if Figure 3As shown, the camera module further includes a positioning module 40. The positioning module 40 is coupled to the main control module 30, and is used to provide timing to the main control module 30 and trigger the color camera 10 and each depth camera 20 to perform exposure.

[0065] In this embodiment, the positioning module can provide time to the main control module, so that the main control module can determine the time when the color camera and each depth camera collect image information. The positioning module can trigger the color camera and each depth camera to be exposed to collect image information of the environment at a fixed time. Since the positioning module has the function of providing time and triggering camera exposure, the target image information can be added to the exposure time so that the robot can determine the time when each target image information is obtained, thereby improving the reliability of the camera module.

[0066] For example, the positioning module 40 may include a global positioning system (GPS) module or a Beidou positioning module. Figure 4 As shown, when the positioning module 40 includes a global positioning system module and the main control processing unit 32 includes a field programmable gate array circuit, the first end of the positioning module 40 is coupled to the first interrupt interface of the programmable logic unit 321 of the main control processing unit 32 and the second interrupt interface of the processing system 322, for transmitting a pulse per second (Pulse Per Second, PPS) signal. The second end of the positioning module 40 is coupled to the second interrupt interface of the processing system 322, for receiving the target signal generated after the pulse per second signal triggers the processing system 322. The frequency of the target signal can be, for example, 20Hz or 30Hz. At the same time, the target signal is also output from the processing system 322 to the color camera 10 and each depth camera 20 to trigger the color camera 10 and each depth camera 20 to expose. Moreover, the trigger edge of the target signal is at the same time as the trigger edge of the pulse per second signal after a certain period of time. That is, the target signal is synchronized with the pulse per second signal. The third and fourth ends of the positioning module 40 are coupled to the serial communication interface of the programmable logic unit 321 for transmitting data. At this time, the color camera 10 is coupled to the processing system 322 via the image signal processing unit 31 , and each depth camera 20 is coupled to the processing system 322 .

[0067] Exemplarily, in order to prevent the main control processing unit 32 or the positioning module 40 from malfunctioning due to voltage mismatch, a level shift circuit may be provided between the main control processing unit 32 and the positioning module 40 so that the voltages of the signals transmitted between the main control processing unit 32 and the positioning module 40 can be matched.

[0068] For example, Figure 5As shown, the first end of the positioning module 40 is coupled to the main control processing unit 32 through the first resistor R1, and a first bidirectional voltage regulator ZD1 is coupled between the first end and the ground terminal GND. The second end of the positioning module 40 is coupled to the main control processing unit 32 through the second resistor R2, and a second bidirectional voltage regulator ZD2 is coupled between the second end and the ground terminal GND. The third end of the positioning module 40 is coupled to the main control processing unit 32 through the third resistor R3, and a third bidirectional voltage regulator ZD3 is coupled between the third end and the ground terminal GND. The fourth end of the positioning module 40 is coupled to the main control processing unit 32 through the fourth resistor R4, and a fourth bidirectional voltage regulator ZD4 is coupled between the fourth end and the ground terminal GND. The common end of the positioning module 40 is used to couple with the ground terminal GND. The power supply end of the positioning module 40 is used to couple with the first power supply VCC1, and a first capacitor C1 and a unidirectional voltage regulator ZD are coupled in parallel between the power supply end and the ground terminal GND.

[0069] In one embodiment, if Figure 3 As shown, the camera module further includes an inertial measurement unit (IMU) 50 . The IMU 50 is coupled to the main control module 30 .

[0070] In this embodiment, the vibration of the camera module can be suppressed by the inertial measurement unit, thereby improving the shooting performance of the camera module.

[0071] Exemplarily, the inertial measurement unit 50 may include a 16-bit three-axis gyroscope and a 16-bit three-axis accelerometer, which are used to suppress the resonance of the circuit board where the camera module is located or the vibration that may occur in the camera module structure, and has high vibration robustness. The inertial measurement unit 50 has an extremely wide accelerometer range. In addition, the inertial measurement unit 50 can provide stable and reliable data for simultaneous localization and mapping (SLAM) to improve the quality of target image information.

[0072] In one embodiment, the camera module further includes a data transmission interface 60. The data transmission interface 60 is coupled to the main control module 30 and is used to transmit target image information.

[0073] In this embodiment, the main control module can transmit the target image information to the robot body through the data transmission interface, thereby reducing the complexity of the target image information transmission.

[0074] Exemplarily, the data transmission interface 60 may include a USB interface, supporting USB 3.0 protocol and USB 2.0 protocol, so as to facilitate the transmission of target image information and improve the speed of the transmission of target image information.

[0075] In one embodiment, if Figure 3 and Figure 6 As shown, the camera module further includes a storage unit 70. The storage unit 70 is coupled to the main control module 30 and is used to store programs to implement program guidance and configuration.

[0076] In this embodiment, the program of the camera module is stored in the storage unit, so that the main control module can read the corresponding program to guide and configure the program, thereby improving the reliability of the camera module.

[0077] Exemplarily, the first end, the second end, the third end, the fourth end, the fifth end and the sixth end of the storage unit 70 are all coupled to the main control module 30. Among them, the first end of the storage unit 70 is used to transmit the clock signal, the second end is used to transmit the data signal corresponding to the clock signal, and the third end to the sixth end are used to transmit the input and output data signals. The fifth end of the storage unit 70 is also used for write protection. The sixth end of the storage unit 70 is also used for bus request. The second end of the storage unit 70 is coupled to the second power supply VCC2 through the fifth resistor R5, the fifth end is coupled to the second power supply VCC2 through the sixth resistor R6, and the sixth end is coupled to the second power supply VCC2 through the seventh resistor R7. The power supply end of the storage unit 70 is coupled to the second power supply VCC2, the common end is coupled to the ground end GND, and a second capacitor C2 is coupled between the power supply end and the ground end GND.

[0078] In one embodiment, if Figure 3 As shown, the camera module further includes an indicator light 80. The indicator light 80 is coupled to the main control module 30 and is used to indicate the state of the camera module.

[0079] In this embodiment, the status of the camera module is indicated by an indicator light, which facilitates determination of whether the camera module fails, thereby reducing the complexity of determining the status of the camera module.

[0080] In one embodiment, the camera module further includes a power management circuit, which is coupled to the color camera 10 , each depth camera 20 , and the main control module 30 , and is used to supply power to the color camera 10 , each depth camera 20 , and the main control module 30 .

[0081] In this embodiment, the voltage input to the camera module can be converted through the power management circuit to power the color camera, the depth camera and the main control module, thereby improving the reliability of the camera module operation.

[0082] Exemplarily, the power management circuit is also coupled to the positioning module 40, the inertial measurement unit 50 and the storage unit 70, and is used to power the positioning module 40, the inertial measurement unit 50 and the storage unit 70. Among them, the data transmission interface 60 can be directly powered by the voltage of the camera module.

[0083] In one embodiment, the camera module further includes a first circuit board and a second circuit board. The main control module 30 is disposed on the first circuit board. One side of the second circuit board is coupled to the color camera 10 and each depth camera 20, and the other side is coupled to the first circuit board.

[0084] In this embodiment, since the color camera and each depth camera are coupled to the main control module through the second circuit board and the first circuit board, it is convenient to replace the color camera and each depth camera, thereby reducing the complexity of adjusting and repairing the camera module.

[0085] In one embodiment, the first circuit board is provided with a first interface and a second interface, and the second circuit board is provided with a third interface and a fourth interface. The first interface is coupled to the third interface, and the second interface is coupled to the fourth interface. The first interface and the third interface are used to transmit single-ended signals, and the second interface and the fourth interface are used to transmit differential signals. In addition, the first power supply VCC1 and the third power supply are coupled to the first circuit board.

[0086] In this embodiment, by transmitting single-ended signals and differential signals respectively through two groups of interfaces, the interference degree of the signals transmitted by the color camera and each depth camera can be reduced, thereby improving the reliability of the camera module.

[0087] Exemplarily, the main control module 30 further has a plurality of mobile industry processor interfaces (MIPIs). Each MIPI is coupled to the color camera 10 and each depth camera 20 for transmitting color image information and each depth image information.

[0088] Exemplarily, the inertial measurement unit 50 , the data transmission interface 60 , the storage unit 70 , the indicator light 80 , and the power management circuit are all coupled to the main control processing unit 32 of the main control module 30 .

[0089] For example, Figure 7As shown, the camera module includes a color camera 10, a binocular camera 21, a time-of-flight camera 22, an image signal processing unit 31, a main control processing unit 32, a positioning module 40, an inertial measurement unit 50, a data transmission interface 60, a storage unit 70, an indicator light 80, and a power management circuit. The color camera 10 is coupled to the main control processing unit 32 through the image signal processing unit 31. The binocular camera 21, the time-of-flight camera 22, the positioning module 40, the inertial measurement unit 50, the data transmission interface 60, the storage unit 70, and the indicator light 80 are all coupled to the main control processing unit 32. The power management circuit is coupled to the color camera 10, the binocular camera 21, the time-of-flight camera 22, the image signal processing unit 31, the main control processing unit 32, the positioning module 40, the inertial measurement unit 50, and the storage unit 70.

[0090] An exemplary embodiment of the present disclosure further provides a method for controlling a camera module, which can be applied to the above-mentioned camera module, such as Figure 8 As shown, the control method of the camera module includes:

[0091] S100: Acquire color image information and each depth image information.

[0092] S200: Determine target image information according to color image information and each depth image information.

[0093] In this embodiment, the color image information collected by the color camera and the depth image information collected by each depth camera are obtained, and the target image information is determined based on the color image information and each depth image information. Since the width and distance of the environmental information reflected by the multiple depth image information are different, the target image information can reflect the environmental information of a wide viewing angle and a long distance, thereby improving the shooting performance of the camera module.

[0094] Exemplarily, each depth image information can be collected by a binocular camera and a time-of-flight camera.

[0095] In one embodiment, the target image information is determined according to the color image information and each depth image information in step S200 by:

[0096] The color image information and each depth image information are input into the preset model to obtain the target image information output by the preset model.

[0097] In this embodiment, by inputting color image information and each depth image information into a preset model, the target image information output by the preset model through the visual algorithm can be obtained for transmission. Since the preset model is a pre-trained model, the target image information is output by the preset model, which reduces the complexity of the image information output by the camera module.

[0098] Exemplarily, the preset model may be a pre-trained model, which may output corresponding target image information according to the color image information and each depth image information. The preset model may be a simultaneous positioning and mapping model, which outputs the target image information through the visual algorithm in the simultaneous positioning and mapping model.

[0099] In one embodiment, after acquiring the color image information and the depth image information in step S100, the method for controlling the camera module further includes:

[0100] The color image information and each depth image information are time-stamped.

[0101] In this embodiment, by time-stamping the color image information and each depth image information, the main control module can determine the time when the color image information and each depth image information are collected. When processing the color image information and each depth image information to obtain the target image information, it can prevent the color image information used from not corresponding to the depth image information, thereby improving the reliability of the camera module.

[0102] In one embodiment, after acquiring the color image information and the depth image information in step S100, the method for controlling the camera module further includes:

[0103] Time stamp the target image information.

[0104] In this embodiment, by time-stamping the target image information, the output target image information corresponds to different times so that the robot can sort the target image information, thereby improving the reliability of the camera module.

[0105] Exemplarily, when the trigger edge of the target signal arrives, the color camera and each depth camera will be triggered to perform exposure, and after the exposure is completed, the main control processing unit obtains the color image information and each depth image information. When time-stamping the color image information and each depth image information, and / or time-stamping the target image information, when the trigger edge of the target signal arrives, the main control processing unit will be triggered to read the current time in the timer, and the color image information and each depth image information will be time-stamped with the current time in the timer, and / or the target image information will be time-stamped.

[0106] Exemplarily, when the positioning module is used to synchronize time with the main control module, the time T0 of the timer in the main control processing unit can be recorded when the trigger edge of the current second pulse signal arrives. When the main control processing unit performs serial communication with the positioning module, it can determine the world standard time T1 corresponding to the trigger edge of the previous second pulse signal based on the received data, and read the time T2 of the timer in the main control processing unit to determine the time T3=T1+(T2-T0). If the difference between T3 and T2 is greater than the preset threshold, the time T3 is written into the timer of the main control processing unit to complete the timing function. Among them, the preset threshold can be, for example, less than the time difference between the two second pulse signals.

[0107] In an exemplary embodiment, a robot is provided, the robot comprising the camera module as described above. After adopting the camera module, since the target image information can reflect the environmental information of a wide viewing angle and a long distance, the possibility of the robot being bumped or falling is reduced.

[0108] refer to Fig. 9 As shown, the robot 400 may include one or more of the following components: a processing component 402 , a memory 404 , a power component 406 , a multimedia component 408 , an audio component 410 , an input / output (I / O) interface 412 , a sensor component 414 , and a communication component 416 .

[0109] The processing component 402 generally controls the overall operation of the robot 400, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 402 may include one or more processors 420 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 402 may include one or more modules to facilitate the interaction between the processing component 402 and other components. For example, the processing component 402 may include a multimedia module to facilitate the interaction between the multimedia component 408 and the processing component 402.

[0110] The memory 404 is configured to store various types of data to support operations on the robot 400. Examples of such data include instructions for any application or method operating on the robot 400, contact data, phone book data, messages, pictures, videos, etc. The memory 404 can be implemented by any type of volatile or non-volatile storage terminal or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0111] The power supply assembly 406 provides power to various components of the robot 400. The power supply assembly 406 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the robot 400.

[0112] The multimedia component 408 includes a screen that provides an output interface between the robot 400 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0113] The audio component 410 is configured to output and / or input audio signals. For example, the audio component 410 includes a microphone (MIC), and when the robot 400 is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 404 or sent via the communication component 416. In some embodiments, the audio component 410 also includes a speaker for outputting audio signals.

[0114] I / O interface 412 provides an interface between processing component 402 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0115] The sensor assembly 414 includes one or more sensors for providing various aspects of the status assessment of the robot 400. For example, the sensor assembly 414 can detect the open / closed state of the robot 400, the relative positioning of components, such as the display and keypad of the robot 400, the sensor assembly 414 can also detect the position change of the robot 400 or a component of the robot 400, the presence or absence of user contact with the robot 400, the orientation or acceleration / deceleration of the robot 400, and the temperature change of the robot 400. The sensor assembly 414 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 414 can also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 414 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0116] The communication component 416 is configured to facilitate wired or wireless communication between the robot 400 and other terminals. The robot 400 can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G, 5G, or a combination thereof. In an exemplary embodiment, the communication component 416 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 416 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0117] In an exemplary embodiment, the robot 400 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing terminals (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-mentioned methods.

[0118] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 404 including instructions, and the above instructions can be executed by the processor 420 of the robot 400 to complete the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a tape, a floppy disk, and an optical data storage terminal, etc. When the instructions in the storage medium are executed by the processor of the terminal, the terminal is enabled to perform the method shown in the above embodiment.

[0119] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0120] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0121] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A camera module, characterized in that: The camera module comprises: A color camera, wherein the color camera is used to collect color image information; A plurality of depth cameras, at least two of which have different viewing angles and ranging ranges, and each of which is used to collect depth image information; A main control module, the main control module is coupled to the color camera and each of the depth cameras, and the main control module is used to determine target image information according to the color image information and each of the depth image information.

2. The camera module according to claim 1, characterized in that: The multiple depth cameras include a binocular camera and a time-of-flight camera; Among them, the viewing angle of the binocular camera is greater than the viewing angle of the time-of-flight camera; and the ranging range of the binocular camera is smaller than the ranging range of the time-of-flight camera.

3. The camera module according to claim 2, characterized in that: The binocular camera includes two fisheye cameras.

4. The camera module according to claim 1, characterized in that: The main control module comprises: An image signal processing unit, the image signal processing unit is coupled to the color camera, and the image signal processing unit is used to process the color image information; A main control processing unit, the main control processing unit is coupled to the image signal processing unit and each of the depth cameras, and the main control processing unit is used to determine the target image information according to the processed color image information and each of the depth image information.

5. The camera module according to claim 1, characterized in that: The camera module also includes: A positioning module, the positioning module is coupled to the main control module, and the positioning module is used to provide time to the main control module and trigger the color camera and each of the depth cameras to perform exposure.

6. The camera module according to claim 1, characterized in that: The camera module also includes: An inertial measurement unit is coupled to the main control module.

7. The camera module according to claim 1, characterized in that: The camera module also includes: A data transmission interface, the data transmission interface is coupled to the main control module, and the data transmission interface is used to transmit the target image information.

8. The camera module according to any one of claims 1 to 7, characterized in that: The camera module also includes: A first circuit board, wherein the main control module is arranged on the first circuit board; A second circuit board, one side of the second circuit board is coupled to the color camera and each of the depth cameras, and the other side of the second circuit board is coupled to the first circuit board.

9. The camera module according to claim 8, characterized in that: The first circuit board is provided with a first interface and a second interface, and the second circuit board is provided with a third interface and a fourth interface; the first interface is coupled to the third interface, and the second interface is coupled to the fourth interface; the first interface and the third interface are used to transmit single-ended signals, and the second interface and the fourth interface are used to transmit differential signals.

10. A method for controlling a camera module according to any one of claims 1 to 9, characterized in that: The control method of the camera module includes: Acquire the color image information and each of the depth image information; The target image information is determined according to the color image information and each of the depth image information.

11. The method for controlling a camera module according to claim 10, characterized in that: The step of determining the target image information according to the color image information and each of the depth image information comprises: The color image information and each of the depth image information are input into a preset model to obtain the target image information output by the preset model.

12. The method for controlling a camera module according to claim 10, characterized in that: After acquiring the color image information and the depth image information, the control method of the camera module further includes: Performing time stamping on the color image information and each of the depth image information; and / or, The target image information is time-stamped.

13. A robot, characterized in that: The robot comprises the camera module as described in any one of claims 1 to 9.

Images