Image acquisition method, device and system, electronic equipment and medium
By performing clock synchronization and data packaging processing on the sensors in the camera module, the problem of being unable to accurately measure one's own motion state in the prior art is solved, and high-precision motion state measurement and data transmission are achieved.
Patent Information
- Application Number
- CN202310868793.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-07-17
- Publication Date
- 2025-01-17
AI Technical Summary
The existing camera modules cannot accurately measure their own motion state, lack the scale information of feature points, and have cumulative errors and noise problems.
By clocking the first sensor and the second sensor, the second sensor is triggered to collect images using the first sensor interrupt signal, and the main control chip sends the sensor data, images and timestamps to the robot host, and the packaging and merging technology is used to solve the problem. Transmission preemption bus problem.
Accurate measurement of one's own motion state is achieved, cumulative errors and noise are eliminated, and measurement accuracy and data transmission efficiency are improved.
Smart Images
Figure CN119325024A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of camera technology, and in particular to an image acquisition method, device, system, electronic device and medium. Background Art
[0002] Camera module is a product with a high usage rate in modern technology. It is a modular device that integrates a camera and related electronic components. Camera modules are widely used in various applications, including home, factory, medical, etc. Its main function is to record and transmit visual information by capturing images and videos.
[0003] Existing camera modules usually only have RGB functions, cannot accurately measure their own motion status, and lack scale information of feature points. Summary of the invention
[0004] The present disclosure provides an image acquisition method, device, system, electronic device and medium, including: clock synchronization of a first sensor and a second sensor; an interrupt signal of the first sensor triggers the second sensor to collect an image; a main control chip sends the first sensor data, image and timestamp to a robot host. Compared with the prior art, the present disclosure selects a combination of two sensors, which can not only accurately measure its own motion state, but also has no cumulative error and noise.
[0005] In order to achieve the above objectives, the present invention adopts the following technical solutions:
[0006] The first aspect of the present disclosure provides an image acquisition method, comprising:
[0007] The first sensor and the second sensor are clock synchronized.
[0008] The first sensor interrupt signal triggers the second sensor to capture an image.
[0009] The main control chip sends the first sensor data, image and timestamp to the robot host.
[0010] Furthermore, the image acquisition method, before the main control chip sends the first sensor data, image and timestamp to the robot host, further includes:
[0011] The main control chip packages the first sensor data, image and timestamp, so that the main control chip sends the packaged first sensor data, image and timestamp to the robot host.
[0012] Furthermore, the image acquisition method, after the main control chip packages the first sensor data, image and timestamp, further includes:
[0013] The first sensor data and the image are combined into a pair of image data.
[0014] Furthermore, the image acquisition method, after the main control chip sends the first sensor data, image and timestamp to the robot host, further includes:
[0015] The first sensor data, image, and timestamp information are stored.
[0016] Furthermore, in the image acquisition method, the main control chip sends the first sensor data, image and timestamp to the robot host, including:
[0017] The main control chip sends the first sensor data, image and timestamp to the robot host through the UVC protocol.
[0018] Furthermore, the image acquisition method combines the first sensor data and the image into a pair of image data, including:
[0019] The image is YUV-encoded, and the first sensor data and the image are combined into a pair of image data.
[0020] Furthermore, in the image acquisition method, the first sensor includes:
[0021] The first sensor is an IMU sensor.
[0022] Furthermore, in the image acquisition method, the second sensor includes:
[0023] The second sensor is the Sensor sensor.
[0024] A second aspect of the present disclosure provides an image acquisition device, comprising:
[0025] The synchronization unit is used to perform clock synchronization on the first sensor and the second sensor.
[0026] The acquisition unit is used for triggering the second sensor to acquire an image by an interrupt signal from the first sensor.
[0027] The sending unit is used for the main control chip to send the first sensor data, image and timestamp to the robot host.
[0028] Furthermore, the image acquisition device further includes:
[0029] The packing unit is used for the main control chip to pack the first sensor data, image and time stamp so that the main control chip can send the packed first sensor data, image and time stamp to the robot host.
[0030] Furthermore, the image acquisition device further includes:
[0031] The merging unit is used to merge the first sensor data and the image into a set of image data.
[0032] Furthermore, the image acquisition device further includes:
[0033] The storage unit is used to store the first sensor data, the image and the time stamp information.
[0034] A third aspect of the present disclosure provides an image acquisition system, comprising:
[0035] The first sensor, the second sensor, the main control chip and the robot host.
[0036] The first sensor is connected to the second sensor and the main control chip respectively, and is used to trigger the second sensor to collect images.
[0037] The second sensor is connected to the first sensor and the main control chip respectively, and is used to collect the image.
[0038] The main control chip is connected to the first sensor and the second sensor respectively, and the main control chip is communicatively connected to the robot host, and is used to send the first sensor data, the image and the timestamp to the robot host.
[0039] The robot host is communicatively connected to the main control chip, and is used to receive and store the first sensor data, the image and the timestamp.
[0040] A fourth aspect of the present disclosure provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above method are implemented.
[0041] A fifth aspect of the present disclosure provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.
[0042] The present disclosure provides an image acquisition method, device, system, electronic device and medium, including: clock synchronization of a first sensor and a second sensor; an interrupt signal of the first sensor triggers the second sensor to collect an image; a main control chip sends the first sensor data, image and timestamp to a robot host. Compared with the prior art, the present disclosure selects a combination of two sensors, which can not only accurately measure its own motion state, but also has no cumulative error and noise.
[0043] At the same time, the main control chip packages the first sensor data, image and timestamp, and merges the first sensor data and image into one image data, thereby solving the problem of two sensors independently transmitting and occupying the bus.
[0044] Finally, the disclosed embodiment stores the first sensor data and image timestamp information for easy retrieval. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0046] Figure 1 This is a flow chart of an image acquisition method according to an embodiment of the present disclosure;
[0047] Figure 2 This is a flow chart of another method for acquiring an image in an embodiment of the present disclosure;
[0048] Figure 3 It is a schematic diagram of the structure of an image acquisition device in an embodiment of the present disclosure;
[0049] Figure 4 It is a schematic diagram of the structure of another image acquisition device in the embodiment of the present disclosure;
[0050] Figure 5 A schematic diagram of the structure of an image acquisition system in an embodiment of the present disclosure;
[0051] Figure 6 The present invention is a schematic diagram of the structure of an electronic device for image acquisition in an embodiment of the present invention. DETAILED DESCRIPTION
[0052] The technical solutions in the embodiments of the present disclosure will be described clearly and completely below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.
[0053] Unless otherwise defined, all technical and scientific terms used in the present disclosure have the same meanings as those commonly understood by technicians in the field of the present disclosure; the terms used in the present disclosure are only for the purpose of describing specific embodiments, and are not intended to limit the terms "including" and "having" and any variations thereof in the specification and claims of the present disclosure and the above-mentioned drawings, and are intended to cover non-exclusive inclusions.
[0054] In the description of the embodiments of the present disclosure, the technical terms "first", "second", etc. are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number, specific order or primary and secondary relationship of the indicated technical features. In the description of the embodiments of the present disclosure, the meaning of "multiple" is more than two, unless otherwise clearly and specifically defined.
[0055] In the description of the embodiments of the present disclosure, the term "and / or" is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.
[0056] In the description of the embodiments of the present disclosure, the term "multiple" refers to more than two (including two). Similarly, "multiple groups" refers to more than two groups (including two groups), and "multiple pieces" refers to more than two pieces (including two pieces).
[0057] In the description of the embodiments of the present disclosure, the orientations or positional relationships indicated by technical terms such as "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", and "circumferential" are based on the orientations or positional relationships shown in the accompanying drawings and are only for the convenience of describing the embodiments of the present disclosure and simplifying the description. They do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the embodiments of the present disclosure.
[0058] In the description of the embodiments of the present disclosure, unless otherwise clearly specified and limited, technical terms such as "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the embodiments of the present disclosure can be understood according to the specific circumstances.
[0059] Example 1
[0060] The present disclosure provides an image acquisition method, such as Figure 1 As shown, including:
[0061] 101. Perform clock synchronization on a first sensor and a second sensor.
[0062] Sensor: (transducer / sensor) is a detection device that can sense the information being measured and can transform the sensed information into electrical signals or other required forms of information output according to certain rules to meet the requirements of information transmission, processing, storage, display, recording and control. Among them, each sensor has its own clock source, and different sensors have different sampling frequencies. In order to effectively integrate the perception data of multiple sensors, time synchronization must be performed.
[0063] It should be noted that: this embodiment does not limit the specific names and models of the first sensor and the second sensor, and all sensors that can realize the functions of this embodiment are within the protection scope of this embodiment. The following is an example of the first sensor being an IMU sensor and the second sensor being a Sensor sensor.
[0064] IMU sensor: (Inertial measurement unit, IMU) is a sensor that measures the three-axis attitude angle (or angular rate) and acceleration of an object.
[0065] Sensor: It is the core component of the camera. Its function is to convert light signals into electrical signals. Sensor types include CCD and CMOS. In specific implementation, the implementer can determine the type of sensor based on actual conditions.
[0066] Clock synchronization: It is achieved by connecting each device that can generate a clock to the SCL line. Each of the above-mentioned devices may have its own independent clock, and the frequency, period, phase and duty cycle of each clock signal may be different. As a result of "line AND", the low-level width of the actual clock generated on the SCL line is determined by the device with the longest low-level duration, while the high-level width is determined by the device with the shortest high-level duration.
[0067] 102. The first sensor interrupt signal triggers the second sensor to capture an image.
[0068] 103. The main control chip sends the first sensor data, image and timestamp to the robot host.
[0069] The present disclosure provides an image acquisition method, including: synchronizing the clocks of a first sensor and a second sensor; an interrupt signal of the first sensor triggers the second sensor to collect an image; a main control chip sends the first sensor data, image and timestamp to a robot host. Compared with the prior art, the present disclosure selects a combination of two sensors, which can not only accurately measure its own motion state, but also has no accumulated error and noise.
[0070] Example 2
[0071] The present disclosure provides an image acquisition method, such as Figure 2 As shown, including:
[0072] 201. Perform clock synchronization on a first sensor and a second sensor.
[0073] Specifically, there are two methods for synchronizing the clocks of the first sensor and the second sensor:
[0074] The first is hardware synchronization or hard synchronization: using the same hardware to simultaneously issue trigger acquisition commands to synchronize the acquisition and measurement of each sensor. This allows the same information to be collected at the same time. For example, if multiple cameras use the same hardware pulse signal to achieve simultaneous exposure control, synchronization between multiple cameras can be achieved.
[0075] The second is software synchronization, namely time synchronization and space synchronization: a unified host provides a reference time for each sensor, and each sensor adds timestamp information to the data collected independently according to its own calibrated time. This can synchronize the timestamps of all sensors. However, since the collection cycles of each sensor are independent of each other, it is impossible to guarantee that the same information is collected at the same time.
[0076] It should be noted here that: the embodiment of the present disclosure does not impose any specific restrictions on the method of synchronizing the clocks of the first sensor and the second sensor, and the implementer may select the method according to his own needs.
[0077] In this embodiment, the first sensor is an IMU sensor, and the second sensor is a Sensor sensor.
[0078] 202. An interrupt signal from the first sensor triggers the second sensor to capture an image.
[0079] 203. The main control chip packages the first sensor data, the image and the timestamp.
[0080] Timestamp: (timestamp), usually a sequence of characters, uniquely identifies a certain moment in time. Digital timestamp technology is a variant application of digital signature technology.
[0081] 204. Combine the first sensor data and the image into a set of image data.
[0082] Specifically, the image is YUV-encoded, and the first sensor data and the image are combined into a pair of image data.
[0083] Among them, YUV is a color space, and color coding based on YUV is a common coding method for streaming media. Y represents lumen, U and V represent chroma and concentration. This expression was originally for signal compatibility between color TV and black and white TV. For each point in the image, Y determines its brightness and UV confirms its chroma. YUV color coding uses brightness and chroma to specify the color of the pixel. Among them, Y represents brightness (Luminance, Luma), while U and V represent chroma. Chroma defines two aspects of color: hue and saturation.
[0084] 205. The main control chip sends the packaged first sensor data, image and timestamp to the robot host.
[0085] Specifically, the main control chip sends the first sensor data, image and timestamp to the robot host through the UVC protocol.
[0086] UVC protocol: (USB video class or USB video deviceclass) is a protocol standard defined for USB video capture devices jointly launched by Microsoft and several other equipment manufacturers. It has now become one of the USBorg standards.
[0087] 206. Store the first sensor data and image timestamp information.
[0088] The present disclosure provides an image acquisition method, including: synchronizing the clocks of a first sensor and a second sensor; an interrupt signal of the first sensor triggers the second sensor to collect an image; a main control chip packages the first sensor data, an image and a timestamp; merges the first sensor data and the image into a pair of image data; the main control chip sends the packaged first sensor data, the image and the timestamp to a robot host; and stores the first sensor data and the image timestamp information. Compared with the prior art, the present disclosure selects a combination of two sensors, which can not only accurately measure its own motion state, but also has no accumulated error and noise.
[0089] At the same time, the main control chip packages the first sensor data, image and timestamp, and merges the first sensor data and image into one image data, thereby solving the problem of two sensors independently transmitting and occupying the bus.
[0090] Finally, the disclosed embodiment stores the first sensor data and image timestamp information for easy retrieval.
[0091] Example 3
[0092] The present disclosure provides an image acquisition device, such as Figure 3 As shown, including:
[0093] The synchronization unit 31 is used to perform clock synchronization on the first sensor and the second sensor.
[0094] The acquisition unit 32 is used for triggering the second sensor to acquire an image by an interrupt signal from the first sensor.
[0095] The sending unit 33 is used for the main control chip to send the first sensor data, image and timestamp to the robot host.
[0096] It should be noted here that the detailed description of each component of this embodiment can refer to other embodiments and will not be repeated here.
[0097] The present disclosure provides an image acquisition device, including: a synchronization unit that performs clock synchronization on a first sensor and a second sensor; a collection unit in which an interrupt signal of the first sensor triggers the second sensor to collect an image; a sending unit in which a main control chip sends the first sensor data, an image and a timestamp to a robot host. Compared with the prior art, the present disclosure selects a combination of two sensors, which can not only accurately measure its own motion state, but also has no accumulated error and noise.
[0098] Example 4
[0099] The present disclosure provides an image acquisition device, such as Figure 4 As shown, including:
[0100] The synchronization unit 41 is used to perform clock synchronization on the first sensor and the second sensor.
[0101] The acquisition unit 42 is used for triggering the second sensor to acquire an image by an interrupt signal from the first sensor.
[0102] The packing unit 43 is used for the main control chip to pack the first sensor data, image and timestamp, so that the main control chip sends the packed first sensor data, image and timestamp to the robot host.
[0103] The merging unit 44 is used to merge the first sensor data and the image into a set of image data.
[0104] The sending unit 45 is used for the main control chip to send the first sensor data, image and timestamp to the robot host.
[0105] The storage unit 46 is used to store the first sensor data, the image and the time stamp information.
[0106] It should be noted here that the detailed description of each component of this embodiment can refer to other embodiments and will not be repeated here.
[0107] The present disclosure provides an image acquisition device, comprising: a synchronization unit for performing clock synchronization on a first sensor and a second sensor; a collection unit for triggering the second sensor to collect images by an interrupt signal of the first sensor; a packaging unit for a main control chip to package the first sensor data, image and timestamp, so that the main control chip sends the packaged first sensor data, image and timestamp to a robot host; a merging unit for merging the first sensor data and image into a pair of image data; a sending unit for the main control chip to send the first sensor data, image and timestamp to the robot host; a storage unit for storing the first sensor data, image and timestamp information; compared with the prior art, the present disclosure selects a combination of two sensors, which can not only accurately measure its own motion state, but also has no accumulated error and noise.
[0108] At the same time, the main control chip of the packing unit packs the first sensor data, image and timestamp, and the merging unit merges the first sensor data and image into a set of image data, thereby solving the problem of two sensors independently transmitting and occupying the bus.
[0109] Finally, the storage unit of the embodiment of the present disclosure stores the first sensor data and image timestamp information for easy retrieval.
[0110] Example 5
[0111] The present disclosure provides an image acquisition system. Figure 5 As shown, including:
[0112] A first sensor 51 , a second sensor 52 , a main control chip 53 and a robot host 54 .
[0113] The first sensor 51 is connected to the second sensor 52 and the main control chip 53 respectively, and is used to trigger the second sensor 52 to capture images.
[0114] The second sensor 52 is connected to the first sensor 51 and the main control chip 53 respectively, and is used to collect images.
[0115] The main control chip 53 is connected to the first sensor 51 and the second sensor 52 respectively, and the main control chip 53 is connected to the robot host 54 for communication, and is used to send the data, image and timestamp of the first sensor 51 to the robot host 54;
[0116] The robot host 54 is in communication connection with the main control chip 53 and is used to receive and store the data, images and timestamps of the first sensor 51 .
[0117] It should be noted here that the detailed description of each component of this embodiment can refer to other embodiments and will not be repeated here.
[0118] The present disclosure provides an image acquisition system, including: a first sensor, a second sensor, a main control chip and a robot host, wherein the first sensor is used to trigger the second sensor to collect images; the second sensor is used to collect images; the main control chip is used to send the first sensor data, images and timestamps to the robot host; the robot host is used to receive and store the first sensor data, images and timestamps. Compared with the prior art, the present disclosure selects a combination of two sensors, which can not only accurately measure its own motion state, but also has no cumulative errors and noise.
[0119] Example 6
[0120] The present disclosure provides an electronic device for acquiring an image. Figure 6 As shown, it includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the above-mentioned image acquisition method when executing the computer program.
[0121] A processing device (e.g., a central processing unit, a graphics processor, etc.) 61 can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 62 or a program loaded from a storage device 68 into a random access memory (RAM) 63. Various programs and data required for the operation of the electronic device are also stored in the RAM 63. The processing device 61, the ROM 62, and the RAM 63 are connected to each other via a bus 64. An input / output (I / O) interface 65 is also connected to the bus 64.
[0122] Typically, the following devices may be connected to the I / O interface 65: input devices 66 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 67 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 68 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 69. The communication device 69 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 6 Each block shown in the figure may represent one device, or may represent multiple devices as required.
[0123] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 69, or installed from the storage device 68, or installed from the ROM 62. When the computer program is executed by the processing device 61, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.
[0124] The present disclosure provides an image acquisition electronic device, comprising: clock synchronization of a first sensor and a second sensor; an interrupt signal of the first sensor triggers the second sensor to collect an image; a main control chip packages the first sensor data, an image and a timestamp; merges the first sensor data and the image into a pair of image data; the main control chip sends the packaged first sensor data, the image and the timestamp to a robot host; and stores the first sensor data and the image timestamp information. Compared with the prior art, the present disclosure selects a combination of two sensors, which can not only accurately measure its own motion state, but also has no accumulated error and noise.
[0125] At the same time, the main control chip packages the first sensor data, image and timestamp, and merges the first sensor data and image into one image data, thereby solving the problem of two sensors independently transmitting and occupying the bus.
[0126] Finally, the disclosed embodiment stores the first sensor data and image timestamp information for easy retrieval.
[0127] Example 7
[0128] An embodiment of the present disclosure provides a computer-readable storage medium for image acquisition, wherein a computer program is stored on the computer-readable storage medium. When the computer program is executed by one or more processors, the above-mentioned image acquisition method is implemented.
[0129] It should be noted that the computer-readable medium in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
[0130] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.
[0131] The computer-readable medium may be included in the device; or it may exist independently without being assembled into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: performs clock synchronization on the first sensor and the second sensor; the first sensor interrupt signal triggers the second sensor to collect an image; the main control chip packages the first sensor data, image and timestamp; merges the first sensor data and image into a pair of image data; the main control chip sends the packaged first sensor data, image and timestamp to the robot host; and stores the first sensor data and image timestamp information.
[0132] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0133] The present disclosure provides a computer-readable storage medium for image acquisition, comprising: clock synchronization of a first sensor and a second sensor; an interrupt signal of the first sensor triggering the second sensor to collect an image; a main control chip packaging the first sensor data, image and timestamp; merging the first sensor data and image into a pair of image data; the main control chip sending the packaged first sensor data, image and timestamp to a robot host; storing the first sensor data and image timestamp information. Compared with the prior art, the present disclosure selects a combination of two sensors, which can not only accurately measure its own motion state, but also has no accumulated error and noise.
[0134] At the same time, the main control chip packages the first sensor data, image and timestamp, and merges the first sensor data and image into one image data, thereby solving the problem of two sensors independently transmitting and occupying the bus.
[0135] Finally, the disclosed embodiment stores the first sensor data and image timestamp information for easy retrieval.
[0136] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0137] The units described in some embodiments of the present disclosure may be implemented by software or hardware. The units described may also be set in a processor, for example, it may be described as: a processor includes a receiving unit, an information acquisition unit, a target determination unit, and a distribution unit. The names of these units do not constitute limitations on the units themselves in some cases.
[0138] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.
[0139] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.
[0140] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, not to limit them; although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention. In particular, as long as there is no structural conflict, the various technical features mentioned in each embodiment can be combined in any way. The present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions that fall within the scope of the claims.
Claims
1. An image acquisition method, characterized in that: include: synchronizing the clocks of the first sensor and the second sensor; The first sensor interrupt signal triggers the second sensor to capture an image; The main control chip sends the first sensor data, the image and the timestamp to the robot host.
2. The image acquisition method according to claim 1, characterized in that: Before the main control chip sends the first sensor data, the image and the timestamp to the robot host, it also includes: The main control chip packages the first sensor data, the image and the timestamp, so that the main control chip sends the packaged first sensor data, the image and the timestamp to the robot host.
3. The image acquisition method according to claim 2, characterized in that: After the main control chip packages the first sensor data, the image and the timestamp, the method further includes: The first sensor data and the image are combined into a pair of image data.
4. The image acquisition method according to claim 1, characterized in that: After the main control chip sends the first sensor data, the image and the timestamp to the robot host, the method further includes: The first sensor data, the image, and timestamp information are stored.
5. The image acquisition method according to claim 1, characterized in that: The main control chip sends the first sensor data, the image and the timestamp to the robot host, including: The main control chip sends the first sensor data, the image and the timestamp to the robot host through the UVC protocol.
6. The image acquisition method according to claim 3, characterized in that: Combining the first sensor data and the image into a pair of image data, comprising: YUV encoding is performed on the image, and the first sensor data and the image are combined into a pair of image data.
7. The image acquisition method according to claim 1, characterized in that: The first sensor comprises: The first sensor is an IMU sensor.
8. The image acquisition method according to claim 1, characterized in that: The second sensor comprises: The second sensor is a Sensor sensor.
9. An image acquisition device, characterized in that: include: A synchronization unit, used for performing clock synchronization on the first sensor and the second sensor; An acquisition unit, configured to trigger the second sensor to acquire an image by an interrupt signal from the first sensor; A sending unit is used for the main control chip to send the first sensor data, the image and the timestamp to the robot host.
10. The image acquisition device according to claim 9, characterized in that: Also includes: A packing unit is used for the main control chip to pack the first sensor data, the image and the timestamp, so that the main control chip sends the packed first sensor data, the image and the timestamp to the robot host.
11. The image acquisition device according to claim 9, characterized in that: Also includes: A merging unit is used to merge the first sensor data and the image into a set of image data.
12. The image acquisition device according to claim 9, characterized in that: Also includes: A storage unit is used to store the first sensor data, the image and time stamp information.
13. An image acquisition system, characterized in that: include: The first sensor, the second sensor, the main control chip and the robot host; The first sensor is connected to the second sensor and the main control chip respectively, and is used to trigger the second sensor to collect images; The second sensor is connected to the first sensor and the main control chip respectively, and is used to collect the image; The main control chip is connected to the first sensor and the second sensor respectively, and the main control chip is connected to the robot host for communication, and is used to send the first sensor data, the image and the timestamp to the robot host; The robot host is communicatively connected to the main control chip, and is used to receive and store the first sensor data, the image and the timestamp.
14. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.
15. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.