Line structured light camera and method with integrated point cloud acceleration module

Abstract

The present application relates to a line structured light camera and method with an integrated point cloud acceleration module, wherein the line structured light camera includes a processing chip and a point cloud acceleration module; the processing chip is used to extract the light bar center of the image acquired by the line structured light camera to obtain the light bar center sequence of the image; the processing chip is used to calculate the point cloud data of the light bar center sequence through the point cloud acceleration module according to the camera internal parameters, camera external parameters and the light bar center sequence. Through the present application, the processing chip and the point cloud acceleration module in the camera are used to migrate the calculation task of converting the light bar center to the point cloud from the external monitoring terminal to the camera, which can greatly reduce the overall processing time and cost of the system and solve the problem of how to reduce the measurement cost of the line structured light camera.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a line structured light camera and method with an integrated point cloud acceleration module. Background Art

[0002] Line structured light 3D cameras are based on the principle of triangulation. They measure the distance of objects by obtaining the center of the light strip in the camera coordinate system and the known light plane. This non-contact detection method improves the accuracy and efficiency of detection, while also reducing damage to the measured object and measurement costs. In general, as an efficient and accurate three-dimensional imaging technology, line structured light 3D cameras have shown great application potential in many fields.

[0003] With the application of large-scale FPGA chips, the technical route of collecting images and extracting light stripes through FPGA has become relatively mature. This method can greatly save network bandwidth, reduce CPU calculation time, and greatly improve the scanning frame rate of line structured light 3D cameras. However, the center of the light strip cannot be directly converted into a point cloud. It needs to go through a complex calculation process in the middle, which is generally processed through an external monitoring terminal, which is costly.

[0004] Currently, no effective solution has been proposed for the problem of how to reduce the measurement cost of line structured light cameras in related technologies. Summary of the invention

[0005] The embodiments of the present application provide a line structured light camera and method with an integrated point cloud acceleration module, so as to at least solve the problem of how to reduce the measurement cost of the line structured light camera in the related art.

[0006] In a first aspect, an embodiment of the present application provides a line structured light camera integrated with a point cloud acceleration module, wherein the line structured light camera includes a processing chip and a point cloud acceleration module;

[0007] The processing chip is used to extract the light stripe centers of the image acquired by the line structured light camera to obtain a light stripe center sequence of the image;

[0008] The processing chip is used to calculate the point cloud data of the light bar center sequence through the point cloud acceleration module according to the camera intrinsic parameters, the camera extrinsic parameters and the light bar center sequence.

[0009] In some embodiments, the point cloud data of the light bar center sequence calculated by the point cloud acceleration module according to the camera intrinsic parameters, the camera extrinsic parameters and the light bar center sequence includes:

[0010] The processing chip converts the pixel coordinates of the center of the light bar in the light bar center sequence into the pixel coordinates of the center of the light bar in the light bar center sequence according to the camera intrinsic parameters and the camera extrinsic parameters through the point cloud acceleration module. Convert to the world coordinate system to obtain the world coordinates of the center sequence of the light bar ;

[0011] The processing chip uses the point cloud acceleration module to calculate the world coordinates of the center sequence of the light strips. Resampling is performed to obtain point cloud data of the center sequence of the light strip.

[0012] In some embodiments, the processing chip converts the pixel coordinates of the center of the light bar in the light bar center sequence into the pixel coordinates of the center of the light bar in the light bar center sequence according to the camera intrinsic parameters and the camera extrinsic parameters through the point cloud acceleration module. Convert to the world coordinate system to obtain the world coordinates of the center sequence of the light bar include:

[0013] The processing chip calculates the pixel coordinates of the center of the light bar in the light bar center sequence through the point cloud acceleration module according to the camera internal parameters. Perform distortion removal;

[0014] The processing chip converts the pixel coordinates of the center of the dedistorted light strip through the point cloud acceleration module according to the camera internal parameters. Convert to the camera coordinate system to get the camera coordinates of the center of the light strip ;

[0015] The processing chip converts the camera coordinates of the center of the light strip into the camera coordinates of the center of the light strip through the point cloud acceleration module according to the camera external parameters. Convert to the world coordinate system to get the world coordinates of the center of the light bar .

[0016] In some embodiments, the processing chip calculates the pixel coordinates of the center of the light bar in the light bar center sequence through the point cloud acceleration module according to the camera internal parameters. Distortion removal includes:

[0017] The processing chip determines the point cloud acceleration module from the center sequence of the light strips. The pixel coordinates of the center of the light bar that are not 0 ;

[0018] The processing chip calculates the pixel coordinates of the center of the light bar through the point cloud acceleration module according to the camera internal parameters. Remove distortion.

[0019] In some embodiments, the processing chip uses the point cloud acceleration module to calculate the world coordinates of the center sequence of the light strips. Resampling is performed to obtain point cloud data of the light strip center sequence, including:

[0020] The processing chip uses the point cloud acceleration module to calculate the world coordinates of the center sequence of the light strips. Quantize to generate sampling point structure and perform FIFO buffering;

[0021] The processing chip reads the sampling point structure in the FIFO buffer through the point cloud acceleration module, and performs interpolation based on the sampling points in the sampling point structure to obtain point cloud data of the center sequence of the light strip.

[0022] In some embodiments, the processing chip reads the sampling point structure of the FIFO buffer through the point cloud acceleration module, and interpolates based on the sampling points in the sampling point structure to obtain the point cloud data of the center sequence of the light bar, including:

[0023] The processing chip reads the sampling point structure of the FIFO buffer through the point cloud acceleration module, wherein the clock cycle of the read port of the FIFO buffer is greater than the clock cycle of the write port;

[0024] If the index index of the currently read sampling point is greater than the index index+1 of the previous sampling point, the FIFO read enable is temporarily turned off to suspend reading, keep the two sampling points unchanged, and perform multiple interpolation calculations;

[0025] If the index index of the currently read sampling point is less than or equal to the index index of the previous sampling point, keep the FIFO read enable turned on to continue reading without performing interpolation calculation;

[0026] If the index index of the currently read sampling point is equal to the index index+1 of the previous sampling point, keep the FIFO read enable turned on to continue reading and perform a single interpolation calculation;

[0027] Through interpolation calculation, point cloud data of the center sequence of the light strip is obtained.

[0028] In some embodiments, the processing chip uses the point cloud acceleration module to calculate the world coordinates of the center sequence of the light strips. After resampling to obtain the point cloud data of the center sequence of the light strip, the method further includes:

[0029] The processing chip writes the point cloud data of the center sequence of the light strip into the port RAM;

[0030] The external monitoring terminal reads the data in the port RAM to obtain the point cloud data of the center sequence of the light bar.

[0031] In some of these embodiments, the port RAM includes a single-port RAM and a dual-port RAM.

[0032] In some of these embodiments, the line structured light camera further includes a memory;

[0033] The memory is used to store camera intrinsic parameters and camera extrinsic parameters of the line structured light camera;

[0034] The processing chip is used to obtain camera intrinsic parameters and camera extrinsic parameters from the memory, wherein the processing chip includes an FPGA processing chip and an ASIC processing chip.

[0035] In a second aspect, an embodiment of the present application provides a camera application method integrating a point cloud acceleration module, the method is executed based on the camera described in the first aspect, and the method includes:

[0036] Acquire an image through a line structured light camera and extract the center of light stripes to obtain a light stripe center sequence of the image; calculate point cloud data of the light stripe center sequence according to camera intrinsic parameters, camera extrinsic parameters and the light stripe center sequence;

[0037] The point cloud data of the center sequence of the light bar is transmitted to an external monitoring terminal.

[0038] Compared with the related art, the embodiment of the present application provides a line structured light camera and method with an integrated point cloud acceleration module, wherein the line structured light camera includes a processing chip and a point cloud acceleration module; the processing chip is used to extract the light stripe centers of the image acquired by the line structured light camera to obtain the light stripe center sequence of the image; the processing chip is used to calculate the point cloud data of the light stripe center sequence through the point cloud acceleration module according to the camera intrinsic parameters, the camera extrinsic parameters and the light stripe center sequence, and the processing chip and the point cloud acceleration module in the camera are used to migrate the calculation task of converting the light stripe center to the point cloud from the external monitoring terminal to the camera, which can greatly reduce the overall processing time and cost of the system and solve the problem of how to reduce the measurement cost of the line structured light camera. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0040] Figure 1 is a schematic diagram of the structural process of a line structured light camera according to an embodiment of the present application;

[0041] Figure 2 is a schematic diagram of scanning measurement performed by a line structured light camera according to an embodiment of the present application;

[0042] Figure 3 is a schematic diagram of resampling according to an embodiment of the present application;

[0043] Figure 4 is a comparative schematic diagram of a single-port RAM and a dual-port RAM according to an embodiment of the present application;

[0044] Figure 5 It is a schematic diagram of data reading by the port RAM according to an embodiment of the present application;

[0045] Figure 6 It is a schematic diagram of the internal structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0047] Obviously, the drawings described below are only some examples or embodiments of the present application. For ordinary technicians in this field, the present application can also be applied to other similar scenarios based on these drawings without creative work. In addition, it can also be understood that although the efforts made in this development process may be complicated and lengthy, for ordinary technicians in this field related to the content disclosed in this application, some changes in design, manufacturing or production based on the technical content disclosed in this application are just conventional technical means, and should not be understood as insufficient content disclosed in this application.

[0048] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0049] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantity limitation, and may indicate the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0050] The present application embodiment provides a line structured light camera with an integrated point cloud acceleration module. Figure 1 is a schematic diagram of the structure flow of a line structured light camera according to an embodiment of the present application, such as Figure 1 As shown, the line structured light camera includes a processing chip and a point cloud acceleration module, wherein the processing chip includes an FPGA processing chip and an ASIC processing chip;

[0051] A processing chip is used to extract the light stripe center of the image acquired by the line structured light camera to obtain the light stripe center sequence of the image;

[0052] It should be noted that the height of the original image obtained by the line structured light camera is preferably 416 pixels and the width is preferably 2048 pixels; the light stripe center extraction of each frame of the original image can obtain 2048 light stripe center sequences (preferably single-precision floating-point sequence, clock frequency is 200M, data width is 32 bits, and one sequence consumes 2048 clock cycles), where the sequence number can be mapped to the light stripe center Coordinates, the values ​​of the sequence correspond to the center of the light bar coordinate.

[0053] The processing chip is used to calculate the point cloud data of the light strip center sequence through the point cloud acceleration module according to the camera intrinsic parameters, camera extrinsic parameters and the light strip center sequence.

[0054] It should be noted that after receiving the above 2048 single-precision floating-point number sequences, the point cloud acceleration module performs a series of calculations and finally outputs 3200 single-precision floating-point number sequences (i.e. point cloud data), among which: Figure 2 is a schematic diagram of scanning measurement performed by a line structured light camera according to an embodiment of the present application, and the distribution of the XYZ axes of the world coordinate system is as follows Figure 2 As shown, the x value of the point cloud data is the sequence number of 3200 single-precision floating-point sequences, the y value of the point cloud data is the frame number of the 3200 single-precision floating-point sequences (i.e., the relative motion direction of the object in the image), and the z value of the point cloud data is the numerical value of the 3200 single-precision floating-point sequences (i.e., the vertical distance between the object surface in the image and the line structured light camera).

[0055] It should be further explained that the complete point cloud of the object surface can be restored through the point cloud data of several frames of images. In other words, it is only necessary to send the z value of the point cloud data to the external monitoring terminal in a certain order, and the external monitoring terminal can restore the complete point cloud of the object surface.

[0056] It should be further explained that if Figure 2 As shown, the line structured light camera also includes a memory for storing camera intrinsic parameters and camera extrinsic parameters of the line structured light camera; the processing chip obtains the camera intrinsic parameters and camera extrinsic parameters from the memory.

[0057] In some embodiments, the point cloud data of the light bar center sequence calculated by the point cloud acceleration module according to the camera intrinsic parameters, the camera extrinsic parameters and the light bar center sequence includes:

[0058] The processing chip converts the pixel coordinates of the center of the light strip in the light strip center sequence into the pixel coordinates of the center of the light strip according to the camera intrinsic parameters and camera extrinsic parameters through the point cloud acceleration module. Convert to the world coordinate system to get the world coordinates of the light bar center sequence ;

[0059] The processing chip uses the point cloud acceleration module to calculate the world coordinates of the light bar center sequence. Resample to obtain the point cloud data of the light bar center sequence.

[0060] It should be noted that for the world coordinates of the light bar center sequence , the number of sequences of each frame is 2048, and after resampling, only output The z value of each frame is 3200. The sequence is OK, because can all change uniformly, which can be inferred from the number of sequences and frames, as mentioned above Figure 2 shown. The y in each frame is 0.0 and can be ignored. As the sampling points of the interpolation function, the corresponding arithmetic sequence is generated The arithmetic progression (x[k]) is used as the input of the interpolation function. The interpolation function outputs is the resampled output. The arithmetic progression and the serial number can correspond one to one, so it can be omitted , only output .

[0061] In some embodiments, the processing chip converts the pixel coordinates of the center of the light strip in the light strip center sequence into the pixel coordinates of the center of the light strip according to the camera intrinsic parameters and the camera extrinsic parameters through the point cloud acceleration module. Convert to the world coordinate system to get the world coordinates of the light bar center sequence include:

[0062] The processing chip uses the point cloud acceleration module to calculate the pixel coordinates of the center of the light strip in the light strip center sequence according to the camera internal parameters. Perform distortion removal;

[0063] Preferably, the processing chip determines the point cloud acceleration module from the light bar center sequence. The pixel coordinates of the center of the light bar that are not 0 ; The processing chip uses the point cloud acceleration module to calculate the pixel coordinates of the center of the light strip according to the camera internal parameters Remove distortion.

[0064] It should be noted that when the distortion When it is equal to 0, it means that there is no corresponding light bar at this point, then the signal zero_point is assigned to "1", otherwise it is "0", zero_point enters the shift register and keeps synchronization with the dedistorted v. When zero_point is detected as "1" during output, the final v is assigned to 0.0.

[0065] It should be further explained that the pixel coordinates of the center of the light bar To remove distortion, it is necessary to first obtain the camera intrinsic parameters in the memory. The general process is as follows: normalization, removal of Sham distortion, removal of radial distortion, removal of tangential distortion, and inverse normalization. This embodiment optimizes the inverse normalization process to reduce hardware resource consumption. The original formula for inverse normalization is as follows:

[0066]

[0067] Since the calculation of multiplying by 0 in the processing chip is not automatically optimized and will occupy a multiplier, in order to reduce the consumption of hardware resources, this embodiment modifies the above calculation into:

[0068] u=x*fx+u0

[0069] v=y*fy+v0

[0070] The processing chip uses the point cloud acceleration module to convert the pixel coordinates of the center of the dedistorted light strip according to the camera internal parameters. Convert to the camera coordinate system to get the camera coordinates of the center of the light strip ;

[0071] Preferably, the pixel coordinates are converted to Convert to the coordinates in the normalized camera coordinate system The intrinsic parameter matrix usually includes the focal length , principal point offset u0, v0 and lens distortion parameters. The conversion formula is as follows:

[0072]

[0073] The processing chip converts the camera coordinates of the center of the light strip into Convert to the world coordinate system to get the world coordinates of the center of the light bar .

[0074] Preferably, the coordinates in the normalized camera coordinate system are converted to coordinates in the world coordinate system. This requires the use of camera extrinsics, including the camera's position and orientation information. The extrinsic matrix usually includes the camera's rotation matrix and translation vectors The conversion formula is as follows:

[0075]

[0076] It should be noted that through the above two conversions, the pixel coordinates can be Convert to the coordinates in the camera coordinate system , and then further converted to coordinates in the world coordinate system In this embodiment, the calculation process is optimized. When the input v is detected to be 0, it means that there is no corresponding light bar at this point, then the signal zero_point is assigned a value of "1", otherwise it is "0", and zero_point enters the shift register and after the conversion is completed Keep synchronization. When zero_point is detected as "1" during output, the final Assign the value to [0,0,0].

[0077] In some embodiments, the processing chip uses a point cloud acceleration module to calculate the world coordinates of the light bar center sequence. Resampling is performed to obtain the point cloud data of the light strip center sequence, including:

[0078] The processing chip uses the point cloud acceleration module to calculate the world coordinates of the center sequence of the light strip Quantize to generate sampling point structure and perform FIFO buffering;

[0079] It should be noted that Figure 3 is a schematic diagram of resampling according to an embodiment of the present application, such as Figure 3 As shown, the world coordinates of the light bar center sequence The range of the x value is roughly -40~+40mm (millimeter), and the width is 80mm (millimeter). However, the x values ​​are not evenly spaced at this time, usually dense in the middle and sparse on both sides, so resampling is required. Figure 3 In 301, the x value is quantized (linearly transformed) to stretch its range to 0.0~3199.0. The formula is as follows:

[0080] x = 3199 * (x - (-40)) / 80

[0081] Then you need to generate the corresponding index. The pseudo code is as follows:

[0082] index=int(x); when (0.0≤x≤3199.0 && z>0.0)

[0083] index=-1; when(0.0>x>3199.0 || z≤0.0)

[0084] like Figure 3 As shown in 302, the above data are concatenated to generate a sampling point structure, and the structure is arranged as follows:

[0085] | index[12:0] | x[31:0] | z[31:0] | last[0:0] |

[0086] like Figure 3 As shown in 303, writing is performed through the FIFO write port. Figure 3 In step 304, the FIFO is read.

[0087] The processing chip reads the sampling point structure in the FIFO buffer through the point cloud acceleration module, and interpolates based on the sampling points in the sampling point structure to obtain the point cloud data of the center sequence of the light strip.

[0088] Preferably, the processing chip reads the sampling point structure of the FIFO cache through the point cloud acceleration module, wherein the clock cycle of the read port of the FIFO cache is greater than the clock cycle of the write port;

[0089] If the index index of the currently read sampling point is greater than the index index+1 of the previous sampling point, the FIFO read enable is temporarily turned off to suspend reading, keep the two sampling points unchanged, and perform multiple interpolation calculations;

[0090] If the index index of the currently read sampling point is less than or equal to the index index of the previous sampling point, keep the FIFO read enable turned on to continue reading without performing interpolation calculation;

[0091] If the index index of the currently read sampling point is equal to the index index+1 of the previous sampling point, keep the FIFO read enable turned on to continue reading and perform a single interpolation calculation;

[0092] Through interpolation calculation, the point cloud data of the light strip center sequence is obtained.

[0093] It should be noted that the sampling point structure is read at the FIFO read port. The read enable of the FIFO is controlled according to the index. Since the FIFO write port has 2048 clock cycles, but the FIFO read port has 3200 clock cycles, the interpolated points are more than the sampling points, so upsampling is required. Therefore, when reading, if a sampling point with a wide interval is encountered, the reading must be paused, and multiple interpolation results are generated using 2 sampling points. The specific process is as follows:

[0094] Current index>previous index+1: Pause reading, temporarily turn off FIFO read enable, keep the two sampling points unchanged, and perform multiple interpolation calculations;

[0095] Current index ≤ previous index: continue reading, keep FIFO read enable on, do not interpolate, and directly output 0xFFFFFFFF;

[0096] Current index = previous index + 1: continue reading, keep FIFO read enable turned on, and perform single interpolation calculation;

[0097] Figure 3 305 is interpolated, and the interpolation formula is shown in the figure:

[0098] k = (current z - previous z) / (current x - previous x)

[0099] z=k*(float(current index)-previous x)+previous[z]

[0100] Figure 3306 is used to store the interpolated z value. The initial value in the RAM is 0xFFFFFFFF. In this embodiment, 0xFFFFFFFF represents that the z value is a null point, because a single-precision floating point number cannot be 0xFFFFFFFF. When the FIFO read port detects that last is "1", it means that the data of a frame has been interpolated, and then the RAM is read.

[0101] In some embodiments, the processing chip uses a point cloud acceleration module to calculate the world coordinates of the light bar center sequence. After resampling to obtain the point cloud data of the light strip center sequence, it also includes:

[0102] The processing chip writes the point cloud data of the center sequence of the light bar into the port RAM; the external monitoring terminal reads the data in the port RAM to obtain the point cloud data of the center sequence of the light bar. Among them, the port RAM includes a single-port RAM and a dual-port RAM.

[0103] It should be noted that if Figure 3 As shown in 307, two data are read from the dual-port RAM at the same time, which takes 1600 clock cycles in the embodiment, and finally 3200 z can be read in one frame.

[0104] Figure 4 This is a comparison diagram of a single-port RAM and a dual-port RAM according to an embodiment of the present application. Because the RAM must be read completely before the next frame of data can be processed in this embodiment, the faster the RAM reading speed is, the better. When reading in this embodiment, the RAM data can be written to 0xFFFFFFFF at the same time, and no separate initialization operation is required. Figure 5 Schematic diagram of data reading by the port RAM according to the embodiment of the present application, and the specific operation is as follows Figure 5 As shown, the read and write mode of the RAM must be changed to readfirst mode.

[0105] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0106] Through the above embodiments, 1. The calculation task of converting the center of the light bar to the point cloud is migrated from the external monitoring terminal to the processing chip of the camera, which can greatly reduce the overall processing time of the system. 2. By adopting the split pipeline + dual-port RAM method, a light bar sequence of 2048 length can be continuously input. After processing one frame sequence, the next frame light bar sequence can be processed immediately after an interval of 0 to 2 cycles. 3. The FIFO cache sampling point structure is used to perform interpolation calculations for upsampling, and the sampling points are classified (according to different categories, in turn: multiple interpolation, single interpolation, no interpolation), to improve the efficiency of interpolation calculations.

[0107] The embodiment of the present application provides a camera application method integrating a point cloud acceleration module. The method is executed based on the camera provided by the above embodiment. The method includes:

[0108] The image is acquired through a line structured light camera and the light stripe center is extracted to obtain the light stripe center sequence of the image; the point cloud data of the light stripe center sequence is calculated based on the camera intrinsic parameters, camera extrinsic parameters and the light stripe center sequence;

[0109] Transmit the point cloud data of the light bar center sequence to an external monitoring terminal.

[0110] This embodiment further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0111] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0112] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0113] In addition, in combination with the method in the above embodiment, the embodiment of the present application can provide a storage medium for implementation. The storage medium stores a computer program; when the computer program is executed by a processor, any one of the methods in the above embodiment is implemented.

[0114] In one embodiment, a computer device is provided, which may be a terminal. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method is implemented. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a key, trackball, or touchpad provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0115] In one embodiment, Figure 6 is a schematic diagram of the internal structure of an electronic device according to an embodiment of the present application, such as Figure 6 As shown, an electronic device is provided, which may be a server, and its internal structure diagram may be as shown in Figure 6 As shown. The electronic device includes a processor, a network interface, an internal memory and a non-volatile memory connected through an internal bus, wherein the non-volatile memory stores an operating system, a computer program and a database. The processor is used to provide computing and control capabilities, the network interface is used to communicate with an external terminal through a network connection, the internal memory is used to provide an environment for the operation of the operating system and the computer program, the computer program is executed by the processor to implement a method, and the database is used to store data.

[0116] Those skilled in the art will understand that Figure 6 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0117] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0118] Those skilled in the art should understand that the technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0119] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A line structured light camera with integrated point cloud acceleration module, characterized in that: The line structured light camera includes a processing chip and a point cloud acceleration module; The processing chip is used to extract the light stripe centers of the image acquired by the line structured light camera to obtain a light stripe center sequence of the image; The processing chip is used to convert the pixel coordinates of the center of the light bar in the light bar center sequence into the pixel coordinates of the center of the light bar according to the camera intrinsic parameters and the camera extrinsic parameters through the point cloud acceleration module. Convert to the world coordinate system to obtain the world coordinates of the center sequence of the light bar ; The processing chip uses the point cloud acceleration module to calculate the world coordinates of the center sequence of the light strips. Quantize to generate sampling point structure and perform FIFO buffering; The processing chip reads the sampling point structure of the FIFO buffer through the point cloud acceleration module, wherein the clock cycle of the read port of the FIFO buffer is greater than the clock cycle of the write port; If the index index of the currently read sampling point is greater than the index index+1 of the previous sampling point, the FIFO read enable is temporarily turned off to suspend reading, keep the two sampling points unchanged, and perform multiple interpolation calculations; If the index index of the currently read sampling point is less than or equal to the index index of the previous sampling point, keep the FIFO read enable turned on to continue reading without performing interpolation calculation; If the index index of the currently read sampling point is equal to the index index+1 of the previous sampling point, keep the FIFO read enable turned on to continue reading and perform a single interpolation calculation; Through interpolation calculation, point cloud data of the center sequence of the light strip is obtained.

2. The camera according to claim 1, characterized in that The processing chip converts the pixel coordinates of the center of the light bar in the light bar center sequence into the pixel coordinates of the center of the light bar in the light bar center sequence according to the camera intrinsic parameters and the camera extrinsic parameters through the point cloud acceleration module. Convert to the world coordinate system to obtain the world coordinates of the center sequence of the light bar include: The processing chip calculates the pixel coordinates of the center of the light bar in the light bar center sequence through the point cloud acceleration module according to the camera internal parameters. Perform distortion removal; The processing chip converts the pixel coordinates of the center of the dedistorted light strip through the point cloud acceleration module according to the camera internal parameters. Convert to the camera coordinate system to get the camera coordinates of the center of the light strip ; The processing chip converts the camera coordinates of the center of the light strip into the camera coordinates of the center of the light strip through the point cloud acceleration module according to the camera external parameters. Convert to the world coordinate system to get the world coordinates of the center of the light bar .

3. The camera according to claim 2, characterized in that The processing chip calculates the pixel coordinates of the center of the light bar in the light bar center sequence through the point cloud acceleration module according to the camera internal parameters. Distortion removal includes: The processing chip determines the point cloud acceleration module from the center sequence of the light strips. The pixel coordinates of the center of the light bar that are not 0 ; The processing chip calculates the pixel coordinates of the center of the light bar through the point cloud acceleration module according to the camera internal parameters. Remove distortion.

4. The camera according to claim 1, characterized in that The processing chip uses the point cloud acceleration module to calculate the world coordinates of the center sequence of the light strips. After resampling to obtain the point cloud data of the center sequence of the light strip, the method further includes: The processing chip writes the point cloud data of the center sequence of the light strip into the port RAM; The external monitoring terminal reads the data in the port RAM to obtain the point cloud data of the center sequence of the light bar.

5. The camera according to claim 4, characterized in that The port RAM includes a single-port RAM and a dual-port RAM.

6. The camera according to claim 1, characterized in that The line structured light camera also includes a memory; The memory is used to store camera intrinsic parameters and camera extrinsic parameters of the line structured light camera; The processing chip is used to obtain camera intrinsic parameters and camera extrinsic parameters from the memory, wherein the processing chip includes an FPGA processing chip and an ASIC processing chip.

7. A camera application method integrating a point cloud acceleration module, characterized in that: The method is performed based on the camera according to any one of claims 1 to 6, and the method comprises: Acquire an image through a line structured light camera and extract the center of light stripes to obtain a light stripe center sequence of the image; calculate point cloud data of the light stripe center sequence according to camera intrinsic parameters, camera extrinsic parameters and the light stripe center sequence; The point cloud data of the center sequence of the light bar is transmitted to an external monitoring terminal.

Images