An industrial intelligent camera and its working method

Abstract

The present invention discloses an industrial intelligent camera and its working method, belonging to the field of industrial digital image sensing. Image preprocessing and real-time optical axis control are implemented in the FPGA system; in the GPU+ARM system, the GPU realizes image wavefront extraction and image recognition; the ARM performs beam quality control and configures the grid for the FPGA system to achieve management functions such as input / output display and data storage. An open-source Linux system is embedded in the ARM, and all control algorithms are open to users. Therefore, as long as different user codes are embedded, the present invention can realize some intelligent industrial controls based on image recognition in addition to adaptive optical control. By integrating the front-end FPGA preprocessing and optical axis control system and the back-end beam quality control system into a single camera unit, the present invention realizes an integrated camera for image acquisition, image recognition, and high-speed real-time control.

Description

Technical Field

[0001] The present invention relates to the field of industrial digital image sensing, and more specifically, to an industrial intelligent camera and its working method. Background Art

[0002] An industrial intelligent camera is an industrial digital image sensing device that integrates a series of functions such as high-speed image acquisition, image intelligent recognition, and some intelligent control algorithms. Laser beam quality control and optical stability control are achieved by means of adaptive optics technology. The present invention provides an industrial intelligent camera dedicated to adaptive optics to fill the gap. Summary of the Invention

[0003] The purpose of the present invention is to overcome the deficiencies of the prior art and provide an industrial intelligent camera and its working method, filling the gap in the industry.

[0004] The purpose of the present invention is achieved through the following solutions:

[0005] An industrial intelligent camera includes an image acquisition system, an image preprocessing system, and a core processing system; the image acquisition system is used to acquire an image and then transfer it to the image preprocessing system; the image preprocessing system is provided with an image spot centroid coordinate extraction module for transferring the extracted image spot centroid coordinates and the original image to the core processing system. At the same time, the image preprocessing system is provided with an optical axis control module for calculating the optical axis adjustment amount and controlling the optical axis through control output; the core processing system is a dual-processor architecture, where the first processor is used for image wavefront extraction and image recognition, and the second processor is used for beam quality control and configuring the image preprocessing system; the image acquisition system, the image preprocessing system, and the core processing system are all integrated in one camera.

[0006] Further, it includes a far-field image acquisition system. An aperture lens is provided in the far-field image acquisition system to acquire a target recognition far-field image by using the aperture lens; for the original image transferred from the far-field image acquisition system to the image preprocessing system, the image preprocessing system does not perform any calculation and only transfers the original image to the core processing system.

[0007] Further, the near-field image acquisition system includes a photosensitive surface and an interface circuit board. The photosensitive surface is connected to the interface circuit board, and the interface circuit board includes a high-speed interface board based on a differential conversion circuit for transmitting photosensitive electrical signals in parallel.

[0008] Further, the second processor embeds an open-source Linux system and opens all control algorithms to users. Users can achieve different industrial control capabilities based on image recognition by embedding different user codes.

[0009] Further, the image preprocessing system includes an image reading circuit, an FPGA processor, an optical axis control RS422 port driving circuit, and a PCIe x4 transmission interface driver;

[0010] The image reading circuit is used to transfer the differential signals transmitted by the image acquisition system to the original image buffer in the FPGA processor in real time, and the image reading circuit includes a differential parallel port image reading circuit;

[0011] The FPGA processor extracts the centroid coordinates of the image, adds a control algorithm to the centroid coordinates, calculates the optical axis adjustment amount using the control algorithm, and then drives the RS422 port output through the optical axis control RS422 port driving circuit to control the optical axis;

[0012] The PCIe x4 transmission interface driver is used to package and transmit the centroid coordinate array and the original image preprocessed by the FPGA processor to the PCIe x4 interface.

[0013] Further, the core processing system includes a circuit system interface board, a solid-state drive, and a first network port driving circuit. The first processor is a GPU processor, and the second processor is an ARM processor. The GPU processor and the ARM processor are integrated into an overall chip to form a GPU+ARM core board;

[0014] The circuit system interface board is used for the conversion and driving of multiple interfaces. The multiple interfaces include a PCIe x4 interface from the FPGA system to the GPU+ARM core board, a PCIE interface between the GPU+ARM core board and the solid-state drive, and network ports, USB interfaces, HDMI interfaces, CAN interfaces, and RS422 interfaces external to the GPU+ARM core board;

[0015] The solid-state drive is used to install the embedded linux operating system, install user programs APP, and store data files;

[0016] The first network port driving circuit is used to output the control amount calculated by the GPU+ARM core board through the network port to control the deformable mirror and thus control the beam quality; the network port includes a high-speed gigabit network port.

[0017] Further, the image preprocessing system includes a second network port driving circuit and a high-speed driving fiber interface circuit, both of which are used to control the function switch.

[0018] Further, the core processing system includes a USB and HDMI interface driving circuit, a CAN and RS422 interface driving circuit;

[0019] The USB and HDMI interface driving circuits are used to drive the USB and HDMI interfaces; the USB interface includes a Mini-USB interface and a USB-Type-C interface, both of which are used for external data input or output; the HDMI provides an external display interface, which is a digital high-definition interface for monitoring.

[0020] The CAN and RS422 interface drivers are used to drive the CAN interface and the RS422 interface, and provide an external multi-functional output interface channel; the multi-functional output interface channel includes an optical axis control output channel based on the RS422 interface and a camera cascading control channel based on the CAN.

[0021] A working method of an industrial intelligent camera includes the steps:

[0022] After starting the industrial intelligent camera described in any one of the above, start the linux system in the GPU+ARM core module and start the user APP; the user APP receives the grid coordinate configuration parameters sent by the upper computer and the configuration parameters related to the optical system through the network port. At the same time, the APP sends the grid configuration parameters to the FPGA system through the PCIEx4 interface for preprocessing by the FPGA system.

[0023] The original image read from the photosensitive surface by the image readout circuit of the FPGA system is stored in the image buffer; after receiving the original image, the FPGA system performs the following real-time processing:

[0024] S1, divide the area according to the configured grid coordinate parameters, calculate the centroid coordinates of the array, including the grid coordinates of the large area for controlling the optical axis and the centroid coordinate array of the two-dimensional grid area for wavefront reconstruction.

[0025] S2, add the centroid coordinates extracted from the large area to the PID control calculation and send out through the RS422 port to control the stability of the optical axis.

[0026] S3, pack the centroid coordinate array extracted from the two-dimensional grid area and the original image and transfer them to the GPU+ARM core board through the PCIEx4 port.

[0027] After receiving the data, the GPU+ARM core board performs the following processing:

[0028] S4, in the GPU processor, reconstruct the wavefront with the centroid coordinates and the grid coordinate configuration parameters, and solve the voltage with the configured response matrix; in the GPU processor, perform deep learning calculation with the original image, train the mode coefficients of the wavefront, and reconstruct the wavefront, and also solve the voltage with the configured response matrix.

[0029] S5. In the ARM processor, the calculated wavefront and voltage are compared, and one of them is preferentially selected for PID control calculation to obtain the final control voltage. The control voltage is sent out through the gigabit Ethernet port to control the two-dimensional deformable mirror, thereby realizing the control of the beam quality. In the ARM processor, the wavefront, control voltage, and other states are displayed through the HDMI port. Thus, the optical stability control and beam quality control are achieved.

[0030] A working method of an industrial intelligent camera, including the above-mentioned industrial intelligent camera, comprising the steps of:

[0031] Step 1. After the original image from the photosensitive surface reaches the image preprocessing system, the FPGA in the image preprocessing system does not perform any calculations, but only sends the original image to the GPU + ARM core board through the PCIe x4 port.

[0032] Step 2. In the GPU + ARM core board, the GPU processor uses algorithms to learn the original image, train, and identify various target features.

[0033] Step 3. The ARM processor sends the recognized target results to the display for display through the HDMI interface, and at the same time sends the recognition results through the RS422 port and CAN port to control different industrial targets.

[0034] The beneficial effects of the present invention include:

[0035] The present invention provides an industrial intelligent camera dedicated to adaptive optics and its working method, filling the gap in the industry. The high-speed image sensing and adaptive optics technologies are highly integrated into an intelligent camera as a whole, and it becomes a general industrial measurement and control application platform from sensing to control, providing a very efficient industrial digital image sensing technology solution.

[0036] The embodiments of the present invention are suitable for the optical axis control and beam quality control of the adaptive optics system. Image preprocessing and real-time optical axis control are implemented in the FPGA system; in the GPU + ARM system, the GPU realizes image wavefront extraction and image recognition; the ARM performs beam quality control and configures the grid for the FPGA system to realize management functions such as input / output display and data storage. At the same time, since the ARM embeds an open Linux system and opens all control algorithms to users, as long as different user codes are embedded, the present invention can realize some intelligent industrial controls based on image recognition in addition to adaptive optics control, such as face recognition and process control.

[0037] The embodiments of the present invention integrate the front-end FPGA preprocessing and optical axis control system and the back-end beam quality control system into a camera as a whole, realizing an integrated camera system for image acquisition, image recognition, and high-speed real-time control. Description of the Drawings

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can also be obtained based on these drawings.

[0039] Figure 1 It is a schematic diagram of the composition of the industrial intelligent camera circuit system;

[0040] Figure 2 It is a three-dimensional design drawing of the industrial intelligent camera;

[0041] Figure 3 It is a three-dimensional design drawing of the industrial intelligent camera with an aperture lens added. Specific embodiments

[0042] The following further illustrates the invention in conjunction with the accompanying drawings and embodiments. All features disclosed in all embodiments in this specification, or all steps in any method or process implicitly disclosed, except for mutually exclusive features and / or steps, can be combined and / or extended and replaced in any manner.

[0043] The present invention aims to solve the problems of realizing beam quality control and optical stability control in intelligent cameras, and provides an industrial intelligent camera and its working method. In the specific embodiments, the circuit system of the industrial intelligent camera is as Figure 1 shown Figure 2 which is its three-dimensional design drawing.

[0044] In the embodiment, in terms of circuit structure, near-field image acquisition, FPGA image preprocessing, and GPU+ARM image recognition and control algorithms can be used to form three systems, and all three systems are integrated into one camera. It includes: an original image acquisition system (including a photosensitive surface and a photosensitive surface substrate). An FPGA system (including an FPGA processor, a differential parallel port image readout circuit, an optical axis control RS422 port drive circuit, a network port drive circuit, a high-speed drive fiber optic interface circuit, and a PCIEx4 transmission interface drive). A GPU+ARM system (including a GPU+ARM core module, a circuit system interface substrate, a solid-state drive, a network port drive circuit, USB and HDMI interface drives, CAN and RS422 interface drives), which are described in detail as follows:

[0045] The original image acquisition system is the input information source system for the original image, and the system is composed of a light surface and a photosensitive surface substrate.

[0046] The photosensitive surface can use the processing chip NOIP1FN5000A, with a resolution of 2592x2048 and a full frame rate of 85HZ.

[0047] The photosensitive surface substrate adopts a high-speed interface board based on a differential conversion circuit to transmit the photosensitive electrical signal in parallel.

[0048] The FPGA system is used as the front-end preprocessing system and realizes the PID control of the optical axis. Its hardware includes FPGA processor, differential parallel port image readout circuit, optical axis control RS422 port drive circuit, network port drive circuit and high-speed drive optical fiber interface circuit, and PCIEx4 transmission interface driver.

[0049] The FPGA processor chip may be XC7K325T or other models with equivalent functions.

[0050] The parallel port image readout circuit transfers the differential signal transmitted from the front-end image acquisition system to the original image buffer in the FPGA in real time.

[0051] The optical axis controls the RS422 port drive circuit. The FPGA extracts the centroid coordinates through the image, adds a control algorithm such as PID to the centroid coordinates, calculates the optical axis adjustment amount, and then outputs it through the RS422 port to achieve the purpose of controlling the optical axis.

[0052] The network port drive circuit and the high-speed drive optical fiber interface circuit are used to control some functional switches that affect the target, such as switch attenuators, voice coil motors, etc.

[0053] The PCIEx4 transmission interface driver is used to package and transmit the centroid coordinate array and original image preprocessed by FPGA to the PCIEx4 interface.

[0054] The GPU+ARM system is the core algorithm processing center of the embodiment of the present invention, including a GPU+ARM core module, a circuit system interface substrate, a solid-state hard disk, a network port driver circuit, USB and HDMI interface drivers, and CAN and RS422 interface drivers.

[0055] GPU+ARM core module, GPU and ARM are integrated into an integral chip. It is a circuit board based on embedded GPU+ARM chip and adopts Jetson Xavier NX module.

[0056] The circuit system interface substrate is responsible for the conversion and driving of various interfaces, including the PCIEx4 interface from the FPGA system to the GPU+ARM core board, the PCIE interface between the GPU+ARM core board and the solid-state drive, and the interface drivers of the GPU+ARM core board's external network port, USB, HDMI, CAN and RS422.

[0057] Solid-state drive, used for installing the embedded Linux operating system, as well as installing user programs APP, storing data files, etc., with a hard disk larger than 64GB.

[0058] Network interface driver circuit, which outputs the control quantity calculated by GPU + ARM through complex algorithms through the network interface to control the deformable mirror and thus control the beam quality. The network interface is a high-speed gigabit network interface.

[0059] USB and HDMI interface drivers. There are two types of USB interfaces, MiniUSB and USB-Type-C, for external data input or output. HDMI provides an external display interface, which is a digital high-definition interface for monitoring.

[0060] CAN and RS422 interface drivers, which provide multi-functional output interface channels externally, such as the optical axis control output based on the RS422 interface (another channel for optical axis control), and the camera cascade control based on CAN, etc.

[0061] In the specific implementation manner, when implemented as the working method of the optical axis stability control and beam quality control in adaptive optics, the following steps are included:

[0062] Start the camera system, start the Linux system in the GPU + ARM core module, and start the APP software developed by the user. The APP receives the grid coordinate configuration parameters and the configuration parameters related to the optical system sent by the host computer through the network interface. At the same time, the APP sends the grid configuration parameters to the FPGA system through the PCIe x4 interface for preprocessing by the FPGA system.

[0063] The original image read by the readout circuit of the FPGA system from the photosensitive surface is stored in the image buffer. After receiving the original image, the FPGA system performs the following three tasks in real time:

[0064] 1) Divide the region according to the configured grid coordinate parameters, and calculate the centroid coordinates of the array, including the grid coordinates of the large region for controlling the optical axis and the centroid coordinate array of the two-dimensional grid region for wavefront reconstruction.

[0065] 2) Add the centroid coordinates extracted from the large region to the PID control calculation, and send out the control to stabilize the optical axis through the RS422 port.

[0066] 3) Package the centroid coordinate array extracted from the two-dimensional grid region and the original image, and send them to the GPU + ARM core module through the PCIe x4 port.

[0067] After receiving the data, the GPU + ARM core module performs the following processing, including steps:

[0068] 1) In the GPU, reconstruct the wavefront from the column centroid coordinates and the previously issued configuration matrix, and calculate the voltage using the configured response matrix.

[0069] 2) Also in the GPU, perform deep learning calculations using the original image, train and verify the mode coefficients of the wavefront, reconstruct the wavefront, and calculate the voltage using the configured response matrix as well.

[0070] 3) In the ARM, compare the wavefronts and voltages calculated in 1) and 2), preferentially select one for PID control calculation to obtain the final control voltage, and send the control voltage through the gigabit Ethernet port to control the two-dimensional deformable mirror to achieve control of the beam quality.

[0071] 4) Also in the ARM, display the wavefront, control voltage, and other states through the HDMI port. Thus, the optical stability control and beam quality control are achieved.

[0072] In the specific implementation manner, as an intelligent recognition system, the present invention adds an aperture lens at the front end, that is, it constitutes a single product solution integrating detection, acquisition, deep learning algorithm inference, and target recognition. As Figure 3 shown, when implemented as the working method of intelligent recognition, it includes the following steps:

[0073] For target recognition, use the far-field image. After the original image from the photosensitive surface reaches the FPGA system, the FPGA does not perform any calculations, but only sends the original image to the GPU+ARM core module through the PCIEx4 port.

[0074] In the GPU+ARM core module, the GPU performs algorithms such as deep learning and intensity learning on the original image, trains and verifies various target features.

[0075] The ARM sends the recognized target results to the display through the HDMI interface, and at the same time sends the recognition results through the RS422 port and CAN port to control different industrial targets.

[0076] Embodiment 1: An industrial intelligent camera includes an image acquisition system, an image preprocessing system, and a core processing system; the image acquisition system is used to collect an image and transfer it to the image preprocessing system; the image preprocessing system is provided with an image spot centroid coordinate extraction module for transferring the extracted image spot centroid coordinates and the original image to the core processing system. At the same time, the image preprocessing system is provided with an optical axis control module for calculating the optical axis adjustment amount and controlling the optical axis through the control output; the core processing system has a dual-processor architecture, where the first processor is used for image wavefront extraction and image recognition, and the second processor is used for beam quality control and configuring the image preprocessing system; the image acquisition system, the image preprocessing system, and the core processing system are all integrated in one camera.

[0077] Example 2: On the basis of Example 1, it includes a far-field image acquisition system. An aperture lens is provided in the far-field image acquisition system, and the aperture lens is used to acquire the far-field image for target recognition. For the original image transmitted from the far-field image acquisition system to the image preprocessing system, the image preprocessing system does not perform any calculations and only transmits the original image to the core processing system.

[0078] Example 3: On the basis of Example 1, the near-field image acquisition system includes a photosensitive surface and an interface circuit board. The photosensitive surface is connected to the interface circuit board, and the interface circuit board includes a high-speed interface board based on a differential conversion circuit for transmitting the photosensitive electrical signal in parallel.

[0079] Example 4: On the basis of Example 1, the second processor embeds an open Linux system and opens all control algorithms to users. Users can embed different user codes to achieve different industrial control capabilities based on image recognition.

[0080] Example 5: On the basis of Example 1, the image preprocessing system includes an image readout circuit, an FPGA processor, an optical axis control RS422 port driver circuit, and a PCIe x4 transmission interface driver;

[0081] The image readout circuit is used to transfer the differential signal transmitted from the image acquisition system to the original image buffer in the FPGA processor in real time, and the image readout circuit includes a differential parallel port image readout circuit;

[0082] The FPGA processor extracts the centroid coordinates of the image, adds a control algorithm to the centroid coordinates, calculates the optical axis adjustment amount using the control algorithm, and then drives the RS422 port output through the optical axis control RS422 port driver circuit to control the optical axis;

[0083] The PCIe x4 transmission interface driver is used to package and transmit the centroid coordinate array and the original image preprocessed by the FPGA processor to the PCIe x4 interface.

[0084] Example 6: On the basis of Example 1, the core processing system includes a circuit system interface board, a solid-state drive, and a first network port driver circuit. The first processor is a GPU processor, and the second processor is an ARM processor. The GPU processor and the ARM processor are integrated into a single integrated chip to form a GPU + ARM core board;

[0085] A circuit system interface substrate for the conversion and driving of multiple interfaces, including a PCIe x4 interface from an FPGA system to a GPU + ARM core board, a PCIe interface between the GPU + ARM core board and a solid-state drive, as well as an Ethernet port, a USB interface, an HDMI interface, a CAN interface, and an RS422 interface external to the GPU + ARM core board;

[0086] A solid-state drive for installing an embedded Linux operating system, installing user programs APP, and storing data files;

[0087] A first Ethernet port driving circuit for outputting the control quantity calculated by the GPU + ARM core board through the control algorithm through the Ethernet port to control the deformable mirror and thus control the beam quality; the Ethernet port includes a high-speed gigabit Ethernet port.

[0088] Example 7: On the basis of Example 5, the image preprocessing system includes a second Ethernet port driving circuit and a high-speed driving fiber optic interface circuit, both of which are used to control function switches.

[0089] Example 8: On the basis of Example 6, the core processing system includes a USB and HDMI interface driving circuit, and a CAN and RS422 interface driving circuit;

[0090] The USB and HDMI interface driving circuit is used to drive the USB and HDMI interfaces; the USB interface includes a Mini-USB interface and a USB-Type-C interface, both of which are used for external data input or output; the HDMI provides an external display interface, which is a digital high-definition interface for monitoring;

[0091] The CAN and RS422 interface driving is used to drive the CAN interface and the RS422 interface, providing an external multi-functional output interface channel; the multi-functional output interface channel includes an optical axis control output channel based on the RS422 interface and a camera cascading control channel based on CAN.

[0092] Example 9: A working method of an industrial intelligent camera, including the steps:

[0093] After starting the industrial intelligent camera described in any one of Examples 1, 3 to 8, start the Linux system in the GPU + ARM core module and start the user APP; the user APP receives the grid coordinate configuration parameters and the configuration parameters related to the optical system sent by the host computer through the Ethernet port, and at the same time, the APP sends the grid configuration parameters to the FPGA system through the PCIe x4 interface for preprocessing by the FPGA system;

[0094] The original image read out from the photosensitive surface by the image readout circuit of the FPGA system is stored in the image buffer; after receiving the original image, the FPGA system performs the following real-time processing:

[0095] S1. Divide the region according to the configured grid coordinate parameters, and calculate the centroid coordinates of the array, including the large-region grid coordinates for controlling the optical axis and the centroid coordinate array of the two-dimensional grid region for wavefront reconstruction;

[0096] S2. Add the centroid coordinates extracted from the large region to the PID control calculation, and send out through the RS422 port to control the stability of the optical axis;

[0097] S3. Package the centroid coordinate array extracted from the two-dimensional grid region and the original image, and transfer them to the GPU+ARM core board through the PCIEx4 port;

[0098] After receiving the data, the GPU+ARM core board performs the following processing:

[0099] S4. In the GPU processor, reconstruct the wavefront with the centroid coordinates and the grid coordinate configuration parameters, and solve the voltage with the configured response matrix; in the GPU processor, perform deep learning calculation with the original image, train the mode coefficients of the wavefront, and reconstruct the wavefront, and also solve the voltage with the configured response matrix;

[0100] S5. In the ARM processor, compare the solved wavefront and voltage, select one of them for PID control calculation to obtain the final control voltage, and send out the control voltage through the gigabit network port to control the two-dimensional deformable mirror to achieve control of the beam quality; in the ARM processor, the wavefront, control voltage and other states are displayed through the HDMI port; thus, the optical stability control and beam quality control are achieved.

[0101] Embodiment 10: A working method of an industrial intelligent camera, including the industrial intelligent camera described in Embodiment 2, including the steps:

[0102] Step 1. After the original image from the photosensitive surface to the image preprocessing system, the FPGA of the image preprocessing system does not perform any calculation, but only sends the original image to the GPU+ARM core board through the PCIEx4 port;

[0103] Step 2. In the GPU+ARM core board, the GPU processor uses the algorithm to learn the original image, train and identify various target features;

[0104] Step 3. The ARM processor sends the recognized target result to the display through the HDMI interface, and at the same time sends the recognition result through the RS422 port and CAN port to control different industrial targets.

[0105] The parts not involved in the present invention are the same as or can be implemented by the prior art.

[0106] In addition to the above examples, those skilled in the art can obtain inspiration based on the above disclosure or make modifications using the knowledge or technology in related fields to obtain other embodiments. The features of each embodiment can be interchanged or replaced. As long as the modifications and changes made by those skilled in the art do not depart from the spirit and scope of the present invention, they should all fall within the protection scope of the appended claims of the present invention.

Claims

1. An industrial intelligent camera, characterized in that, It includes an image acquisition system, an image preprocessing system and a core processing system; the image acquisition system is used to acquire images and then transmit them to the image preprocessing system; the image preprocessing system is provided with an image spot centroid coordinate extraction module, which is used to transmit the extracted image spot centroid coordinates and the original image to the core processing system, and the image preprocessing system is provided with an optical axis control module, which is used to control the optical axis by controlling the output after calculating the optical axis adjustment amount; the core processing system is a dual-processor architecture, wherein the first processor is used for image wavefront extraction and image recognition, and the second processor is used for beam quality control and configuration of the image preprocessing system; the image acquisition system, the image preprocessing system and the core processing system are all integrated in one camera; a far-field image acquisition system, in which an aperture lens is provided in the far-field image acquisition system, and the aperture lens is used to acquire the far-field image of the target recognition; for the original image transmitted from the far-field image acquisition system to the image preprocessing system, the image preprocessing system does not perform any calculation, and only transmits the original image to the core processing system; The near-field image acquisition system includes a photosensitive surface and an interface circuit substrate, wherein the photosensitive surface is connected to the interface circuit substrate, and the interface circuit substrate includes a high-speed interface board based on a differential conversion circuit for transmitting photosensitive electrical signals in parallel; The image preprocessing system includes an image readout circuit, an FPGA processor, an optical axis control RS422 port drive circuit and a PCIEx4 transmission interface driver; The image readout circuit is used to transfer the differential signal transmitted from the image acquisition system to the original image buffer in the FPGA processor in real time, and the image readout circuit includes a differential parallel port image readout circuit; The FPGA processor extracts the centroid coordinates through the image, adds the control algorithm to the centroid coordinates, calculates the optical axis adjustment amount using the control algorithm, and then drives the RS422 port output through the optical axis control RS422 port drive circuit to achieve the control of the optical axis; PCIEx4 transmission interface driver, used to package and transmit the centroid coordinate array and original image preprocessed by the FPGA processor to the PCIEx4 interface; The core processing system includes a circuit system interface substrate, a solid-state hard disk and a first network port driving circuit, the first processor is a GPU processor, the second processor is an ARM processor, and the GPU processor and the ARM processor are integrated into an integral chip to form a GPU+ARM core board; Circuit system interface substrate, used for conversion and driving of multiple interfaces, including PCIEx4 interface from FPGA system to GPU+ARM core board, PCIE interface between GPU+ARM core board and solid state drive, and network port, USB interface, HDMI interface, CAN interface and RS422 interface of GPU+ARM core board; Solid-state hard drive, used to install embedded Linux operating system and user program APP and data file storage; The first network interface driver circuit is used to output the control quantity calculated by the GPU+ARM core board through the control algorithm through the network interface to control the deformable mirror and thus control the beam quality; the network interface includes a high-speed gigabit network interface.

2. The industrial intelligent camera according to claim 1, characterized in that The second processor embeds an open Linux system and opens all control algorithms to users. Users can achieve different industrial control capabilities based on image recognition by embedding different user codes.

3. The industrial intelligent camera according to claim 1, wherein The image preprocessing system includes a second network interface driver circuit and a high-speed drive fiber optic interface circuit, both of which are used to control function switches.

4. The industrial intelligent camera according to claim 1, wherein The core processing system includes USB and HDMI interface driver circuits, CAN and RS422 interface driver circuits; The USB and HDMI interface driver circuits are used to drive the USB and HDMI interfaces; the USB interface includes a MiniUSB interface and a USB-Type-C interface, both of which are used for external data input or output; HDMI provides an external display interface, which is a digital high-definition interface for monitoring; The CAN and RS422 interface drivers are used to drive the CAN interface and the RS422 interface, providing a multi-functional output interface channel externally; The multi-functional output interface channel includes an optical axis control output channel based on the RS422 interface and a camera cascading control channel based on CAN.

5. A working method of an industrial intelligent camera, characterized in that, Including steps: After starting the industrial intelligent camera described in any one of claims 1, 2 to 4, start the linux system in the GPU+ARM core module and start the user APP; the user APP receives the grid coordinate configuration parameters sent by the host computer and the configuration parameters related to the optical system through the network interface. At the same time, the APP sends the grid configuration parameters to the FPGA system through the PCIEx4 interface for preprocessing by the FPGA system; The original image read by the image readout circuit of the FPGA system from the photosensitive surface is stored in the image buffer; after receiving the original image, the FPGA system performs the following real-time processing: S1, divide the area according to the configured grid coordinate parameters, and calculate the centroid coordinates of the array, including the large-area grid coordinates for controlling the optical axis and the centroid coordinate array of the two-dimensional grid area for wavefront reconstruction; S2, add the centroid coordinates extracted from the large area to the PID control calculation and send out the control optical axis stability through the RS422 port; S3, pack the centroid coordinate array extracted from the two-dimensional grid area and the original image and send them to the GPU+ARM core board through the PCIEx4 port; After receiving the data, the GPU+ARM core board performs the following processing: S4, in the GPU processor, reconstruct the wavefront with the centroid coordinates and the grid coordinate configuration parameters, and solve the voltage with the configured response matrix; in the GPU processor, perform deep learning calculation with the original image, train the mode coefficients of the wavefront, and reconstruct the wavefront, and also solve the voltage with the configured response matrix; S5. In the ARM processor, the resolved wavefront and voltage are compared, and one of them is preferentially selected for PID control calculation to obtain the final control voltage. The control voltage is sent out through the gigabit Ethernet port to control the two-dimensional deformable mirror, thereby realizing the control of the beam quality. In the ARM processor, the wavefront, control voltage, and other states are displayed through the HDMI port. Thus, the optical stability control and beam quality control are achieved.

6. A working method of an industrial intelligent camera, characterized in that, Including the industrial intelligent camera described in claim 1, comprising the steps of: Step 1: After the original image from the photosensitive surface reaches the image preprocessing system, the FPGA in the image preprocessing system does not perform any calculation, but only sends the original image to the GPU+ARM core board through the PCIe x4 port. Step 2: In the GPU+ARM core board, the GPU processor uses an algorithm to learn the original image, train, and identify various target features. Step 3: The ARM processor sends the recognized target result to the display through the HDMI interface, and at the same time sends the recognition result through the RS422 port and CAN port to control different industrial targets.

Images