Environmental perception system based on laser ranging and camera imaging and its use method

Abstract

The present invention discloses an environment perception system based on laser ranging and camera imaging and a method for using the same. The environment perception system includes a camera imaging system and a laser ranging system. The environment perception system uses the optical center of the camera imaging system as the origin O, uses the optical axis of the camera imaging system as the coaxial axis, and constructs an environment perception system coordinate system O-xyz based on the camera imaging system coordinate system. The laser light path formed by the laser emitted by the laser ranging system can pass through the optical center and coaxially with the optical axis to reach the target object located in the area to be detected. When the environment perception system based on laser ranging and camera imaging provided by the present invention is used, the coaxial fusion of the camera imaging system and the laser ranging system is used to achieve the ranging of the target point (object) on the optical axis of the camera imaging system, thereby achieving accurate positioning of the target point (object), so as to accurately perceive the surrounding environment. The output content of the environment perception system is concise, the computing power requirement is small, and the system is stable and reliable.

Description

Technical Field

[0001] The present invention belongs to the technical field of environmental perception, and in particular to an environmental perception system based on laser ranging and camera imaging and a method of using the same. Background Art

[0002] Social labor productivity is first of all the power of science. As China's industrial digital transformation and upgrading are advancing further and further, intelligent industrial equipment is becoming more and more popular. As the eyes of intelligent devices, the development and application of environmental perception sensors are becoming more and more popular in the industry, but the challenges that come with it are also increasing. Environmental perception is the premise and basis for intelligent devices to understand the surrounding environment and subsequent work. The results of environmental perception have a very important impact on the entire system. Therefore, the continuous improvement of the performance of environmental perception sensors will infinitely help the transformation and upgrading of industrial digitalization.

[0003] With the advancement of technology, the environmental perception sensors commonly used in smart devices include cameras, millimeter-wave radars, ultrasonic radars, and lidars.

[0004] A camera is a device that uses the principle of optical imaging to form an image and uses film or image sensors to record the image. The current camera technology and industry chain are mature, and digital cameras can provide real-time feedback of the captured images, so cameras are one of the most popular sensors among perception sensors. Due to their high frame rate, high resolution, high reliability, and high cost-effectiveness, cameras are used in various industries, including medical imaging, surgical robots, photography, video, monitoring, guidance, and measurement.

[0005] The laser system is a system that uses laser to accurately measure the distance to the target. When working, it emits a very thin laser beam to the target, and the photoelectric element receives the laser beam reflected by the target. According to the time from the laser beam emission to the reception, the distance of the target is calculated. The laser distance measurement system is light in weight, small in size, simple to operate, reliable in performance, fast and accurate, so it is widely used in terrain measurement, battlefield measurement, cloud measurement, industrial measurement and control, mines and ports.

[0006] Different sensors have different advantages. The application scenarios of the above-mentioned single sensors will be restricted by their limitations. Therefore, some scholars study the combination system of different sensors, combining the advantages of different sensors to achieve functional superposition, so that the entire system can obtain richer information, which is more intuitive and more effective than a single sensor.

[0007] At present, the mainstream combination mostly uses the calibration of the coordinate systems of the camera imaging system and the laser or radar system to achieve relative combination in space. The target object can only be located by scanning the camera optical axis in space through the laser optical axis. It has the disadvantages of data waste, high computing power requirements, relatively complex motion structure, and low system stability and reliability. Summary of the invention

[0008] The purpose of the present invention is to provide an environment perception system based on laser ranging and camera imaging and a method for using the system in order to address the problems existing in the prior art.

[0009] The purpose of the present invention is to be solved by the following technical solutions:

[0010] An environmental perception system based on laser ranging and camera imaging, characterized in that: the environmental perception system includes a camera imaging system and a laser ranging system, the environmental perception system uses the optical center of the camera imaging system as the origin O, uses the optical axis of the camera imaging system as the coaxial axis, and constructs an environmental perception system coordinate system O-xyz based on the camera imaging system coordinate system, and the laser light path formed by the laser emitted by the laser ranging system can pass through the optical center and reach a target object located in a to-be-detected area coaxially with the optical axis.

[0011] The environmental perception system also includes a motion system, which can drive the camera imaging system and the laser ranging system to rotate around the optical center.

[0012] The camera imaging system comprises an image sensor module and a lens module, wherein the lens module is located at the optical center of the camera imaging system.

[0013] When the laser light path formed by the laser emitted by the laser ranging system directly extends along the direction of the optical axis of the camera imaging system, the image sensor module is located between the laser ranging system and the lens module, and at this time, a through hole with an aperture of φ is set at the center of the effective pixel of the image sensor module, φ≥ω(L), where ω(L) is the beam diameter of the laser emitted by the laser ranging system when it reaches the through hole with an aperture of φ on the image sensor module after being collimated and passing through a distance L; the laser light path passes through the image sensor module through the through hole, passes through the optical center and is coaxial with the optical axis.

[0014] When the laser light path formed by the laser emitted by the laser ranging system needs to be reflected in order to extend along the direction of the optical axis of the camera imaging system, a reflector module is arranged between the image sensor module and the lens module, and the reflector module can allow the laser reflection light path formed by the reflection of the laser emitted by the laser ranging system to pass through the optical center and be coaxial with the optical axis.

[0015] The reflector used in the reflector module has a semi-reflective and semi-transmissive performance, and the size of the reflector module is not less than the size of the image sensor module divided by cosβ; the angle between the reflector module and the optical axis is β and 0<β<90°, and the laser incident angle formed on the reflector module by the laser emitted by the laser ranging system is α and 0<α<90°.

[0016] The image sensor module is composed of one or more independently arranged image sensors; the multiple image sensors are arranged in the same plane and adjacent image sensors are separated by light blocking gratings, and the image sensors in the image sensor module meet the requirements of central symmetry.

[0017] The environment perception system coordinate system O-xyz is the coordinate system of the camera imaging system. The straight line where the optical axis of the camera imaging system is located in the O-xyz coordinate system is represented as l1, and the straight line where the laser optical path of the laser ranging system is located in the O-xyz coordinate system is represented as l2; since the optical axis l1 passes through the point M1 (x1, y1, z1) and its direction vector is S1 = (m1, n1, p1), the straight line equation of the optical axis l1 is expressed as: Since the laser light path l2 passes through point M2 (x2, y2, z2) and its direction vector is S2 = (m2, n2, p2), the straight line equation of the laser light path l2 is expressed as:

[0018] When m d 、n d 、p d When (d=1,2) are not 0, the limiting condition that the optical axis l1 and the laser optical path l2 are coaxial is: and

[0019] When m d 、n d 、p d One of (d=1,2) is zero, such as m d =0, n d ≠0, p d When ≠0, the straight line equations of the optical axis l1 and the laser optical path l2 are: At this time, the limiting condition that the optical axis l1 and the laser optical path l2 are coaxial is: and

[0020] When m d 、n d 、p d (d=1,2) has two zeros, such as m d =n d =0, p dWhen ≠0, the straight-line equations of the optical axis l1 and the laser optical path l2 are: l1: x-x1=0, y-y1=0, l2: x-x2=0, y-y2=0. At this time, the limiting conditions for the coaxiality of the optical axis l1 and the laser optical path l2 are: m1=m2=0, n1=n2=0 and x1=x2, y1=y2; further, when x1=x2=0, y1=y2=0, the optical axis l1 and the laser optical path l2 coincide with the Z axis of the environmental perception system coordinate system O-xyz.

[0021] A method of using an environment perception system based on laser ranging and camera imaging is as follows:

[0022] Step A: The camera imaging system generates a corresponding image signal according to the optical signal of the surrounding area to be detected;

[0023] Step B, identifying the target in the image signal in step A and marking it;

[0024] Step C, identifying the central pixel of the marked target in step B, selecting b points within the central pixel, and calculating the azimuth angle θ of the b points in the environment perception system coordinate system O-xyz;

[0025] Step D, taking the origin O of the environment perception system coordinate system O-xyz as the rotation center, rotating the optical axis to b points respectively according to the azimuth angles θ corresponding to the b points, and starting the laser ranging system at each point to emit a laser signal of a set wavelength, a set frequency and a set intensity to detect the target at the corresponding point, and receiving the laser echo signals of b targets at b points;

[0026] Step E: The laser ranging system calculates the average distance, target moving speed and target moving acceleration of the target on the image according to the laser echo signal;

[0027] Step F, repeating steps A to E, outputting the image of the surrounding area to be detected and the target average distance, direction, target moving speed and target moving acceleration information.

[0028] Compared with the prior art, the present invention has the following advantages:

[0029] The environment perception system based on laser ranging and camera imaging provided by the present invention realizes the ranging of the target point (object) on the optical axis of the camera imaging system through the coaxial fusion of the camera imaging system and the laser ranging system, thereby realizing the accurate positioning of the target point (object), so as to accurately perceive the surrounding environment. The output content of the environment perception system is concise, the computing power requirement is small, and the system is stable and reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Attached Figure 1A schematic diagram of the optical path coaxial structure of the environment perception system based on laser ranging and camera imaging provided by the present invention;

[0031] Attached Figure 2 A schematic diagram of the application principle of the environment perception system based on laser ranging and camera imaging provided by the present invention;

[0032] Attached Figure 3 A schematic diagram of the optical path structure of the environment perception system based on laser ranging and camera imaging provided by the present invention;

[0033] Attached Figure 4 A schematic diagram of three-dimensional perception of an environment perception system based on laser ranging and camera imaging provided by the present invention;

[0034] Attached Figure 5 A schematic diagram of a pixel point selection method for an environment perception system based on laser ranging and camera imaging provided by the present invention;

[0035] Attached Figure 6 A schematic diagram of the structure of an environment perception system based on laser ranging and camera imaging provided in the first embodiment of the present invention;

[0036] Attached Figure 7 A schematic diagram of the structure of an image sensor module provided by the present invention when it is composed of one image sensor;

[0037] Attached Figure 8 One of the structural schematic diagrams when the image sensor module provided by the present invention is composed of two image sensors;

[0038] Attached Fig. 9 The second structural schematic diagram of the image sensor module provided by the present invention when it is composed of two image sensors;

[0039] Attached Fig.10 A schematic diagram of the structure of an image sensor module provided by the present invention when it is composed of four image sensors;

[0040] Attached Fig.11 A schematic diagram of the structure of an environment perception system based on laser ranging and camera imaging provided in Embodiment 2 of the present invention.

[0041] Among them: 1—camera imaging system; 11—image sensor module; 110—light blocking grating; 12—lens module; 2—laser ranging system; 3—reflector module. DETAILED DESCRIPTION

[0042] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0043] like Figure 1-11As shown: an environment perception system based on laser ranging and camera imaging, the environment perception system includes a camera imaging system 1 and a laser ranging system 2. The environment perception system uses the optical center of the camera imaging system 1 as the origin O, the optical axis of the camera imaging system 1 as the coaxial axis, and constructs an environment perception system coordinate system O-xyz based on the coordinate system of the camera imaging system 1. The laser light path formed by the laser ranging system 2 can pass through the optical center and coaxially with the optical axis to reach the target object located in the area to be detected. Furthermore, the environment perception system also includes a motion system, which can drive the camera imaging system 1 and the laser ranging system 2 to rotate around the optical center.

[0044] In simple terms, the environment perception system places the laser ranging system 2 and the camera imaging system 1 in a composite sensor. The coordinates of the laser ranging system 2 and the camera imaging system 1 are synthesized into the same axis in the same direction (such as the Z direction). The same direction is also the optical path of the two systems, thus achieving the coaxial optical path (see Figure 1 ), when the camera imaging system 1 takes a picture and captures the target (assuming it is in the AA plane area), the X and Y coordinates of the target with respect to the coordinate system of the camera imaging system 1 can be obtained, and the orientation of the target through the optical axis of the camera imaging system 1 can also be obtained, that is, the spatial angle θ with the coordinate system of the camera imaging system 1, but the camera imaging system 1 itself cannot obtain the distance of the target. Through the motion system, the composite sensor is rotated according to the spatial angle θ (such as from Figure 2 ① in the figure is rotated to ②), so that the laser optical path of the laser ranging system 2, the optical path of the camera imaging system 1 and the target object are on the same straight line, and the laser ranging system 2 measures the distance, thereby obtaining the accurate position of the target object (see Figure 2 ).

[0045] On the basis of the above, the camera imaging system 1 provided in the present application includes an image sensor module 11 and a lens module 12, wherein the lens module 12 is located at the optical center of the camera imaging system 1; the image sensor module 11 is composed of one or more independently arranged image sensors (such as Figure 7-Figure 10 ), multiple image sensors are arranged in the same plane and adjacent image sensors are separated by light blocking gratings 110, and the image sensors in the image sensor module 11 meet the requirement of central symmetry.

[0046] According to the needs, there are generally two ways to set the initial position of the laser ranging system 2 relative to the camera imaging system 1. First, when the laser light path formed by the laser emitted by the laser ranging system 2 directly extends along the direction of the optical axis of the camera imaging system 1, the image sensor module 11 is located between the laser ranging system 2 and the lens module 12 (such as Figure 6As shown), and at this time, a through hole with an aperture of φ is set at the center of the effective pixel of the image sensor module 11, φ≥ω(L), where ω(L) is the beam diameter of the laser emitted by the laser ranging system 2 after being collimated and reaching the through hole with an aperture of φ on the image sensor module 11 after a distance L, and the laser light path passes through the image sensor module 11 through the through hole, passes through the optical center and is coaxial with the optical axis; the second type, when the laser light path formed by the laser emitted by the laser ranging system 2 needs to be reflected to extend along the direction of the optical axis of the camera imaging system 1, a reflector module 3 (as shown) is arranged between the image sensor module 11 and the lens module 12 Fig.11 As shown), the angle between the reflector module 3 and the optical axis is β and 0<β<90°, and the laser incident angle formed on the reflector module 3 by the laser emitted by the laser ranging system 2 is α and 0<α<90°; the reflector module 3 can allow the laser reflection light path formed after the laser emitted by the laser ranging system 2 is reflected to pass through the optical center and be coaxial with the optical axis.

[0047] When assembling the environment perception system based on laser ranging and camera imaging provided in the present application, it is necessary to adjust to a coaxial structure. The environment perception system coordinate system O-xyz of the above-mentioned environment perception system is the coordinate system of the camera imaging system 1. The straight line where the optical axis of the camera imaging system 1 is located in the O-xyz coordinate system is represented as l1, and the straight line where the laser optical path of the laser ranging system 2 is located in the O-xyz coordinate system is represented as l2 (such as Figure 3 As shown); since the optical axis l1 passes through point M1 (x1, y1, z1) and its direction vector is S1 = (m1, n1, p1), the straight line equation of the optical axis l1 is expressed as: Since the laser light path l2 passes through point M2 (x2, y2, z2) and its direction vector is S2 = (m2, n2, p2), the straight line equation of the laser light path l2 is expressed as:

[0048] When m d 、n d 、p d When (d=1,2) are not 0, based on the above linear equations of the optical axis l1 and the laser optical path l2, the limiting condition that the optical axis l1 and the laser optical path l2 are coaxial is: and

[0049] When m d 、n d 、p d One of (d=1,2) is zero, such as m d =0, n d ≠0, p d ≠0, the linear equations of the optical axis l1 and the laser optical path l2 can be changed to: At this time, the limiting condition that the optical axis l1 and the laser optical path l2 are coaxial is: and

[0050] When m d 、n d 、p d (d=1,2) has two zeros, such as m d =n d =0, p d ≠0, the linear equations of the optical axis l1 and the laser optical path l2 can be changed to: l1: x-x1=0, y-y1=0, l2: x-x2=0, y-y2=0. At this time, the limiting conditions for the coaxiality of the optical axis l1 and the laser optical path l2 are: m1=m2=0, n1=n2=0 and x1=x2, y1=y2; and when x1=x2=0, y1=y2=0, the optical axis l1 and the laser optical path l2 coincide with the Z axis of the environmental perception system coordinate system O-xyz.

[0051] In summary, when the parameters (x1, y1, z1; m1, n1, p1; x2, y2, z2; m2, n2, p2) meet the corresponding conditions, it means that the optical axis l1 and the laser optical path l2 are coaxial.

[0052] like Figure 4 , Figure 5 As shown, when the environment perception system based on laser ranging and camera imaging provided by the present invention is used, the method of using the environment perception system is as follows:

[0053] Step A: The camera imaging system 1 generates a corresponding image signal according to the optical signal of the surrounding area to be detected;

[0054] Step B: Identify and mark the target in the image signal in step A. The identification method adopts a deep learning method such as a convolutional neural network (CNN) or a Transformer large model. The optional targets include but are not limited to people, animals, plants, vehicles, etc. The optional marking methods include but are not limited to rectangular boxes, such as Figure 4 In the rectangle ABCD, the number of pixels of the target is n (n≥1);

[0055] Step C: Identify the central pixel of the marked target in step B. The central pixel refers to the area containing a (1≤a≤n) pixels at the center of the marked target, and select b points from the pixels in the area according to the rule of the maximum inscribed positive N edges: P1, P2…P b (1≤b≤a), the optional rules include but are not limited to drawing the largest inscribed circle in the a pixel area (in this case N=∞), dividing the inscribed circle into b equal parts, and the equal points on the circle are points P1, P2…P b (like Figure 5 as shown); and according to the formula Calculate the azimuth angle θ (θ1, θ2…θ b ), that is, points P1, P2…P b The angles with the optical axis are θ1, θ2…θ b , where: (μ p , ν p ) is point P (including P1, P2…P b ) is the coordinate of the pixel coordinate system of the camera, (μ0, ν0) represents the pixel coordinate of the intersection of the optical axis and the image plane, is the intrinsic parameter of the camera, (d x , d y ) represents the physical size of each pixel in the image coordinate system, is the intrinsic parameter of the camera, f is the focal length of the camera, is the intrinsic parameter of the camera;

[0056] Step D, taking the origin O of the environment perception system coordinate system O-xyz as the rotation center, rotating the optical axis to b points according to the azimuth angles θ corresponding to the b points, and starting the laser ranging system 2 at each point to emit a laser signal of a set wavelength, a set frequency and a set intensity to detect the target at the corresponding point, and receiving the laser echo signals of b targets at b points;

[0057] Step E: The laser ranging system 2 calculates the average distance m of the target on the image based on the laser echo signal: Where c is the speed of light, t1, t2...t b They represent the time interval from the emission to the reception of the laser of b point targets respectively;

[0058] Calculate the target moving speed v on the image ij :v ij =(m j +s ij -m i ) / Δt ij , where m i The average target distance calculated for the i-th image, m j The average target distance calculated for the jth image (1≤i<j), is the time interval between the i-th image and the j-th image, s ij for The distance traveled by the vehicle in the time period;

[0059] Calculate the target movement acceleration on the image Where V ij is the target moving speed calculated based on the i-th image and the j-th image, V jk is the target moving speed calculated based on the j-th image and the k-th image (1≤i<j<k), ΔT jkis the time interval between the j-th image and the k-th image;

[0060] Step F, repeating steps A to E, outputting the image of the surrounding area to be detected and the target average distance, azimuth, target moving speed and target moving acceleration information.

[0061] In order to more clearly understand the above-described features, purposes and advantages of the present application, the environment perception system based on laser ranging and camera imaging provided by the present invention is further described below. The present application is further described below in conjunction with the accompanying drawings and specific preferred embodiments, but the protection scope of the present application is not limited thereby.

[0062] Embodiment 1

[0063] Reference Figure 6 , Figure 6 An environment perception system based on laser ranging and camera imaging is provided in Example 1 of the present application, and the environment perception system includes a camera imaging system 1, a laser ranging system 2 and a motion system, wherein the camera imaging system 1 includes but is not limited to an image sensor module 11 and a lens module 12, wherein the image sensor module 11 includes n (n≥1) independent image sensors, when n≥2, each independent image sensor is arranged in the same plane and meets the requirement of central symmetry; l1 is the optical axis of the camera imaging system 10, l2 is the laser optical path of the laser ranging system 2; O-xyz is the coordinate system of the camera imaging system 1 (that is, the coordinate system of the environment perception system), the origin O of the O-xyz coordinate system is the optical center, the Y axis is vertically outward, and the Z axis points to the direction of the optical axis.

[0064] In order to enable the laser to successfully pass through the optical axis to measure the distance to the target in front of the center of the optical axis, a through hole with an aperture of φ is set at the center of the effective pixel of the image sensor module 11. The size of φ is determined according to the formula φ≥ω(L), where ω(L) is the beam diameter of the laser emitted by the laser ranging system 2 when it reaches the through hole with an aperture of φ on the image sensor module 11 after being collimated and passing through a distance L.

[0065] because Figure 6 The Z-axis of the environmental perception system coordinate system O-xyz points to the optical axis, so the straight line equation of the optical axis l1 is expressed as: l1: x-x1=0, y-y1=0, where x1=0, y1=0, and the direction vector is (m1, n1, p1)=(0,0,1); therefore, when the laser optical path l2: x-x2=0, y-y2=0, the direction vector satisfies m1=m2=0, n1=n2=0, p2≠0 and x1=x2=0, y1=y2=0, the laser optical path l2 and the optical axis l1 are coaxial.

[0066] like Figure 7-10As shown, it is assumed that the side lengths of the image sensors in the image sensor module 11 are C and D respectively. Figure 7 The figure shows a Z-direction view of the image sensor module 11 when the number n=1; preferably, when n=2, the Z-direction view of the image sensor module 11 is as follows: Figure 8 and Fig. 9 As shown, Figure 8 The image sensor module 11 shown is composed of C-side splicing of image sensors. Fig. 9 The image sensor module 11 shown is formed by splicing the D edges of the image sensors; optionally, when n=4, the Z-direction view of the image sensor module 11 is as follows: Fig.10 As shown; a light blocking grating 110 is arranged between the above-mentioned spliced ​​image sensors.

[0067] Embodiment 2

[0068] Reference Fig.11 , Fig.11 The environment perception system based on laser ranging and camera imaging provided in the second embodiment of the present application comprises a camera imaging system 1, a laser ranging system 2, a reflector module 3 and a motion system, wherein the camera imaging system 1 comprises but is not limited to an image sensor module 11 and a lens module 12, l1 is an optical axis of the camera imaging system 10, l 21 is the laser emission optical path of the laser ranging system 2, l 22 is the laser reflection optical path of the laser ranging system 2; O-xyz is the coordinate system of the camera imaging system 1 (that is, the coordinate system of the environment perception system), the origin O of the O-xyz coordinate system is the optical center, the Y axis is vertically outward, and the Z axis points to the direction of the optical axis; the reflector module 3 (such as Fig.11 The angle between the laser beam emitted by the laser ranging system 2 and the optical axis is β and 0<β<90°, the laser incident angle formed by the laser beam emitted by the laser ranging system 2 on the reflector module 3 is α and 0<α<90°, and the reflector module 3 can make the laser reflected light path formed by the laser beam emitted by the laser ranging system 2 pass through the optical center and be coaxial with the optical axis; the optical axis l1 and the laser emission light path l 21 The coplanar surface is perpendicular to the reflecting surface of the reflector module 3, and the optical axis l1 is perpendicular to the laser emission optical path l 21 intersects on the reflecting surface.

[0069] because Fig.11 The Z axis of the environment perception system coordinate system O-xyz points to the optical axis, so the straight line equation of the optical axis l1 is expressed as: l1:x-x1=0, y-y1=0, where x1=0, y1=0, and the direction vector is (m1,n1,p1)=(0,0,1); α=45°, β=45°, at this time the laser emission optical path l 21 It can be expressed as: 21:yy 21 =0,zz 21 =0, where y 21 =0, direction vector is (m 21 ,n 21 ,p 21 )=(1,0,0), due to the laser reflection light path l 22 The laser emission path 21 vertical and coplanar, so the laser reflection light path l 22 It can be expressed as l 22 :xx 22 =0,yy 22 =0, where x 22 =0,y 22 =0, (m 22 ,n 22 ,p 22 )=(0,0,1), satisfying m1=m 22 =0, n1=n 22 =0 and x1=x 22 =0,y1=y 22 =0, at this time, the laser reflection light path l 22 It is coaxial with the optical axis l1.

[0070] The environment perception system based on laser ranging and camera imaging provided by the present invention realizes the ranging of the target point (object) on the optical axis of the camera imaging system through the coaxial fusion of the camera imaging system and the laser ranging system, thereby realizing the accurate positioning of the target point (object), so as to accurately perceive the surrounding environment. The output content of the environment perception system is concise, the computing power requirement is small, and the system is stable and reliable.

[0071] The above embodiments are only for illustrating the technical idea of ​​the present invention, and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the present invention; any technology not involved in the present invention can be realized by existing technologies.

Claims

1. An environmental perception system based on laser ranging and camera imaging, characterized in that: The environment perception system comprises a camera imaging system (1) and a laser ranging system (2). The environment perception system uses the optical center of the camera imaging system (1) as an origin O, uses the optical axis of the camera imaging system (1) as a coaxial axis, and constructs an environment perception system coordinate system O-xyz based on the coordinate system of the camera imaging system (1). The laser light path formed by the laser emitted by the laser ranging system (2) can pass through the optical center and reach a target object located in a to-be-detected area coaxially with the optical axis. The camera imaging system (1) comprises an image sensor module (11) and a lens module (12), wherein the lens module (12) is located at the optical center of the camera imaging system (1); When the laser light path formed by the laser emitted by the laser ranging system (2) directly extends along the direction of the optical axis of the camera imaging system (1), the image sensor module (11) is located between the laser ranging system (2) and the lens module (12), and at this time, a through hole with an aperture of φ is provided at the effective pixel center of the image sensor module (11), φ≥ω(L), wherein ω(L) is the beam diameter of the laser emitted by the laser ranging system (2) after being collimated and passing through a distance L to reach the through hole with an aperture of φ on the image sensor module (11); the laser light path passes through the image sensor module (11) through the through hole, passes through the optical center and is coaxial with the optical axis.

2. The environment perception system based on laser ranging and camera imaging according to claim 1, characterized in that: The environment perception system also includes a motion system, which can drive the camera imaging system (1) and the laser ranging system (2) to rotate around the optical center.

3. The environment perception system based on laser ranging and camera imaging according to claim 1, characterized in that: The image sensor module (11) is composed of one or more independently arranged image sensors; the multiple image sensors are arranged on the same plane and adjacent image sensors are separated by light blocking gratings (110), and the image sensors in the image sensor module (11) meet the requirement of central symmetry.

4. The environment perception system based on laser ranging and camera imaging according to claim 1, characterized in that: The environment perception system coordinate system O-xyz is the coordinate system of the camera imaging system (1), the straight line where the optical axis of the camera imaging system (1) is located in the O-xyz coordinate system is represented by l1, and the straight line where the laser optical path of the laser ranging system (2) is located in the O-xyz coordinate system is represented by l2; since the optical axis l1 passes through the point M1 (x1, y1, z1) and its direction vector is S1 = (m1, n1, p1), the straight line equation of the optical axis l1 is expressed as: Since the laser light path l2 passes through point M2 (x2, y2, z2) and its direction vector is S2 = (m2, n2, p2), the straight line equation of the laser light path l2 is expressed as: When m d 、n d 、p d When d=1,2 are not 0, the limiting condition that the optical axis l1 and the laser optical path l2 are coaxial is: and When m d 、n d 、p d d=1,2 One of them is zero, such as m d =0, n d ≠0, p d When ≠0, the straight line equations of the optical axis l1 and the laser optical path l2 are: At this time, the limiting condition that the optical axis l1 and the laser optical path l2 are coaxial is: and When m d 、n d 、p d d=1,2 Two of them are zero, such as m d =n d =0, p d When ≠0, the straight-line equations of the optical axis l1 and the laser optical path l2 are: l1: x-x1=0, y-y1=0, l2: x-x2=0, y-y2=0. At this time, the limiting conditions for the coaxiality of the optical axis l1 and the laser optical path l2 are: m1=m2=0, n1=n2=0 and x1=x2, y1=y2; further, when x1=x2=0, y1=y2=0, the optical axis l1 and the laser optical path l2 coincide with the Z axis of the environmental perception system coordinate system O-xyz.

5. The environment perception system based on laser ranging and camera imaging according to any one of claims 1 to 4, characterized in that: The method of using the environment perception system is as follows: Step A, the camera imaging system (1) generates a corresponding image signal according to the optical signal of the surrounding area to be detected; Step B, identifying the target in the image signal in step A and marking it; Step C: Identify the central pixel of the marked target in step B, select b points within the central pixel, and calculate the azimuth angle θ of the b points in the environment perception system coordinate system O-xyz. Step D: With the origin O of the environment perception system coordinate system O-xyz as the rotation center, the optical axis is rotated to b points respectively according to the azimuth angles θ corresponding to the b points, and the laser ranging system (2) is started at each point to emit a laser signal with a set wavelength, a set frequency and a set intensity to detect the target at the corresponding point, and the laser echo signals of b targets at the b points are received; Step E: The laser ranging system (2) calculates the average distance of the target, the target moving speed and the target moving acceleration on the image according to the laser echo signals; Step F, repeating steps A to E, outputting the image of the surrounding area to be detected and the target average distance, direction, target moving speed and target moving acceleration information.

Images