Environmental perception device and method for mobile vehicle

Abstract

This invention provides an environmental sensing device and method for a mobile vehicle. The environmental sensing device includes a camera module, a lidar module, a database, and a processing circuit. The camera module captures images of the area near the mobile vehicle, generating three-dimensional image frames. The lidar module scans the area, generating three-dimensional scan frames. The processing circuit fuses the three-dimensional image frames and the three-dimensional scan frames to generate three-dimensional object information. The processing circuit compares the three-dimensional object information with a three-dimensional map in the database to determine whether the object is stationary. When the object is not stationary, the processing circuit analyzes and calculates the three-dimensional object information to determine the object's motion characteristics. When the object is stationary, the processing circuit skips the analysis and calculation of the three-dimensional object information.

Description

Technical Field

[0001] The present invention relates to a mobile vehicle, and in particular to an environment perception device and method for a mobile vehicle. Background Art

[0002] When a mobile vehicle (such as a car) is driving on the road, the mobile vehicle needs to sense obstacles in the environment, including static objects and dynamic objects (moving objects). For example, when an object approaches the rear of the car, the reversing radar will sound an alarm. The existing technology only determines the distance between the object and the mobile vehicle, and cannot effectively identify the outline of the surrounding objects, and thus cannot distinguish what the object actually is. When there are unobstructed objects such as weeds around the mobile vehicle, the existing technology will still produce warning sounds, and may even trigger automatic braking, etc., causing unnecessary trouble. Environmental perception is one of the important technical topics in the field of mobile vehicle technology. Summary of the invention

[0003] The present invention provides an environment perception device and an environment perception method for a mobile vehicle, which combine the characteristics of a camera module and a lidar module to perform environment perception.

[0004] In an embodiment of the present invention, the above-mentioned environment perception device includes a camera module, a lidar module, a database and a processing circuit. The camera module and the lidar module are configured on a mobile vehicle. The camera module is suitable for photographing a field near the mobile vehicle to generate a three-dimensional image frame. The lidar module is suitable for scanning the field to generate a three-dimensional scanning frame. The database is suitable for storing a three-dimensional map. The processing circuit is coupled to the camera module and the lidar module to receive a three-dimensional image frame and a three-dimensional scanning frame. The processing circuit is configured to fuse the three-dimensional image frame and the three-dimensional scanning frame to generate three-dimensional object information corresponding to the object in the field. The processing circuit compares the three-dimensional object information with the three-dimensional map to determine whether the object is a stationary object. In the case where the object is not a stationary object, or in the case where the object has not been recorded in the three-dimensional map, the processing circuit analyzes and calculates the three-dimensional object information to obtain the motion characteristics of the object. In the case where the object is a stationary object, the processing circuit skips the analysis and calculation of the three-dimensional object information.

[0005] In an embodiment of the present invention, the above-mentioned environmental perception method includes: a camera module configured on a mobile vehicle photographs a field near the mobile vehicle to generate a three-dimensional image frame; a lidar module configured on the mobile vehicle scans the field to generate a three-dimensional scanning frame; a processing circuit fuses the three-dimensional image frame and the three-dimensional scanning frame to generate three-dimensional object information corresponding to the object in the field; the processing circuit compares the three-dimensional object information with a three-dimensional map stored in a database to determine whether the object is a stationary object; when the object is not a stationary object, or when the object has not been recorded in the three-dimensional map, the processing circuit analyzes and calculates the three-dimensional object information to obtain the motion characteristics of the object; and when the object is a stationary object, the processing circuit skips the analysis and calculation of the three-dimensional object information.

[0006] Based on the above, the environment perception device described in the embodiments of the present invention is configured with a camera module and a lidar module. The processing circuit can fuse the three-dimensional information of the three-dimensional image frame of the camera module with the three-dimensional information of the three-dimensional scanning frame of the lidar module to generate three-dimensional object information corresponding to a certain object in the field. Therefore, the processing circuit can combine the characteristics of the camera module and the lidar module to perform environment perception. In some embodiments, the processing circuit can compare the three-dimensional object information with the three-dimensional map stored in the database. In the case where the three-dimensional object information has not been analyzed (that is, the three-dimensional object information has not been recorded in the database), the processing circuit can analyze and calculate the three-dimensional object information to obtain the motion characteristics of the object, and then record the three-dimensional object information and the analysis and calculation results in the database to construct a three-dimensional map. Moving objects (people or cars) generally do not belong to stationary objects in the three-dimensional map, so the processing circuit can analyze and calculate the moving objects to obtain the motion characteristics of the object. In the case where the object is a stationary object, the processing circuit can skip the analysis and calculation of the stationary object that has been analyzed to reduce the duplication of calculations.

[0007] In order to make the above features and advantages of the present invention more clearly understood, embodiments are given below with reference to the accompanying drawings for detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 is a schematic diagram of a circuit block of an environment sensing device according to an embodiment of the present invention;

[0009] Figure 2 is a flow chart of an environment perception method according to an embodiment of the present invention;

[0010] Figure 3 According to an embodiment of the present invention, Figure 1 A circuit block diagram of the processing circuit shown;

[0011] Figure 4 It is a flowchart of an environment perception method according to another embodiment of the present invention.

[0012] Description of Reference Numerals

[0013] 10: Field

[0014] 11:Object

[0015] 100: Environmental sensing device

[0016] 110: Camera module

[0017] 111: 3D image frame

[0018] 120: LiDAR module

[0019] 121: 3D scanning frame

[0020] 130: Processing circuit

[0021] 131, 132: Partition circuit

[0022] 133: Object recognition circuit

[0023] 140: Database

[0024] 150: Prompt unit

[0025] S210~S250: Steps DETAILED DESCRIPTION

[0026] Reference will now be made in detail to exemplary embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Whenever possible, the same reference numerals are used in the drawings and the description to refer to the same or like parts.

[0027] The term "coupled (or connected)" used in the entire specification of this case (including the claims) may refer to any direct or indirect means of connection. For example, if the text describes a first device coupled (or connected) to a second device, it should be interpreted that the first device can be directly connected to the second device, or the first device can be indirectly connected to the second device through other devices or some connection means. The terms "first", "second", etc. mentioned in the entire specification of this case (including the claims) are used to name the components (element) or to distinguish different embodiments or scopes, and are not used to limit the upper or lower limit of the number of components, nor to limit the order of components. In addition, wherever possible, components / components / steps with the same number in the drawings and embodiments represent the same or similar parts. Components / components / steps using the same number or the same terminology in different embodiments can refer to the relevant descriptions of each other.

[0028] Figure 1 1 is a circuit block diagram of an environment sensing device 100 according to an embodiment of the present invention. The environment sensing device 100 may be configured in a mobile vehicle (eg, a car, not shown) to sense the environment of the mobile vehicle. Figure 1 The environment sensing device 100 shown includes a camera module 110, a LiDAR module 120, a processing circuit 130, a database 140, and a prompt unit 150. The camera module 110 and the LiDAR module 120 are configured on the mobile vehicle. The camera module 110 is suitable for photographing a field 10 near the mobile vehicle to generate a three-dimensional image frame 111. The LiDAR module 120 is suitable for scanning the field 10 to generate a three-dimensional scanning frame 121. Figure 1 In the illustrated example, the object 11 is in the scene 10 . The processing circuit 130 is coupled to the camera module 110 and the lidar module 120 to receive the 3D image frame 111 and the 3D scanning frame 121 .

[0029] Figure 2 is a flow chart of an environment perception method according to an embodiment of the present invention. Figure 1 and Figure 2 . In step S210, the camera module 110 configured on a mobile vehicle (such as a car, not shown) can capture the field 10 near the mobile vehicle to generate a three-dimensional image frame 111. The implementation of the camera module 110 can be determined according to design requirements. For example, in some embodiments, the camera module 110 may include a depth camera (not shown), and the three-dimensional image frame 111 may include a depth image frame. The camera module 110 can detect whether there are objects that may affect driving safety in the space (field 10) within tens of meters near the mobile vehicle, such as pedestrians, animals, cars, motorcycles and / or other moving objects, or fixed fences, bushes, etc.

[0030] In other embodiments, the camera module 110 may include two color cameras (not shown), one infrared (IR) camera (not shown) and two IR light sources (not shown). If there are other objects (such as the object 11) around the mobile vehicle, the light from the IR light source will be projected onto the surface of the object 11. The reflected light from the object 11 will be received by the camera, thereby generating a three-dimensional image frame 111.

[0031] In step S220, the lidar module 120 configured on a mobile vehicle (such as a car, not shown) can scan the field 10 to generate a three-dimensional scanning frame 121. The laser light source of the lidar module 120 can irradiate a beam of light (such as a pulsed laser) to the field 10 to measure parameters such as the distance of the target (such as object 11). The laser light source of the lidar module 120 has the characteristics of good coherence, slow attenuation, long measurement distance, high accuracy, etc., and the laser light source is not easily affected by other light sources, so it is better than general light projection. Based on the three-dimensional information of the three-dimensional scanning frame 121 of the lidar module 120, the environmental perception device 100 can predict in advance the dynamic behavior (such as moving speed, acceleration and directionality) of various objects within a radius of 200 meters of the mobile vehicle.

[0032] In step S230, the processing circuit 130 of the environmental sensing device 100 can fuse the three-dimensional image frame 111 and the three-dimensional scanning frame 121 to generate three-dimensional object information corresponding to the object 11 in the field 10. Generally speaking, compared with the camera module 110, the lidar module 120 is not suitable for use in fog. Compared with the lidar module 120, the camera module 110 is not suitable for use in dark places and / or backlit places. The environmental sensing device 100 can fuse the characteristics of the camera module 110 with the characteristics of the lidar module 120. Therefore, the environmental sensing device 100 can be unaffected by the day and night environment, thereby improving the recognition accuracy to enhance driving safety.

[0033] The database 140 is suitable for storing a three-dimensional map. In step S240, the processing circuit 130 can compare the three-dimensional object information corresponding to the object 11 with the three-dimensional map stored in the database 140. Based on the comparison result of step S240, the processing circuit 130 can decide whether to skip the analysis and calculation of the three-dimensional object information in step S250. For example, in the case where the three-dimensional object information has not been analyzed (that is, the object 11 has not been recorded in the three-dimensional map of the database), the processing circuit 130 can analyze and calculate the three-dimensional object information to obtain the motion characteristics of the object 11, and then record the three-dimensional object information and the analysis and calculation results in the database 140 to construct a three-dimensional map.

[0034] The meaning of "the processing circuit 130 can find the object 11 in the three-dimensional map" is that the position of the object 11 detected by the environment perception device 100 in the field 10 at the previous time is the same as the position of the object 11 detected by the environment perception device 100 in the field 10 at the current time. That is, the object 11 is a stationary object in the field 10. Therefore, the processing circuit 130 can compare the three-dimensional object information corresponding to the object 11 with the three-dimensional map stored in the database 140 in step S240 to determine whether the object 11 is a stationary object. In the case that the object 11 is not a stationary object, the processing circuit 130 can analyze and calculate the three-dimensional object information in step S250 to obtain the motion characteristics of the object 11. According to design requirements, the motion characteristics include at least one of the size, position, distance, speed, acceleration and direction of the object 11. This embodiment does not limit the calculation details of the analysis and calculation of the motion characteristics. For example, in some embodiments, the analysis and calculation may include an existing object motion characteristic calculation method or other analysis and calculation. In the case that the object 11 is a stationary object, the processing circuit 130 may skip analyzing and calculating the three-dimensional object information in step S250 .

[0035] There are different types of objects around the mobile vehicle, and different types of objects have different degrees of impact on the safety of driving. For example, object 11 (such as a vehicle or a pedestrian) may collide with the mobile vehicle in a short time. When object 11 is a large amount of weeds, "the mobile vehicle collides with object 11" is actually not a dangerous thing. The processing circuit 130 can classify the three-dimensional object information corresponding to the object 11 to determine whether the object 11 poses a threat to the mobile vehicle. The classification method can use the appearance features of different objects for identification. The processing circuit 130 can use the features of the object 11 to compare with the data pre-stored in the database 140. This database 140 can be a storage module coupled to the processing circuit 130. In other embodiments, the database 140 can be a database located in the cloud. After classification, the processing circuit 130 can determine whether the object 11 affects the safety of the mobile vehicle. For example, when the object 11 is grass that does not pose a danger to driving safety, the processing circuit 130 can ignore the object 11. The “ignore” means that computing resources are not consumed to estimate the movement trend of the object 11 (the possibility of the object 11 colliding with the moving vehicle).

[0036] On the contrary, if the classification of the object 11 will affect the safety of the mobile vehicle, the processing circuit 130 can perform the analysis and calculation to obtain the movement trend of the object 11 based on the movement characteristics. The processing circuit 130 can use the movement information of the mobile vehicle itself (including information such as the movement speed and the movement direction) and the movement information of the object 11 (motion characteristics, including information such as the movement speed and the movement direction) to estimate the possibility of collision between the object 11 and the mobile vehicle. If the estimated result shows that there is a possibility of collision between the object 11 and the mobile vehicle, the processing circuit 130 can drive the prompt unit 150 to trigger an obstacle warning so that the driver can be aware of the existence of the crisis.

[0037] The processing circuit 130 may decide whether to drive the prompt unit 150 to trigger an obstacle warning based on the motion trend. For example, when the motion trend of the object 11 indicates that "a collision with the object 11 will occur", the processing circuit 130 may trigger an obstacle warning. For example, when the distance from a mobile vehicle (such as a car, not shown) to an obstacle (object 11) is less than 100 meters and the object 11 moves parallel to the mobile vehicle, the processing circuit 130 may mark the obstacle in the screen of the display panel (prompt unit 150). When the distance from the mobile vehicle to the obstacle (object 11) is less than 10 meters and the motion trend of the object 11 is "approaching the mobile vehicle", the processing circuit 130 may mark the obstacle in the screen of the display panel (prompt unit 150) and drive the speaker (prompt unit 150) to emit a warning sound. When the distance from the mobile vehicle to the obstacle (object 11) is less than 5 meters and the movement trend of object 11 is "moving parallel to the mobile vehicle", the processing circuit 130 can mark the obstacle on the screen of the display panel (prompt unit 150) and drive the speaker (prompt unit 150) to emit an intermittent warning sound. When the distance from the mobile vehicle to the obstacle (object 11) is less than 2 meters and the movement trend of object 11 is "approaching the mobile vehicle", the processing circuit 130 can mark the obstacle on the screen of the display panel (prompt unit 150) and drive the speaker (prompt unit 150) to emit a continuous warning sound. In addition, if the mobile vehicle is automatically driven by a computer, the processing circuit 130 can trigger a corresponding avoidance mechanism to prevent the mobile vehicle from colliding with the object 11.

[0038] This embodiment mainly generates three-dimensional environmental information through deep image environment scanning and image fusion to perform high-precision three-dimensional scanning and recognition. Through the construction of a three-dimensional map of repeated paths, this embodiment can be applied to complex and crowded environments in urban areas and accurately provide obstacle avoidance warnings. Based on the fusion of the three-dimensional image frame 111 and the three-dimensional scanning frame 121, this embodiment can be applied to the entire environment without being easily affected by environmental noise, thereby improving the recognition accuracy.

[0039] Figure 3According to an embodiment of the present invention, Figure 1 FIG. 1 is a circuit block diagram of the processing circuit 130 . Figure 3 The camera module 110, the lidar module 120 and the processing circuit 130 can refer to Figure 1 The camera module 110, the lidar module 120 and the processing circuit 130 are described in detail, so they are not repeated here. Figure 3 In the illustrated embodiment, the processing circuit 130 includes a partition circuit 131, a partition circuit 132 and an object recognition circuit 133. The partition circuit 131 is coupled to the camera module 110 to receive the 3D image frame 111. The partition circuit 132 is coupled to the lidar module 120 to receive the 3D scanning frame 121.

[0040] Figure 4 It is a flowchart of an environment perception method according to another embodiment of the present invention. Figure 4 The steps S210, S220, S230 and S240 shown can refer to Figure 2 The relevant descriptions of steps S210, S220, S230 and S240 are not repeated here. Figure 4 In the illustrated embodiment, step S230 includes steps S231, S232, and S233. Figure 3 and Figure 4 . In step S231, the partitioning circuit 131 may perform a first algorithm to segment a first region corresponding to the object 11 from the three-dimensional image frame 111. The present embodiment does not limit the implementation details of the first algorithm. For example, the partitioning circuit 131 may perform a maximally stable extremal regions (MSER) ​​algorithm (first algorithm) to segment a first region corresponding to the object 11 from the three-dimensional image frame 111. In step S232, the partitioning circuit 132 may perform a second algorithm to segment a second region corresponding to the object 11 from the three-dimensional scanning frame 121. The present embodiment does not limit the implementation details of the second algorithm. For example, the partitioning circuit 132 may perform a maximally stable colour regions (MSCR) algorithm (second algorithm) to segment a second region corresponding to the object 11 from the three-dimensional scanning frame 121.

[0041] The object recognition circuit 133 is coupled to the partition circuit 131 and the partition circuit 132. In step S233, the object recognition circuit 133 may perform a fusion operation to fuse the first region and the second region that match each other into three-dimensional object information corresponding to the object 11. In some embodiments, the fusion operation may include an edge detection algorithm. The edge detection algorithm may process the first region and the second region to generate the first region after edge detection and the second region after edge detection. For example, the object recognition circuit 133 may perform Sobel edge detection to fuse the three-dimensional image frame 111 and the three-dimensional scanning frame 121. According to design requirements, the object recognition circuit 133 may use a Maximum Entropy Markov Model (MEMM) to fuse the three-dimensional image frame 111 and the three-dimensional scanning frame 121 to obtain the three-dimensional object information corresponding to the object 11. The object recognition circuit 133 may compare the three-dimensional object information with the three-dimensional map to determine whether the object 11 is a stationary object. In the case where the object 11 is not a stationary object, the object recognition circuit 133 may perform the analysis calculation on the three-dimensional object information. In the case where the object 11 is a stationary object, the object recognition circuit 133 may skip the analysis calculation on the three-dimensional object information.

[0042] According to different design requirements, the processing circuit 130, the partitioning circuit 131, the partitioning circuit 132 and (or) the object identification circuit 133 may be implemented in the form of hardware, firmware, software (i.e., program) or a combination of more than one of the above three.

[0043] In terms of hardware, the processing circuit 130, the partitioning circuit 131, the partitioning circuit 132 and / or the object identification circuit 133 may be implemented as a logic circuit on an integrated circuit. The functions of the processing circuit 130, the partitioning circuit 131, the partitioning circuit 132 and / or the object identification circuit 133 may be implemented as hardware using hardware description languages ​​(such as Verilog HDL or VHDL) or other suitable programming languages. For example, the functions of the processing circuit 130, the partitioning circuit 131, the partitioning circuit 132 and / or the object identification circuit 133 may be implemented in various logic blocks, modules and circuits in one or more controllers, microcontrollers, microprocessors, application-specific integrated circuits (ASICs), digital signal processors (DSPs), field programmable gate arrays (FPGAs) and / or other processing units.

[0044] In terms of software and / or firmware, the related functions of the processing circuit 130, the partitioning circuit 131, the partitioning circuit 132 and / or the object identification circuit 133 can be implemented as programming codes. For example, the processing circuit 130, the partitioning circuit 131, the partitioning circuit 132 and / or the object identification circuit 133 can be implemented using general programming languages ​​(such as C, C++ or assembly language) or other suitable programming languages. The programming codes can be recorded / stored in a recording medium. In some embodiments, the recording medium includes, for example, a read-only memory (ROM), a random access memory (RAM) and / or a storage device. The storage device includes a hard disk drive (HDD), a solid-state drive (SSD) or other storage devices. In other embodiments, the recording medium may include a "non-transitory computer readable medium". For example, tape, disk, card, semiconductor memory, programmable logic circuit, etc. can be used to implement the non-temporary computer-readable medium. A computer, a central processing unit (CPU), a controller, a microcontroller or a microprocessor can read and execute the programming code from the recording medium to implement the related functions of the processing circuit 130, the partition circuit 131, the partition circuit 132 and (or) the object recognition circuit 133. Moreover, the programming code can also be provided to the computer (or CPU) via any transmission medium (communication network or broadcast wave, etc.). The communication network is, for example, the Internet, a wired communication network, a wireless communication network or other communication medium.

[0045] In summary, the environment sensing device 100 described in the above embodiments is configured with a camera module 110 and a lidar module 120. The processing circuit 130 can fuse the three-dimensional information of the three-dimensional image frame 111 of the camera module 110 and the three-dimensional information of the three-dimensional scanning frame 121 of the lidar module 120 to generate three-dimensional object information corresponding to a certain object 11 in the field 10. Therefore, the processing circuit 130 can combine the characteristics of the camera module 110 and the lidar module 120 to perform environment perception. In some embodiments, the processing circuit 130 can compare the three-dimensional object information with the three-dimensional map stored in the database 140. In the case where the three-dimensional object information has not been analyzed (that is, the three-dimensional object information has not been recorded in the database 140), the processing circuit 130 can analyze and calculate the three-dimensional object information to obtain the motion characteristics of the object 11, and then record the three-dimensional object information and the analysis and calculation results in the database 140 to construct a three-dimensional map. Moving objects (people or cars) are generally not static objects in the three-dimensional map, so the processing circuit 130 can analyze and calculate the moving objects to obtain the motion characteristics of the objects. In the case where the object 11 is a static object, the processing circuit can skip the analysis and calculation of the static object 11 that has been analyzed before to reduce the duplication of calculations.

[0046] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An environment sensing device for a mobile vehicle, characterized in that: The environment sensing device comprises: A camera module, disposed on the mobile vehicle, adapted to capture a scene near the mobile vehicle to generate a three-dimensional image frame; A lidar module, disposed on the mobile vehicle, adapted to scan the field to generate a three-dimensional scanning frame; a database suitable for storing three-dimensional maps; and A processing circuit is coupled to the camera module and the lidar module to receive the three-dimensional image frame and the three-dimensional scanning frame, wherein the processing circuit is configured to fuse the three-dimensional image frame and the three-dimensional scanning frame to generate three-dimensional object information corresponding to the object in the field, the processing circuit compares the three-dimensional object information with the three-dimensional map to determine whether the object is a stationary object, and when the object is not the stationary object or when the object has not been recorded in the three-dimensional map, the processing circuit analyzes and calculates the three-dimensional object information to obtain the motion characteristics of the object, and when the object is the stationary object, the processing circuit skips the analysis and calculation of the three-dimensional object information, and the processing circuit includes: a first partitioning circuit, coupled to the camera module to receive the 3D image frame, wherein the first partitioning circuit is configured to perform a first algorithm to segment a first area corresponding to the object from the 3D image frame; A second partitioning circuit, coupled to the lidar module to receive the three-dimensional scanning frame, wherein the second partitioning circuit is configured to perform a second algorithm to segment a second area corresponding to the object from the three-dimensional scanning frame; and An object recognition circuit is coupled to the first partition circuit and the second partition circuit, wherein the object recognition circuit performs a fusion operation to fuse the first area and the second area that match each other into the three-dimensional object information, the object recognition circuit compares the three-dimensional object information with the three-dimensional map to determine whether the object is the stationary object, and when the object is not the stationary object, the object recognition circuit performs the analysis and calculation on the three-dimensional object information, and when the object is the stationary object, the object recognition circuit skips the analysis and calculation on the three-dimensional object information.

2. The environment sensing device according to claim 1, characterized in that: The camera module includes a depth camera, and the three-dimensional image frame includes a depth image frame.

3. The environment sensing device according to claim 1, characterized in that: The fusion operation includes an edge detection algorithm, and the edge detection algorithm is suitable for processing the first region and the second region to generate the first region after edge detection and the second region after edge detection.

4. The environment sensing device according to claim 1, characterized in that: The motion characteristic includes at least one of size, position, distance, speed, acceleration and direction.

5. The environment sensing device according to claim 1, characterized in that: The analysis calculation further obtains a movement trend of the object based on the movement characteristics, and the processing circuit further determines whether to trigger an obstacle warning based on the movement trend.

6. A method for environmental perception of a mobile vehicle, characterized in that: The environment perception method comprises: A camera module disposed on the mobile vehicle captures a scene near the mobile vehicle to generate a three-dimensional image frame; A LiDAR module disposed on the mobile vehicle scans the field to generate a three-dimensional scanning frame; The processing circuit fuses the three-dimensional image frame with the three-dimensional scanning frame to generate three-dimensional object information corresponding to the object in the field; The processing circuit compares the three-dimensional object information with a three-dimensional map stored in a database to determine whether the object is a stationary object; When the object is not the stationary object or has not been recorded in the three-dimensional map, the processing circuit analyzes and calculates the three-dimensional object information to obtain the motion characteristics of the object; In the case where the object is the stationary object, the processing circuit skips the analysis and calculation of the three-dimensional object information; The first partitioning circuit of the processing circuit performs a first algorithm to segment a first area corresponding to the object from the three-dimensional image frame; The second partitioning circuit of the processing circuit performs a second algorithm to segment a second area corresponding to the object from the three-dimensional scanning frame; The object recognition circuit of the processing circuit performs a fusion operation to fuse the first region and the second region that match each other into the three-dimensional object information; The object recognition circuit compares the three-dimensional object information with the three-dimensional map to determine whether the object is the stationary object; In the case where the object is not the stationary object, the object recognition circuit performs the analysis and calculation on the three-dimensional object information; and In the case where the object is the stationary object, the object recognition circuit skips the analysis calculation of the three-dimensional object information.

7. The environment perception method according to claim 6, characterized in that: The camera module includes a depth camera, and the three-dimensional image frame includes a depth image frame.

8. The environment perception method according to claim 6, characterized in that: The fusion operation includes an edge detection algorithm, and the edge detection algorithm is suitable for processing the first region and the second region to generate the first region after edge detection and the second region after edge detection.

9. The environment perception method according to claim 6, characterized in that: The motion characteristic includes at least one of size, position, distance, speed, acceleration and direction.

10. The environment perception method according to claim 6, characterized in that: The environment perception method further includes: The processing circuit performs the analysis calculation to obtain the movement trend of the object based on the movement characteristics, and The processing circuit determines whether to trigger an obstacle warning based on the movement trend.

Images