Target motion state detection method, device, mobile device, and storage medium

Abstract

Embodiments of the present application provide a target motion state detection method, device, mobile device and storage medium. The method comprises: obtaining three-dimensional laser point clouds of a current frame and a previous frame observed by a laser radar; performing rasterized point cloud coding on the three-dimensional laser point clouds of the current frame and the previous frame to obtain two-dimensional raster images of the two frames; estimating an optical flow of each raster in the two-dimensional raster image of the current frame by using a change in image intensity of each raster in the two-dimensional raster images of the two frames; obtaining an envelope box of the target on the two-dimensional raster image of the current frame, determining an optical flow intensity of the envelope box based on the optical flow of the rasters occupied by the envelope box; and detecting a motion state of the target based on the optical flow intensity of the envelope box. Embodiments of the present application do not perform target tracking on multiple frames. The embodiments avoid the dependence on target detection consistency and target tracking, determine the motion state of the target by fusing the optical flow intensity of each raster, and reduce the probability of dynamic and static estimation errors.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving, and more particularly to a method, device, mobile device, and storage medium for detecting the motion state of a target. Background Technology

[0002] With the continuous advancement of LiDAR and artificial intelligence technologies, the foundation for the rapid development of autonomous driving technology has been laid, leading to its widespread application in areas such as driverless taxis and advanced driver assistance systems. On the one hand, autonomous driving technology aims to free the driver's hands, making driving simpler and more convenient; on the other hand, it must ensure driving safety, guaranteeing the safety of the vehicle and its passengers, as well as the safety of vehicles and pedestrians in the environment. Therefore, this requires autonomous vehicles to be able to detect the movement of vehicles, pedestrians, and other targets in the environment. The challenge lies in how to efficiently and accurately detect the movement of targets in the environment.

[0003] Autonomous vehicles typically use LiDAR to collect point cloud data of a scene, and then use deep learning algorithms to identify targets within the point cloud. A common method for multi-target motion state estimation is to predict the position of a target in the current frame by combining its historical motion state with methods such as Kalman filtering. Then, methods such as Hungarian matching or KM matching are used to match the predicted target position with the target position in the current frame. For the matched targets, Kalman filtering is then used to weight the observed and predicted positions in the current frame to obtain the final output position for the current frame, thus completing target tracking. Based on target tracking, trajectory fitting is performed on the target's center point across the output positions of multiple frames to obtain motion information such as the target's direction and velocity.

[0004] In the process of realizing this invention, the inventors discovered at least the following problems in the related technology:

[0005] Existing technologies require high accuracy and consistency in target detection for each frame. Target tracking is achieved through matching; a match is only possible if the same target is accurately detected in both consecutive frames.

[0006] In practice, target detection may encounter errors such as multiple detections and false detections. This means that the targets detected in consecutive frames may not be the same object. If a forced match is performed, the motion state of the target will be incorrectly estimated.

[0007] If target detection and tracking are correct, existing technologies use center points or feature points to calculate the target's velocity when calculating its motion state, and then determine whether the target is static or dynamic based on the magnitude of the velocity. When the target shape changes significantly, or when the center point or feature point matching is incorrect, the calculation of the target velocity may also introduce large errors, which could lead to incorrect judgments about the target's motion state. Summary of the Invention

[0008] To address the issue that existing technologies require high accuracy and consistency in target motion state detection for each frame, and that detected targets in consecutive frames may not be the same object or exhibit significant shape changes, leading to inaccurate motion state detection, this invention provides a target motion state detection method, comprising:

[0009] Acquire the 3D laser point cloud of the current frame and the previous frame observed by the lidar;

[0010] The three-dimensional laser point clouds of the current frame and the previous frame are rasterized into point cloud encoding to obtain two consecutive two-dimensional raster images. The image intensity of each grid in the two-dimensional raster image of each frame is determined by the intensity of each point cloud in that grid.

[0011] By utilizing the changes in image intensity of each grid in the two consecutive frames of two-dimensional raster images, the optical flow of each grid in the current frame of two-dimensional raster image is estimated to represent the motion that has occurred in each grid in the current frame of two-dimensional raster image relative to the previous frame.

[0012] Obtain the bounding box of the target on the two-dimensional raster image of the current frame, and determine the optical flow intensity of the bounding box based on the optical flow of the raster occupied by the bounding box;

[0013] The motion state of the target is detected based on the optical flow intensity of the envelope.

[0014] In a second aspect, embodiments of the present invention provide a target motion state detection execution device, comprising:

[0015] The point cloud acquisition module is used to acquire the 3D laser point cloud of the current frame and the previous frame observed by the lidar.

[0016] The encoding module is used to perform rasterized point cloud encoding on the three-dimensional laser point cloud of the current frame and the previous frame to obtain two consecutive two-dimensional raster images, wherein the image intensity of each grid in the two-dimensional raster image of each frame is determined by the intensity of each point cloud in that grid.

[0017] The optical flow estimation module is used to estimate the optical flow of each grid in the current frame's two-dimensional raster image by utilizing the changes in image intensity of each grid in the two consecutive frames of the two-dimensional raster image, so as to represent the motion that has occurred in each grid in the current frame's two-dimensional raster image relative to the previous frame.

[0018] An optical flow intensity determination module is used to obtain the envelope of the target on the two-dimensional raster image of the current frame, and to determine the optical flow intensity of the envelope based on the optical flow of the raster occupied by the envelope;

[0019] A motion state detection module is used to detect the motion state of the target based on the optical flow intensity of the envelope.

[0020] Thirdly, an electronic device is provided, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the steps of the motion state detection method for a target according to any embodiment of the present invention.

[0021] Fourthly, embodiments of the present invention provide a mobile device, including a main body and an electronic device according to any embodiment of the present invention mounted on the main body.

[0022] Fifthly, embodiments of the present invention provide a storage medium storing a computer program thereon, characterized in that, when the program is executed by a processor, it implements the steps of a motion state detection method for a target of any embodiment of the present invention.

[0023] Sixthly, embodiments of the present invention also provide a computer program product that, when the computer program product is run on a computer, causes the computer to execute the motion state detection method of the target as described in any one of the embodiments of the present invention.

[0024] The beneficial effects of this invention are as follows: It provides a novel raster-level motion state estimation method that avoids target tracking across multiple consecutive frames. By encoding the 3D point cloud onto a 2D raster image through rasterized point cloud encoding, the dimensionality of the data is reduced, thereby reducing the computational load and improving the estimation speed. More importantly, it determines the image intensity of each raster in the 2D raster image, providing effective data for optical flow estimation. Furthermore, the motion state detection problem is transformed into a dense optical flow estimation problem, avoiding the requirement for consistency in target detection and the dependence on target tracking. The motion state of the target is determined by fusing the optical flow intensity of each raster within the envelope, reducing the probability of errors in dynamic and static estimation. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart of a target motion state detection method provided in an embodiment of the present invention;

[0027] Figure 2 This is an overall flowchart of a target motion state detection method provided in an embodiment of the present invention;

[0028] Figure 3 This is a schematic diagram of the structure of a target motion state detection system according to an embodiment of the present invention;

[0029] Figure 4 This is a schematic diagram of an embodiment of an electronic device for detecting the motion state of a target, provided by an embodiment of the present invention. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] Those skilled in the art will recognize that embodiments of this application can be implemented as a system, apparatus, device, method, or computer program product. Therefore, this disclosure can be specifically implemented in the following forms: entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0032] For ease of understanding, the technical terms used in this application are explained below:

[0033] The term "mobile device" as used in this application includes, but is not limited to, vehicles with L0-L5 autonomous driving technology levels as defined by the Society of Automotive Engineers International (SAE International) or the national standard "Classification of Driving Automation for Motor Vehicles".

[0034] In some embodiments, the mobile device may be a vehicle or robot having various functions, such as:

[0035] (1) Passenger transport function, such as family cars, buses, etc.;

[0036] (2) Cargo carrying function, such as ordinary trucks, box trucks, trailers, enclosed trucks, tank trucks, flatbed trucks, container trucks, dump trucks, special structure trucks, etc.

[0037] (3) Tool functions, such as logistics delivery vehicles, automated guided vehicles (AGVs), patrol vehicles, cranes, hoists, excavators, bulldozers, loaders, road rollers, loaders, off-road engineering vehicles, armored engineering vehicles, sewage treatment vehicles, sanitation vehicles, vacuum trucks, floor scrubbers, water sprinkler trucks, sweeping robots, food delivery robots, shopping guide robots, lawnmowers, golf carts, etc.

[0038] (4) Entertainment functions, such as recreational vehicles, amusement park automatic driving devices, balance bikes, etc.;

[0039] (5) Special rescue functions, such as fire trucks, ambulances, power repair vehicles, and engineering emergency rescue vehicles.

[0040] like Figure 1 The diagram shows a flowchart of a target motion state detection method according to an embodiment of the present invention, which includes the following steps:

[0041] S11: Obtain the 3D laser point cloud of the current frame and the previous frame observed by the lidar;

[0042] S12: Rasterize the three-dimensional laser point clouds of the current frame and the previous frame to obtain two consecutive two-dimensional raster images, wherein the image intensity of each grid in the two-dimensional raster image of each frame is determined by the intensity of each point cloud in that grid.

[0043] S13: Using the changes in image intensity of each grid in the two consecutive frames of two-dimensional grid images, estimate the optical flow of each grid in the current frame of two-dimensional grid images to represent the motion that has occurred in each grid in the current frame of two-dimensional grid images relative to the previous frame.

[0044] S14: Obtain the envelope of the target on the two-dimensional raster image of the current frame, and determine the optical flow intensity of the envelope based on the optical flow of the raster occupied by the envelope;

[0045] S15: Detect the motion state of the target based on the optical flow intensity of the envelope.

[0046] In this embodiment, the method can be applied to various types of mobile devices, each with its own application scenarios. For example, an automated floor cleaning robot can be used in industrial parks, shopping malls, and other similar areas. In an industrial park scenario, there are relatively more pedestrians at specific times, such as 8:00 AM to 12:00 PM and 5:00 PM to 7:00 PM, while there are relatively fewer pedestrians at other times. When there are many pedestrians, they intersect each other, and the same target pedestrian may be detected multiple times or incorrectly due to overlapping occlusions in consecutive frames, making it impossible to detect accurately. Similarly, in a shopping mall scenario, with many tourists, multiple targets of pedestrians may intersect, leading to multiple detections or incorrect detections. Another example is an automated delivery vehicle; target vehicles on the road intersect each other, also resulting in multiple detections or incorrect detections. In actual roads, targets can be pedestrians, other vehicles, or other obstacles that may be detected while driving on the road.

[0047] In step S11, as the mobile device equipped with a lidar moves, the lidar continuously observes the 3D lidar point cloud for each frame. Because this method avoids the requirements for target detection accuracy and consistency in existing technologies, it does not perform target tracking across multiple consecutive frames. This method acquires the 3D lidar point cloud PC of the current frame observed by the lidar in the scene. t The 3D laser point cloud PC of the previous frame t-1 .

[0048] For step S12, the 3D laser point cloud PC of the current frame is encoded using rasterized point cloud encoding. t The 3D laser point cloud PC of the previous frame t-1 Projecting onto the XY plane (a two-dimensional plane), two consecutive frames of bird's-eye view two-dimensional raster images of the point cloud are obtained. This two-dimensional raster image is a w*h single-channel image. Each pixel in the two-dimensional raster image corresponds to a raster, and the pixel's intensity value is the result of the point cloud encoding within the raster. The raster image is an image of the point cloud raster; one raster corresponds to one pixel, and the intensity of the raster is the intensity of a single pixel. The pixel intensity is the pixel's brightness value. For example, if 10 frames are observed using LiDAR, and the same pixel contains point clouds, then the pixel's grayscale value increases tenfold, making its color darker compared to other pixels.

[0049] As one implementation, the step of performing rasterized point cloud encoding on the three-dimensional laser point clouds of the current frame and the previous frame to obtain two consecutive two-dimensional raster images includes:

[0050] Based on the x-coordinate and y-coordinate of the 3D laser point cloud of each frame, and the length and width of the 2D raster image, the attribution relationship between the 3D laser point cloud of each frame and the 2D raster image is determined, and the set of pixels contained in the 2D raster image is obtained, wherein each pixel in the set of pixels corresponds to a raster in the 2D raster image.

[0051] The pixel set is encoded to determine the pixel intensity of each grid, and a single-channel bird's-eye view two-dimensional grid image of each frame is obtained through the pixel intensity of each grid.

[0052] In this embodiment, since the 3D laser point cloud acquired by the lidar is 3D, it needs to be converted into a 2D image. This is done through rasterized point cloud encoding. For the acquired 3D laser point cloud P... i (x i y i , z i ), through x i y i The value is projected onto the two-dimensional grid S i,j In a grid (where a single grid cell represents one pixel), let the length and width of the grid cell be a and b, respectively. Then P i With S i,j The attribution relationship can be expressed by the following formula:

[0053]

[0054] The two-dimensional grid S is obtained using the above formula. i,j The point set M can be encoded using the following formula to obtain a scalar I. i,j , will be I i,j The intensity of the two-dimensional grid:

[0055] I i，j =f(M)

[0056] f(M) represents the point cloud encoder, whose function is to encode the points in M ​​into a scalar. Let M... i (x i y i , z i Let be the i-th point cloud in M, and a special case of it can be represented by the following formula:

[0057] f(M)=∑z i

[0058] Finally, the raster is organized into a w×h single-channel two-dimensional raster image S (bird's-eye view raster image), with S pixels. i,j The intensity is I i,j For example, the higher the intensity of a pixel, the darker the color in a two-dimensional raster image.

[0059] For step S13, using the intensity changes of the two consecutive frames of two-dimensional raster images determined in step S12, the optical flow estimation algorithm is used to estimate the optical flow of each grid (pixel) in the current frame's bird's-eye view raster. Specifically, the optical flow estimation process can input the raster images of the two consecutive frames into an optical flow estimation convolutional neural network and output an optical flow image.

[0060] The optical flow L of each grid is obtained through the above optical flow estimation. i,j It can be expressed by the following formula:

[0061] L i，j =(u, v)

[0062] Where i represents the i-th row and j represents the j-th column. u represents the amount of motion of the grid relative to the previous frame in the row direction, and v represents the amount of motion of the grid relative to the previous frame in the column direction.

[0063] For step S14, this method does not perform target tracking across multiple consecutive frames; it only needs to detect the target's bounding box on the current frame's 2D raster image and determine the optical flow intensity of the bounding box based on the optical flow of the raster occupied by the bounding box. Specifically, the target's bounding box on the current frame's 2D raster image is determined by the target detection module.

[0064] Determine the grid cells occupied by the envelope frame on the two-dimensional grid image;

[0065] The optical flow intensity of the envelope is determined using the optical flow of all occupied grid cells.

[0066] The optical flow intensity of each occupied grid is determined by calculating the amount of motion in different directions corresponding to the optical flow of each occupied grid.

[0067] The average optical flow intensity of all occupied grid cells is taken as the optical flow intensity of the envelope, wherein the optical flow intensity of the envelope is positively correlated with the movement speed of the envelope.

[0068] In this embodiment, the target detection module can be used to obtain the target's bounding box on the two-dimensional raster image. It should be noted that other methods can also be used to obtain the target's bounding box on the two-dimensional raster image; the specific detection model will not be described in detail here. For example, if there are a total of n grids in the two-dimensional raster image, the bounding box of the detected target occupies n / 5 grids in the region from left to right and from bottom to top in the two-dimensional raster image.

[0069] The sum of the optical flow intensities of the n / 5 grid cells within the envelope, divided by the area of ​​the envelope, determines the average optical flow intensity of the grid cells occupied by the target, which is then used as the optical flow intensity of the envelope. The optical flow intensity LI of a single pixel is expressed by the following formula:

[0070]

[0071] It is represented by the movement speed of the pixel (each pixel corresponds to a grid), so the average intensity of the envelope is positively correlated with the speed of the envelope.

[0072] For step S15, after determining the optical flow intensity of the target's bounding box in the two-dimensional raster image, the target's motion state is determined. Specifically, the optical flow intensity of the bounding box is compared with a motion state threshold. When the optical flow intensity of the bounding box exceeds the motion state threshold, the target is dynamic; when the optical flow intensity of the bounding box does not exceed the motion state threshold, the target is static. The motion state threshold can be set to a static value, which can be determined by technicians based on their experience in judging the motion states of historical targets.

[0073] By comparing the optical flow intensity of the envelope with a motion state threshold, the target is determined to be dynamic when it exceeds the threshold and static otherwise. The overall process of the above steps is as follows: Figure 2 As shown.

[0074] As can be seen from this implementation, this method provides a novel raster-level motion state estimation approach that does not require target tracking across multiple consecutive frames. By encoding the 3D point cloud onto a 2D raster image through rasterized point cloud encoding, the dimensionality of the data is reduced, thereby reducing computational load and improving estimation speed. More importantly, it determines the image intensity of each raster in the 2D raster image, providing effective data for optical flow estimation. Furthermore, the motion state detection problem is transformed into a dense optical flow estimation problem, avoiding the requirement for consistency in target detection and the dependence on target tracking. The motion state of the target is determined by fusing the optical flow intensity of each raster within the envelope, reducing the probability of errors in both dynamic and static estimation.

[0075] like Figure 3 The diagram shown is a schematic diagram of a target motion state detection execution device provided in an embodiment of the present invention. The system can execute the target motion state detection method described in any of the above embodiments and is configured in a terminal.

[0076] This embodiment provides a target motion state detection execution device 10, which includes: a point cloud acquisition module 11, an encoding module 12, an optical flow estimation module 13, an optical flow intensity determination module 14, and a motion state detection module 15.

[0077] The system includes the following modules: Point Cloud Acquisition 11, which acquires the three-dimensional laser point clouds of the current and previous frames observed by the lidar; Encoding 12, which performs rasterized point cloud encoding on the three-dimensional laser point clouds of the current and previous frames to obtain two consecutive two-dimensional raster images, wherein the image intensity of each raster in each frame's two-dimensional raster image is determined by the intensity of each point cloud in that raster; Optical Flow Estimation 13, which estimates the optical flow of each raster in the current frame's two-dimensional raster image by utilizing the changes in the image intensity of each raster in the two consecutive two-dimensional raster images, to represent the motion that has occurred in each raster in the current frame's two-dimensional raster image relative to the previous frame; Optical Flow Intensity Determination 14, which obtains the target's envelope box on the current frame's two-dimensional raster image and determines the optical flow intensity of the envelope box based on the optical flow of the raster occupied by the envelope box; and Motion State Detection 15, which detects the motion state of the target based on the optical flow intensity of the envelope box.

[0078] Furthermore, the encoding module is used for:

[0079] Based on the x-coordinate and y-coordinate of the 3D laser point cloud of each frame, and the length and width of the 2D raster image, the attribution relationship between the 3D laser point cloud of each frame and the 2D raster image is determined, and the set of pixels contained in the 2D raster image is obtained, wherein each pixel in the set of pixels corresponds to a raster in the 2D raster image.

[0080] The pixel set is encoded to determine the pixel intensity of each grid, and a single-channel bird's-eye view two-dimensional grid image of each frame is obtained through the pixel intensity of each grid.

[0081] Furthermore, the optical flow intensity determination module is used for:

[0082] The target detection module determines the bounding box of the target on the two-dimensional raster image of the current frame;

[0083] Determine the grid cells occupied by the envelope frame on the two-dimensional grid image;

[0084] The optical flow intensity of the envelope is determined using the optical flow of all occupied grid cells.

[0085] Furthermore, the optical flow intensity determination module is also used for:

[0086] The optical flow intensity of each occupied grid is determined by calculating the amount of motion in different directions corresponding to the optical flow of each occupied grid.

[0087] The average optical flow intensity of all occupied grid cells is taken as the optical flow intensity of the envelope, wherein the optical flow intensity of the envelope is positively correlated with the movement speed of the envelope.

[0088] Furthermore, the motion state detection module is used for:

[0089] The optical flow intensity of the envelope is compared with a motion state threshold. When the optical flow intensity of the envelope exceeds the motion state threshold, the target is dynamic; when the optical flow intensity of the envelope does not exceed the motion state threshold, the target is static.

[0090] This invention also provides a non-volatile computer storage medium storing computer-executable instructions that can execute the target motion state detection method in any of the above method embodiments.

[0091] In one embodiment, the non-volatile computer storage medium of the present invention stores computer-executable instructions, which are configured as follows:

[0092] Acquire the 3D laser point cloud of the current frame and the previous frame observed by the lidar;

[0093] The three-dimensional laser point clouds of the current frame and the previous frame are rasterized into point cloud encoding to obtain two consecutive two-dimensional raster images. The image intensity of each grid in the two-dimensional raster image of each frame is determined by the intensity of each point cloud in that grid.

[0094] By utilizing the changes in image intensity of each grid in the two consecutive frames of two-dimensional raster images, the optical flow of each grid in the current frame of two-dimensional raster image is estimated to represent the motion that has occurred in each grid in the current frame of two-dimensional raster image relative to the previous frame.

[0095] Obtain the bounding box of the target on the two-dimensional raster image of the current frame, and determine the optical flow intensity of the bounding box based on the optical flow of the raster occupied by the bounding box;

[0096] The motion state of the target is detected based on the optical flow intensity of the envelope.

[0097] As a non-volatile computer-readable storage medium, it can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the methods in the embodiments of the present invention. One or more program instructions are stored in the non-volatile computer-readable storage medium, and when executed by a processor, the motion state detection method of the target in any of the above method embodiments is performed.

[0098] In some embodiments, the present invention also provides a computer program product that, when run on a computer, causes the computer to execute the motion state detection method of the target as described in any one of the embodiments of the present invention.

[0099] Figure 4 This is a schematic diagram of the hardware structure of an electronic device for a target motion state detection method provided in another embodiment of this application, as shown below. Figure 4 As shown, the device includes:

[0100] One or more processors 410 and memory 420, Figure 4 Taking a processor 410 as an example, the device for the target motion state detection method may further include an input device 430 and an output device 440.

[0101] The processor 410, memory 420, input device 430, and output device 440 can be connected via a bus or other means. Figure 4 Taking the example of a connection between China and Israel via a bus.

[0102] The memory 420, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the motion state detection method of the target in the embodiments of this application. The processor 410 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions, and modules stored in the memory 420, thereby realizing the motion state detection method of the target in the above-described method embodiments.

[0103] The memory 420 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; the data storage area may store data, etc. Furthermore, the memory 420 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 420 may optionally include memory remotely located relative to the processor 410, and these remote memories may be connected to the mobile device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0104] Input device 430 can receive input numerical or character information. Output device 440 may include display devices such as a display screen.

[0105] The one or more modules are stored in the memory 420, and when executed by the one or more processors 410, they execute the target motion state detection method in any of the above method embodiments.

[0106] The above-described product can perform the methods provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects for performing the methods. Technical details not described in detail in this embodiment can be found in the methods provided in the embodiments of this application.

[0107] Non-volatile computer-readable storage media may include a stored program area and a stored data area, wherein the stored program area may store an operating system and an application program required for at least one function; the stored data area may store data created based on the use of the device, etc. Furthermore, the non-volatile computer-readable storage medium may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the non-volatile computer-readable storage medium may optionally include memory remotely located relative to the processor, and these remote memories may be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0108] This invention also provides an electronic device comprising: at least one processor and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the steps of the motion state detection method for a target according to any embodiment of this invention.

[0109] The aforementioned electronic devices can exist in the form of an autonomous driving domain controller. Specifically, the autonomous driving domain controller communicates with various sensors (including but not limited to LiDAR, cameras, millimeter-wave radar, ultrasonic radar, inertial measurement units, wheel speed sensors, etc.) mounted on the mobile device. It acquires environmental perception data and vehicle speed information through these sensors, extracts obstacle information and generates the mobile device's position information based on the acquired environmental perception data and vehicle speed information, and performs path planning based on the position information and obstacle information.

[0110] The electronic device described in this application embodiment may also exist in various other forms, including but not limited to:

[0111] (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and primarily aim to provide voice and data communication. These terminals include smartphones, multimedia phones, feature phones, and low-end phones.

[0112] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers, possessing computing and processing capabilities, and generally also have mobile internet access features. These terminals include PDAs, MIDs, and UMPCs, such as tablet computers.

[0113] (3) Portable entertainment devices: These devices can display and play multimedia content. This category includes audio and video players, handheld game consoles, e-book readers, as well as smart toys and portable car navigation devices.

[0114] (4) Other mobile devices with data processing functions.

[0115] In some embodiments, the present invention also provides a mobile device including the electronic device described in any of the foregoing embodiments.

[0116] In this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, without necessarily requiring or implying any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising" or "including" include not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0117] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0118] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0119] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting the motion state of a target, comprising: Acquire the 3D laser point cloud of the current frame and the previous frame observed by the lidar; Rasterized point cloud encoding is performed on the 3D laser point clouds of the current frame and the previous frame to obtain two consecutive 2D raster images, specifically including: Based on the x and y coordinates of the 3D laser point cloud for each frame, and the length and width of the 2D raster image, the attribution relationship between the 3D laser point cloud and the 2D raster image for each frame is determined, resulting in the pixel set contained in the 2D raster image. Each pixel in the pixel set corresponds to a raster in the 2D raster image. The pixel set is encoded to determine the pixel intensity of each grid, and a single-channel bird's-eye view two-dimensional grid image of each frame is obtained through the pixel intensity of each grid. By utilizing the changes in image intensity of each grid in the two consecutive frames of two-dimensional raster images, the optical flow of each grid in the current frame of two-dimensional raster image is estimated to represent the motion that has occurred in each grid in the current frame of two-dimensional raster image relative to the previous frame. The target detection module determines the bounding box of the target on the two-dimensional raster image of the current frame, and determines the raster area occupied by the bounding box on the two-dimensional raster image. Determining the optical flow intensity of the envelope frame using the optical flow of all occupied grid cells specifically includes: The optical flow intensity of each occupied grid cell is determined by calculating the amount of motion in different directions corresponding to the optical flow of each occupied grid cell. The average optical flow intensity of all occupied grid cells is taken as the optical flow intensity of the envelope, wherein the optical flow intensity of the envelope is positively correlated with the movement speed of the envelope; Without performing target tracking across multiple consecutive frames, the motion state of the target is detected based on the optical flow intensity of the envelope.

2. The method according to claim 1, characterized in that, The detection of the target's motion state based on the optical flow intensity of the envelope includes: The optical flow intensity of the envelope is compared with a motion state threshold. When the optical flow intensity of the envelope exceeds the motion state threshold, the target is dynamic; when the optical flow intensity of the envelope does not exceed the motion state threshold, the target is static.

3. A target motion state detection execution device, used to implement the method as described in claim 1 or 2, the device comprising: The point cloud acquisition module is used to acquire the 3D laser point cloud of the current frame and the previous frame observed by the lidar. The encoding module is used to perform rasterized point cloud encoding on the three-dimensional laser point cloud of the current frame and the previous frame to obtain two consecutive two-dimensional raster images, wherein the image intensity of each grid in the two-dimensional raster image of each frame is determined by the intensity of each point cloud in that grid. The optical flow estimation module is used to estimate the optical flow of each grid in the current frame's two-dimensional raster image by utilizing the changes in image intensity of each grid in the two consecutive frames of the two-dimensional raster image, so as to represent the motion that has occurred in each grid in the current frame's two-dimensional raster image relative to the previous frame. An optical flow intensity determination module is used to obtain the envelope of the target on the two-dimensional raster image of the current frame, and to determine the optical flow intensity of the envelope based on the optical flow of all the grates occupied by the envelope; The motion state detection module is used to detect the motion state of the target based on the optical flow intensity of the envelope without performing target tracking for multiple consecutive frames.

4. An electronic device, comprising: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the steps of the method of claim 1 or 2.

5. A mobile device comprising the electronic device according to claim 4.

6. A storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method described in claim 1 or 2.

Images