Target detection method and device, electronic equipment and computer readable storage medium

Abstract

The invention provides a target detection method and device, electronic equipment and a computer readable storage medium. The method comprises the steps of performing target detection on to-be-processed data to obtain an initial detection result; when a first area corresponding to the initial detection result intersects with a preset area, obtaining point cloud data corresponding to the first area, and determining a target point cloud from the point cloud data; a projection coordinate range of the target point cloud projected to a first edge of the first area is determined, the projection coordinate range is updated based on the first edge, an updated projection coordinate range is obtained, and the first edge is the longest edge in the edges of the first area; adjusting the first area based on the updated projection coordinate range to obtain an updated first area; and determining a target detection result based on the updated first region. According to the invention, the target detection result with the rotation angle error can be adjusted.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and particularly relates to a target detection method and device, electronic equipment and computer readable storage medium. BACKGROUND

[0002] Three-dimensional target detection is an important research direction in the field of computer vision. Three-dimensional target detection is a technology for detecting and positioning objects in three-dimensional space. Three-dimensional point cloud data or image information is obtained through different sensors (such as radar, camera, etc.), and then target objects in the environment are identified and positioned. The three-dimensional target detection technology is widely used in automatic driving, robot navigation, augmented reality and other fields, and provides more accurate and comprehensive environmental perception capability.

[0003] In related technologies, the target detection frame obtained by three-dimensional target detection has inaccurate rotation angle or the target detection frame is larger than the actual point cloud range. The target detection result with rotation angle error cannot be adjusted, which affects subsequent intelligent navigation. SUMMARY

[0004] Embodiments of the present application provide a target detection method, device, electronic equipment and computer readable storage medium, which can adjust the target detection result with rotation angle error.

[0005] The technical scheme of the embodiments of the present application is as follows:

[0006] The embodiments of the present application provide a target detection method, which comprises:

[0007] Perform target detection on the to-be-processed data to obtain an initial detection result;

[0008] When the first region corresponding to the initial detection result intersects with a preset region, obtain point cloud data corresponding to the first region, and determine a target point cloud from the point cloud data, the target point cloud being a point cloud located inside the first region;

[0009] Determine a projection coordinate range of the target point cloud projected to a first edge of the first region, and update the projection coordinate range based on the first edge to obtain an updated projection coordinate range, the first edge being the longest edge among edges of the first region;

[0010] Adjust the first region based on the updated projection coordinate range to obtain an updated first region;

[0011] Determine a target detection result based on the updated first region.

[0012] The embodiments of the present application provide a target detection device, which comprises:

[0013] The first detection module is configured to perform target detection on the to-be-processed data to obtain an initial detection result.

[0014] The first determination module is configured to, when the first region corresponding to the initial detection result intersects with a preset region, acquire point cloud data corresponding to the first region, and determine a target point cloud from the point cloud data, the target point cloud being a point cloud located inside the first region.

[0015] The point cloud projection module is configured to determine a projection coordinate range of the target point cloud projected to a first edge of the first region, and update the projection coordinate range based on the first edge to obtain an updated projection coordinate range, the first edge being a longest edge among edges of the first region.

[0016] The region adjustment module is configured to adjust the first region based on the updated projection coordinate range to obtain an updated first region.

[0017] The second determination module is configured to determine a target detection result based on the updated first region.

[0018] An electronic device is provided in an embodiment of the present application, and the electronic device comprises:

[0019] A memory is configured to store computer executable instructions or computer programs.

[0020] A processor is configured to execute the computer executable instructions or computer programs stored in the memory to implement the target detection method provided in the embodiments of the present application.

[0021] A computer readable storage medium is provided in an embodiment of the present application, and the computer readable storage medium stores computer executable instructions or computer programs, and is configured to be executed by a processor to implement the target detection method provided in the embodiments of the present application.

[0022] A computer program product is provided in an embodiment of the present application, and the computer program product comprises computer executable instructions or computer programs, and the computer executable instructions or computer programs are executed by a processor to implement the target detection method provided in the embodiments of the present application.

[0023] The embodiments of the present application have the following beneficial effects:

[0024] In an embodiment of the present application, target detection is performed on the processed data to obtain an initial detection result; when a first area corresponding to the initial detection result intersects a preset area, point cloud data corresponding to the first area is obtained, and a target point cloud is determined from the point cloud data, where the target point cloud is a point cloud located within the first area; a projection coordinate range of the target point cloud projected onto a first side of the first area is determined, and the projection coordinate range is updated based on the first side to obtain an updated projection coordinate range, where the first side is the longest side among the sides of the first area. The optimal projection coordinate range can be determined by combining the point cloud within the first area corresponding to the initial detection result and the longest side, thereby optimizing the adjustment strategy of the target detection result using the optimal projection coordinate range; then, based on the updated projection coordinate range, the first area is adjusted to obtain an updated first area, and a target detection result is determined based on the updated first area. In this way, the initial detection result is dynamically adjusted based on the first side of the first area and the projection coordinate range of the target point cloud projected onto the first side to obtain a target detection result, thereby adjusting the target detection result with a rotation angle error, thereby avoiding the situation where the target detection result is larger than the actual point cloud range, thereby improving the accuracy of the target detection result. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 Schematic diagram of the application mode of the target detection method provided in the embodiment of the present application;

[0026] Figure 2 is a structural diagram of an electronic device provided in an embodiment of the present application;

[0027] Figure 3A This is a schematic diagram of the first flow chart of the target detection method provided in an embodiment of the present application;

[0028] Figure 3B This is a second flow chart of the target detection method provided in an embodiment of the present application;

[0029] Figure 3C 3 is a schematic diagram of a third flow chart of the target detection method provided in an embodiment of the present application;

[0030] Figure 3D 4 is a schematic diagram of a fourth flow chart of the target detection method provided in an embodiment of the present application;

[0031] Figure 4 Schematic diagram of the target detection frame provided in the embodiment of the present application;

[0032] Figure 5 is a schematic diagram of the first area and the preset area provided in an embodiment of the present application;

[0033] Figure 6 This is a schematic diagram of adjusting the first area provided in an embodiment of the present application.

[0034] It should be pointed out that the above-mentioned "first" and "second" are only used to distinguish different solutions, and do not represent the degree of distinction between the advantages and disadvantages of the solutions or the priority in the implementation process. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solutions and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limiting this application. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0036] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0037] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0038] The relevant data collection and processing in the embodiments of this application should be strictly in accordance with the requirements of relevant laws and regulations when applied in examples, and the informed consent or separate consent of the personal information subject should be obtained. Subsequent data use and processing should be carried out within the scope of authorization of laws and regulations and the personal information subject.

[0039] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0040] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meanings as those commonly understood by those skilled in the art. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0041] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0042] 1) Three-dimensional bounding box: is a cube box used to accurately locate the object in three-dimensional space, including the position, size, rotation angle, etc. of the object.

[0043] 2) Point cloud: is a data set composed of a large number of three-dimensional points, each point containing three-dimensional coordinates.

[0044] The embodiment of the present application provides a target detection method, device, electronic equipment and computer readable storage medium, which can adjust the target detection result with rotation angle error.

[0045] The following describes an exemplary application of the electronic device provided by the embodiment of the present application. The electronic device provided by the embodiment of the present application can be implemented as a notebook computer, a tablet computer, a desktop computer, a set-top box, a smart phone, a smart speaker, a smart watch, a smart television, a vehicle-mounted terminal and various types of terminals, and can also be implemented as a server. In the following, an exemplary application when the device is implemented as a server will be described.

[0046] Referring to Figure 1 , Figure 1 is an application mode schematic diagram of the target detection method provided by the embodiment of the present application, and the example, Figure 1 involved in the server 200, the network 300 and the terminal 400. In Figure 1 , the terminal 400 is taken as an example of a vehicle-mounted terminal, and the terminal 400 can also be a mobile intelligent robot. The terminal 400 connects the server 200 through the network 300, and the network 300 can be a wide area network or a local area network, or a combination of the two.

[0047] In the target detection process, the terminal 400 collects the to-be-processed data through the data acquisition device, and sends the to-be-processed data to the server 200. The server 200 performs target detection on the to-be-processed data to obtain an initial detection result. When a first region corresponding to the initial detection result intersects with a preset region, point cloud data corresponding to the first region is obtained, and a target point cloud is determined from the point cloud data. The target point cloud is a point cloud located inside the first region. A projection coordinate range of the target point cloud projected to a first edge of the first region is determined, and the projection coordinate range is updated based on the first edge to obtain an updated projection coordinate range. The first edge is the longest edge among edges of the first region. The first region is adjusted based on the updated projection coordinate range to obtain an updated first region. A target detection result is determined based on the updated first region. The server 200 determines a control instruction based on the target detection result. When the target detection result indicates that an obstacle exists in a preset range of the terminal 400, the control instruction can be a parking instruction, a lane changing instruction, or the like. Then, the control instruction is returned to the terminal 400. The terminal 400 performs a corresponding parking operation or lane changing operation based on the received control instruction. When the target detection method is performed by the server 200, the server 200 can use high-performance computing resources to run a complex model, thereby improving detection and decision-making accuracy.

[0048] In some embodiments, after the terminal 400 collects the to-be-processed data through the data acquisition device, the terminal 400 can also perform target detection on the to-be-processed data to obtain an initial detection result. When a first region corresponding to the initial detection result intersects with a preset region, point cloud data corresponding to the first region is obtained, and a target point cloud is determined from the point cloud data. The target point cloud is a point cloud located inside the first region. A projection coordinate range of the target point cloud projected to a first edge of the first region is determined, and the projection coordinate range is updated based on the first edge to obtain an updated projection coordinate range. The first edge is the longest edge among edges of the first region. The first region is adjusted based on the updated projection coordinate range to obtain an updated first region. A target detection result is determined based on the updated first region. Then, a control instruction is generated and executed based on the target detection result. When the target detection method is performed by the terminal 400, network delay can be avoided, which is suitable for high-speed dynamic scenarios (such as emergency obstacle avoidance). In addition, since the method does not depend on the network, the reliability is high, and thus the method is suitable for weak network environments such as tunnels and remote areas.

[0049] In some embodiments, the server (for example, the server 200) can be a standalone physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and basic cloud computing services such as big data and artificial intelligence platforms. The terminal 400 can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, a vehicle-mounted terminal, and the like, but is not limited thereto. The terminal and the server can be connected directly or indirectly through wired or wireless communication, which is not limited in the embodiments of the present application.

[0050] Referring to Figure 2 , Figure 2 is a structural schematic diagram of an electronic device provided by an embodiment of the present application, which can be a terminal or a server, Figure 2 The electronic device shown in FIG. 1 includes at least one processor 410, a memory 450, and at least one network interface 420. The various components in the electronic device are coupled together by a bus system 440. It can be understood that the bus system 440 is used to realize the connection and communication between the components. In addition to including a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, all the buses are marked as the bus system 440 in Figure 2 .

[0051] The processor 410 can be an integrated circuit chip with signal processing capability, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., wherein the general-purpose processor can be a microprocessor or any conventional processor.

[0052] The memory 450 can be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memories, hard disk drives, optical disk drives, and the like. The memory 450 can optionally include one or more storage devices physically located in proximity to the processor 410.

[0053] The memory 450 includes volatile memory or non-volatile memory, and can also include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), and the volatile memory can be random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0054] In some embodiments, the memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or a subset or superset thereof, which are exemplarily illustrated below.

[0055] The operating system 451 includes system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, and the like, for implementing various basic services and processing hardware-based tasks.

[0056] The network communication module 452 is used to communicate with other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including Bluetooth, Wireless Fidelity (WiFi), Universal Serial Bus (USB), and the like.

[0057] In some embodiments, the device provided by the embodiments of the present application can be realized in a software manner, Figure 2 The target detection device 455 stored in the memory 450 is shown, which can be software in the form of programs and plug-ins, including the following software modules: a first detection module 4551, a first determination module 4552, a point cloud projection module 4553, a region adjustment module 4554, and a second determination module 4555. These modules are logical, and thus can be combined or further split according to the implemented functions. The functions of each module will be described below.

[0058] The target detection method provided by the embodiments of the present application will be described in conjunction with an exemplary application and implementation of a server device provided by the embodiments of the present application.

[0059] Next, the target detection method provided by the embodiments of the present application will be described. To facilitate understanding of the target detection method provided by the embodiments of the present application, in the embodiments of the present application, a three-dimensional target detection scenario applied to automatic driving is taken as an example for description. In actual applications, the target detection method provided by the embodiments of the present application can also be applied to three-dimensional target detection scenarios such as robot navigation and augmented reality.

[0060] As described above, the electronic device implementing the target detection method of the embodiments of the present application can be a terminal, a server, or a combination of the two. Next, the target detection method provided by the embodiments of the present application will be described taking the electronic device as a server as an example. Referring to Figure 3A , Figure 3A is a first flowchart of the target detection method provided by the embodiments of the present application, which will be described in conjunction with Figure 3A the steps shown.

[0061] In step 301, target detection is performed on the to-be-processed data to obtain an initial detection result.

[0062] Here, the to-be-processed data includes image data and point cloud data. The image data includes multi-view images, and the multi-view images are acquired by a camera at the same time for a target object to be detected. The multi-view images include six camera images, which can be front, rear, left front, left rear, right front, and right rear images. The point cloud data is acquired by a laser radar, and the point cloud data refers to a set of vectors in a three-dimensional coordinate system, each point cloud containing three-dimensional coordinates. The point cloud data corresponding to the timestamp of the image data is acquired, that is, the image data and the point cloud data are acquired for the target object in the same environment.

[0063] The image data can be pre-processed for distortion correction and the like, and the point cloud data can be processed for filtering, downsampling, and the like. Then, the image data is processed by a convolutional neural network to extract features, to obtain image features. The point cloud data is processed by a point cloud processing network to extract features, to obtain point cloud features. An aerial view fusion algorithm is used to project the image features and the point cloud features to an aerial view perspective space, to obtain fusion features. Target detection is performed based on the fusion features to obtain an initial detection result. The initial detection result includes an initial three-dimensional target detection box, which includes center coordinates, size, rotation angle, category information (such as people, cars, trucks, etc.), and confidence and the like.

[0064] Continuing to refer to Figure 3A In step 302, when a first region corresponding to the initial detection result intersects with a preset region, point cloud data corresponding to the first region is acquired, and a target point cloud is determined from the point cloud data.

[0065] Here, the first region is a region composed of four vertices of the bottom of the initial three-dimensional target detection box in the initial detection result. For example, refer to Figure 4 , Figure 4 is a schematic diagram of a target detection box provided by an embodiment of the present application. The first region is composed of four vertices v0, v2, v4, and v6 of the bottom of the initial three-dimensional target detection box.

[0066] The preset region can be a region extending a preset distance forward, backward, left, and right (such as a region of -5-5 meters left and right and -10-10 meters forward and backward) with the laser radar as the center. When the first region intersects with the preset region, the point cloud data corresponding to the first region is acquired. The point cloud data corresponding to the first region is point cloud data excluding ground point cloud. The ground point cloud in the point cloud data is filtered by a ground filtering algorithm, and the point cloud data corresponding to the first region is obtained. The target point cloud located inside the first region is determined from the point cloud data.

[0067] In some embodiments, refer toFigure 3B , Figure 3B is a second flowchart of a target detection method provided by an embodiment of the present application, Figure 3A The step 302 shown as "determining a target point cloud from the point cloud data" can be implemented by Figure 3B The steps 3021 to 3022 are implemented as follows.

[0068] In the step 3021, the first vertex coordinates, the second vertex coordinates, the center coordinates, the rotation angle, the length and the width of the first region are determined.

[0069] Here, the first vertex coordinates, the second vertex coordinates, the center coordinates, the rotation angle, the length and the width of the first region are determined based on the center coordinates, the rotation angle, the length, the width and the height of the initial three-dimensional target detection box, respectively. For example, the center coordinates of the initial three-dimensional target detection box are (C x , C y , C z ), the rotation angle is γ, and the rotation matrix R is The length is l, the width is w, and the height is h. The first vertex coordinates are represented as R*(C x -w / 2, C y -l / 2, C z -h / 2), and the second vertex coordinates are represented as R*(C x +w / 2, C y -l / 2, C z -h / 2). The first vertex coordinates include the first depth coordinates, and the second vertex coordinates include the second depth coordinates, which are the third dimension z coordinates. The center coordinates of the first region are represented as (C x , C y , C z -h / 2), which include the center horizontal coordinates and the center vertical coordinates. The rotation angle of the first region is γ, the length of the first region is l, and the width of the first region is w.

[0070] For example, referring to Figure 4 , Figure 4 is a schematic diagram of a target detection box provided by an embodiment of the present application. The first region is composed of four vertices v0, v2, v4 and v6 at the bottom of the initial three-dimensional target detection box. The first vertex coordinates are the vertex coordinates of the vertex v0, and the second vertex coordinates are the vertex coordinates of the vertex v4.

[0071] In the step 3022, for each point cloud in the point cloud data, when the depth coordinates of the point cloud are greater than or equal to the first depth coordinates and the depth coordinates of the point cloud are less than or equal to the second depth coordinates, the horizontal coordinates and the vertical coordinates of the point cloud are updated based on the rotation angle, the center horizontal coordinates and the center vertical coordinates, respectively, to obtain the updated horizontal coordinates and the updated vertical coordinates of the point cloud.

[0072] Here, for each point cloud in the point cloud data, when the depth coordinate of the point cloud is greater than or equal to the first depth coordinate, and the depth coordinate of the point cloud is less than or equal to the second depth coordinate, it is preliminarily determined that the point cloud is located inside the first region, at which time the horizontal coordinate of the point cloud and the vertical coordinate of the point cloud are updated based on the rotation angle, the center horizontal coordinate and the center vertical coordinate to obtain the updated horizontal coordinate of the point cloud and the updated vertical coordinate of the point cloud, so as to further determine whether the point cloud is located inside the first region based on the updated horizontal coordinate of the point cloud and the updated vertical coordinate of the point cloud.

[0073] In some embodiments, based on the rotation angle, the center horizontal coordinate and the center vertical coordinate, the horizontal coordinate of the point cloud and the vertical coordinate of the point cloud are updated respectively to correspondingly obtain the updated horizontal coordinate of the point cloud and the updated vertical coordinate of the point cloud, which can be achieved by the following steps: determining a rotation matrix corresponding to the rotation angle; based on the center horizontal coordinate and the center vertical coordinate, respectively performing translation conversion on the horizontal coordinate of the point cloud and the vertical coordinate of the point cloud to correspondingly obtain a first translation amount and a second translation amount; based on the rotation matrix, performing rotation conversion on the first translation amount and the second translation amount to correspondingly obtain the updated horizontal coordinate of the point cloud and the updated vertical coordinate of the point cloud.

[0074] Here, based on the sine value and the cosine value of the rotation angle, a second-order matrix is constructed to obtain the rotation matrix. Based on the center horizontal coordinate, the horizontal coordinate of the point cloud is translated and converted, that is, the center horizontal coordinate and the horizontal coordinate of the point cloud are subtracted to obtain the first translation amount. Based on the center vertical coordinate, the vertical coordinate of the point cloud is translated and converted, that is, the center vertical coordinate and the vertical coordinate of the point cloud are subtracted to obtain the second translation amount. Based on the rotation matrix, the first translation amount is rotated and converted, that is, the rotation matrix and the first translation amount are multiplied to obtain the updated horizontal coordinate of the point cloud. Based on the rotation matrix, the second translation amount is rotated and converted, that is, the rotation matrix and the second translation amount are multiplied to obtain the updated vertical coordinate of the point cloud.

[0075] For example, the updated horizontal coordinate of the point cloud and the updated vertical coordinate of the point cloud can be determined by the following formula (1):

[0076]

[0077] Wherein, P'x represents the updated horizontal coordinate of the point cloud, P'y represents the updated vertical coordinate of the point cloud, γ represents the rotation angle, represents the rotation matrix, P x -C x represents the first translation amount, P y -C y represents the second translation amount, C x represents the center horizontal coordinate, C y represents the center vertical coordinate.

[0078] With reference to the foregoing Figure 3B In step 3023, a length threshold is determined based on the length, and a width threshold is determined based on the width.

[0079] Here, half of the length of the first region can be determined as the length threshold, and half of the width of the first region can be determined as the width threshold.

[0080] In step 3024, the point cloud whose absolute value of the updated horizontal coordinate is less than or equal to the width threshold, and whose absolute value of the updated vertical coordinate is less than or equal to the length threshold, is determined as the target point cloud.

[0081] Here, for each point cloud in the point cloud data, when the absolute value of the updated horizontal coordinate of the point cloud is less than or equal to the width threshold, and the absolute value of the updated vertical coordinate of the point cloud is less than or equal to the length threshold, the point cloud is determined as the target point cloud.

[0082] In the embodiments of the present application, based on the rotation angle, the center horizontal coordinate and the center vertical coordinate, the horizontal coordinate of the point cloud and the vertical coordinate of the point cloud are updated respectively, the updated horizontal coordinate of the point cloud and the updated vertical coordinate of the point cloud are obtained correspondingly, and the point cloud whose absolute value of the updated horizontal coordinate is less than or equal to the width threshold, and whose absolute value of the updated vertical coordinate is less than or equal to the length threshold, is determined as the target point cloud, which can realize the conversion of the point cloud from the world coordinate system to the local coordinate system of the target detection frame, so as to accurately determine the point cloud located in the target detection frame.

[0083] With reference to the foregoing Figure 3A In step 303, a projection coordinate range of the target point cloud projected onto the first edge of the first region is determined, and the projection coordinate range is updated based on the first edge to obtain an updated projection coordinate range.

[0084] Here, the first edge is the longest edge among the edges of the first region. The plurality of target point clouds are projected onto the first edge of the first region to obtain a plurality of projection coordinates. The minimum projection horizontal coordinate, the minimum projection vertical coordinate, the maximum projection horizontal coordinate and the maximum projection vertical coordinate are determined from the plurality of projection coordinates, and the projection coordinate range includes the minimum projection horizontal coordinate, the minimum projection vertical coordinate, the maximum projection horizontal coordinate and the maximum projection vertical coordinate. The minimum projection vertical coordinate and the maximum projection vertical coordinate in the projection coordinate range are updated based on the slope of the first edge of the first region to obtain the updated projection coordinate range.

[0085] In some embodiments, referring to Figure 3C , Figure 3C is a third flow diagram of the target detection method provided by the embodiments of the present application, Figure 3A The step 303 of "determining the projection coordinate range of the target point cloud projected onto the first edge of the first region" shown in the figure can be implemented by Figure 3CThe steps 3031 to 3035 of the method 3000 are implemented, which are explained in detail as follows.

[0086] In step 3031, the start point coordinate and the end point coordinate of the first side are determined, and based on the start point coordinate and the end point coordinate, the direction vector of the first side is determined.

[0087] Here, the left end point coordinate of the first side is determined as the start point coordinate, and the right end point coordinate of the first side is determined as the end point coordinate. The coordinates of corresponding dimensions in the start point coordinate and the end point coordinate are calculated by difference, to obtain the direction vector of the first side.

[0088] In some embodiments, based on the start point coordinate and the end point coordinate, the direction vector of the first side can be determined by the following steps: the difference between the second horizontal coordinate and the first horizontal coordinate is determined as the first vector component; the difference between the second vertical coordinate and the first vertical coordinate is determined as the second vector component; and the first vector component and the second vector component are used to construct the direction vector of the first side.

[0089] Here, the start point coordinate includes the first horizontal coordinate and the first vertical coordinate, and the end point coordinate includes the second horizontal coordinate and the second vertical coordinate. The second horizontal coordinate is subtracted from the first horizontal coordinate, and the obtained difference is determined as the first vector component. The second vertical coordinate is subtracted from the first vertical coordinate, and the obtained difference is determined as the second vector component. The first vector component and the second vector component are combined to form the direction vector of the first side.

[0090] For example, the first horizontal coordinate of the start point coordinate is lineStart x , the first vertical coordinate of the start point coordinate is lineStart y , the second horizontal coordinate of the end point coordinate is lineEnd x , the second vertical coordinate of the end point coordinate is lineEnd y , the first vector component dx is represented as (lineEnd x -lineStart x ), and the second vector component dy is represented as (lineEnd y -lineStart y ).

[0091] Continuing to refer to Figure 3C In step 3032, for each target point cloud, based on the point cloud coordinate of the target point cloud, the start point coordinate and the direction vector, the projection ratio of the target point cloud projected onto the first side is determined.

[0092] Here, the projection ratio is a ratio used to adjust the projection coordinate of the target point cloud.

[0093] In some embodiments, determining the projection ratio of the target point cloud projected on the first edge based on the point cloud coordinate of the target point cloud, the start point coordinate and the direction vector can be achieved by the following steps: determining a displacement vector from the start point of the first edge to the target point cloud based on the point cloud coordinate of the target point cloud and the start point coordinate; performing dot product processing on the displacement vector and the direction vector to obtain a dot product result; performing normalization processing on the dot product result based on the direction vector to obtain the projection ratio of the target point cloud projected on the first edge.

[0094] Here, the point cloud coordinate of the target point cloud and the corresponding dimension coordinate in the start point coordinate are calculated by difference to obtain the displacement vector from the start point of the first edge to the target point cloud. For example, the point cloud coordinate is represented as (P x ,P y ), the start point coordinate is represented as (lineStart x ,lineStart y ), and the displacement vector can be represented as {(P x -lineStart x ),(P y -lineStart y )}.

[0095] The displacement vector is dotted with the direction vector, i.e. the first vector component and the second vector component in the displacement vector and the direction vector are multiplied respectively to obtain the dot product result. For example, the dot product result can be represented as (P x -lineStart x )*dx+(P y -lineStart y )*dy.

[0096] The dot product result is normalized, i.e. the dot product result is divided by the modulus square of the direction vector to obtain the projection ratio of the target point cloud projected on the first edge. For example, the projection ratio can be represented as ((P x -lineStart x )*dx+(P y -lineStart y )*dy) / (dx*dx+dy*dy).

[0097] With reference to Figure 3C , in step 3033, the projection coordinate of the target point cloud is determined based on the start point coordinate, the projection ratio and the direction vector.

[0098] Here, the projection coordinate includes a projection horizontal coordinate and a projection vertical coordinate.

[0099] In some embodiments, based on the start point coordinates, the projection scale and the direction vector, determining the projection coordinates of the target point cloud can be achieved by the following steps: scaling the first vector component by the projection scale to obtain a first displacement; superimposing the first horizontal coordinate and the first displacement to obtain the projection horizontal coordinate of the target point cloud; scaling the second vector component by the projection scale to obtain a second displacement; superimposing the first vertical coordinate and the second displacement to obtain the projection vertical coordinate of the target point cloud.

[0100] Here, scaling the first vector component by the projection scale, that is, multiplying the projection scale by the first vector component, the product obtained is determined as the first displacement. Superimpose the first horizontal coordinate of the start point coordinates and the first displacement to obtain the projection horizontal coordinate of the target point cloud. Scale the second vector component by the projection scale, that is, multiply the projection scale by the second vector component, and the product obtained is determined as the second displacement. Superimpose the first vertical coordinate of the start point coordinates and the second displacement to obtain the projection vertical coordinate of the target point cloud. The projection coordinates of the target point cloud can be determined by the following formula (2):

[0101]

[0102] Where x represents the projection horizontal coordinate of the target point cloud, y represents the projection vertical coordinate of the target point cloud, lineStart x represents the first horizontal coordinate of the start point coordinates, lineStart y represents the first vertical coordinate of the start point coordinates, u represents the projection scale, dx represents the first vector component, and dy represents the second vector component.

[0103] Continuing to refer to Figure 3C In step 3034, the minimum value of the projection horizontal coordinates of the plurality of target point clouds is determined as the minimum projection horizontal coordinate, and the minimum value of the projection vertical coordinates of the plurality of target point clouds is determined as the minimum projection vertical coordinate.

[0104] Here, after obtaining the projection horizontal coordinates and the projection vertical coordinates of the plurality of target point clouds, the minimum value of the plurality of projection horizontal coordinates is determined as the minimum projection horizontal coordinate, and the minimum value of the plurality of projection vertical coordinates is determined as the minimum projection vertical coordinate.

[0105] In step 3035, the maximum value of the projection horizontal coordinates of the plurality of target point clouds is determined as the maximum projection horizontal coordinate, and the maximum value of the projection vertical coordinates of the plurality of target point clouds is determined as the maximum projection vertical coordinate.

[0106] Here, after obtaining the projection abscissa and ordinate of the plurality of target point clouds, the maximum value in the plurality of projection abscissas is determined as the maximum projection abscissa, and the maximum value in the plurality of projection ordinates is determined as the maximum projection ordinate.

[0107] In the embodiments of the present application, the projection coordinates of the target point cloud are determined based on the projection proportion of the target point cloud projected onto the first side, the starting point coordinates of the first side, and the direction vector of the first side, and the projection coordinate range is determined based on the projection coordinates of the plurality of target point clouds, so that the projection coordinate range can be determined in combination with the point cloud inside the first region corresponding to the initial detection result and the longest side, thereby dynamically adjusting the target detection result by using the projection coordinate range.

[0108] In some embodiments, the updated projection coordinate range includes the minimum projection abscissa, the updated minimum projection ordinate, the maximum projection abscissa, and the updated maximum projection ordinate. The projection coordinate range is updated to obtain the updated projection coordinate range, which can be achieved by the following steps: determining the slope of the first side as the ratio of the second vector component to the first vector component; determining the first projection ordinate on the straight line where the first side is located as the minimum projection abscissa is mapped to based on the slope and the starting point coordinates, and determining the first projection ordinate as the updated minimum projection ordinate; determining the second projection ordinate on the straight line where the first side is located as the maximum projection abscissa is mapped to based on the slope and the starting point coordinates, and determining the second projection ordinate as the updated maximum projection ordinate.

[0109] Here, the second vector component is divided by the first vector component, and the obtained ratio result is determined as the slope of the first side. The linear function of the straight line where the first side is located is constructed by using the slope and the starting point coordinates. The minimum projection abscissa is mapped to the straight line where the first side is located, that is, the minimum projection abscissa is input into the linear function to obtain the first projection ordinate, and at this time the first projection ordinate is determined as the updated minimum projection ordinate. The maximum projection abscissa is mapped to the straight line where the first side is located, that is, the maximum projection abscissa is input into the linear function to obtain the second projection ordinate, and at this time the second projection ordinate is determined as the updated maximum projection ordinate.

[0110] For example, the updated minimum projection ordinate and the updated maximum projection ordinate can be determined by the following formula (3):

[0111]

[0112] wherein y min represents the updated minimum projection ordinate, y max represents the updated maximum projection ordinate, x min represents the minimum projection abscissa, x maxrepresents the maximum projection abscissa, K represents the slope of the first side, and the slope is represented as (lineEnd y -lineStart y ) / (lineEnd x -lineStart x ).

[0113] In the embodiments of the present application, the projection coordinate range is updated based on the slope of the first side and the starting point coordinates of the first side, so that the optimal projection coordinate range can be determined in combination with the longest side of the first region corresponding to the initial detection result, thereby optimizing the adjustment strategy of the target detection result by using the optimal projection coordinate range.

[0114] With reference to Figure 3A , in step 304, the first region is adjusted based on the updated projection coordinate range to obtain an updated first region.

[0115] Here, the first side of the first region and the center coordinates of the first region are adjusted based on the minimum projection abscissa, the updated minimum projection ordinate, the maximum projection abscissa and the updated maximum projection ordinate in the updated projection coordinate range, respectively, to obtain the updated first region.

[0116] In some embodiments, with reference to Figure 3D , Figure 3D is a fourth flowchart of the target detection method provided by the embodiments of the present application, Figure 3A The step 304 shown in the figure can be implemented by the steps 3041 to 3043 of the fourth flowchart, which will be described in detail below. Figure 3D In step 3041, the average of the minimum projection abscissa and the maximum projection abscissa is determined as the updated center abscissa, and the average of the updated minimum projection ordinate and the updated maximum projection ordinate is determined as the updated center ordinate.

[0117] Here, the minimum projection abscissa and the maximum projection abscissa are summed and averaged, and the obtained average is determined as the updated center abscissa. The minimum projection ordinate and the updated maximum projection ordinate are summed and averaged, and the obtained average is determined as the updated center ordinate. In an example, the updated center abscissa and the updated center ordinate can be determined by the following formula (4):

[0118]

[0119]

[0120] wherein C′ x represents the updated center abscissa, C′ y represents the updated center ordinate, x min ​denotes the minimum projection horizontal coordinate, x max denotes the maximum projection horizontal coordinate, y min denotes the minimum projection vertical coordinate, y max denotes the maximum projection vertical coordinate.

[0121] In step 3042, a first length is determined based on the minimum projection horizontal coordinate, the maximum projection horizontal coordinate, the updated minimum projection vertical coordinate, and the updated maximum projection vertical coordinate.

[0122] Here, the Euclidean distance is calculated by using the minimum projection horizontal coordinate, the maximum projection horizontal coordinate, the updated minimum projection vertical coordinate, and the updated maximum projection vertical coordinate, and the obtained distance length is determined as the first length. In an example, the first length can be determined by the following formula (5):

[0123] dist = sqrt((x max -x min ) 2 +(y max -y min ) 2 ) (5)

[0124] wherein dist denotes the first length, x min denotes the minimum projection horizontal coordinate, x max denotes the maximum projection horizontal coordinate, y min denotes the minimum projection vertical coordinate, y max denotes the maximum projection vertical coordinate.

[0125] In step 3043, the first region is adjusted based on the first length, the updated center horizontal coordinate, and the updated center vertical coordinate to obtain an updated first region.

[0126] Here, the first region is reconstructed according to the first length, the updated center horizontal coordinate, and the updated center vertical coordinate to adjust the first region to obtain the updated first region.

[0127] In some embodiments, adjusting the first region based on the first length, the updated center horizontal coordinate, and the updated center vertical coordinate to obtain the updated first region can be achieved by the following steps: obtaining a region length and a region width of the first region; when the region length is less than or equal to the region width, determining the first length as an updated region width, and determining the updated first region based on the updated region width, the region length, the updated center horizontal coordinate, and the updated center vertical coordinate; when the region length is greater than the region width, determining the first length as an updated region length, and determining the updated first region based on the updated region length, the region width, the updated center horizontal coordinate, and the updated center vertical coordinate.

[0128] Here, the length and the width of the initial three-dimensional target detection box are determined as the region length and the region width of the first region respectively. When the region length is less than or equal to the region width, the region width of the first region is updated by using the first length, the first length is determined as the updated region width, and the first region is updated based on the updated region width, the region length, the updated center horizontal coordinate and the updated center vertical coordinate, to obtain an updated first region.

[0129] For example, the region length of the first region is 3, the region width is 5, the first length is 4, the updated center horizontal coordinate is 4, and the updated center vertical coordinate is 5. Since the region length is less than the region width, the updated region width is 4, the updated first region length is 3, the width is 4, and the center coordinate is (4, 5).

[0130] When the region length is greater than the region width, the region length of the first region is updated by using the first length, the first length is determined as the updated region length, and the first region is updated based on the updated region length, the region width, the updated center horizontal coordinate and the updated center vertical coordinate, to obtain an updated first region.

[0131] For example, the region length of the first region is 3, the region width is 2, the first length is 4, the updated center horizontal coordinate is 4, and the updated center vertical coordinate is 5. Since the region length is greater than the region width, the updated region length is 4, the updated first region length is 4, the width is 2, and the center coordinate is (4, 5).

[0132] In the embodiments of the present application, the first region is adjusted based on the first length, the updated center horizontal coordinate and the updated center vertical coordinate, the initial detection result is dynamically adjusted, the longest side of the initial detection result can be reduced to the range where the point cloud exists, the target detection result is obtained, and the situation that the target detection result is larger than the actual point cloud range is avoided.

[0133] With reference to Figure 3A In step 305, the target detection result is determined based on the updated first region.

[0134] Here, based on the updated first region, an updated region length, an updated region width, an updated center horizontal coordinate, and an updated center vertical coordinate are determined. The length and the width of the initial three-dimensional target detection box in the initial detection result are updated respectively by using the updated region length and the updated region width, to obtain an updated length and an updated width of the three-dimensional target detection box. The center coordinate of the initial three-dimensional target detection box in the initial detection result is updated by using the updated center horizontal coordinate, the updated center vertical coordinate, and the depth z coordinate of the first region, to obtain an updated center coordinate of the three-dimensional target detection box. Based on the updated length, the updated width, and the updated center coordinate of the three-dimensional target detection box, an updated three-dimensional target detection box is constructed, to obtain the target detection result.

[0135] In some embodiments, in the application scenario of automatic driving, after obtaining the target detection result, environment understanding and target classification can be first performed based on the target detection result, to distinguish categories such as vehicles, pedestrians, and obstacles, to track the motion trajectory of a dynamic target through multi-frame data association (such as Kalman filtering), and to identify drivable areas (such as lane lines and road shoulders), static obstacles (such as road stakes), and traffic signs (such as traffic lights). Then, behavior decision-making is performed based on the environment understanding, where the decision-making types include: following a vehicle, changing lanes, stopping, and emergency obstacle avoidance. Next, the behavior decision-making is converted into a specific motion trajectory, to obtain a planning result. Finally, the planning result is converted into a control instruction of a vehicle actuator.

[0136] Exemplarily, when it is determined based on the target detection result that there is a stationary vehicle 10 meters ahead, the target is classified as an “obstacle”, the behavior decision-making is to trigger a “lane changing” strategy (the left lane is free and no vehicle is coming), and the motion planning is to generate a left 3rd-degree polynomial trajectory, with the speed reduced to 20 km / h. The control instruction is a steering angle of +8° and a brake pressure of 30%, and after the lane changing is completed, the following mode is restored.

[0137] In some embodiments, in the application scenario of robot navigation, after obtaining the target detection result, target classification is also needed, and moving obstacles (such as pedestrians) and static obstacles (such as tables and chairs) can be distinguished through continuous frame matching, and deformable obstacles (such as curtains) and rigid obstacles (such as walls) can be identified through crossability analysis, so as to establish the spatial correlation (such as the positional relationship between the door frame and the door) between the obstacles. Then, the three-dimensional detection result is cross-modally aligned with the preloaded semantic map, real-time two-dimensional visual information (such as ground marking lines), and inertial measurement data (step height change), to form a unified environment representation, and then the navigation strategy decision is made based on the environment representation and the target classification, for example, if the target is a rigid obstacle, the navigation strategy can be conservative detouring, and if the target is a temporary obstacle, the navigation strategy decision can be a wait-evaluate cycle, then the three-dimensional trajectory optimization and real-time obstacle avoidance adjustment are performed based on the navigation strategy, to obtain the planning result, and finally the planning result is converted into a bottom-level control instruction, and exemplarily, the robot is a wheeled chassis, and the control instruction can be a differential instruction (left / right wheel speed ratio).

[0138] In some embodiments, the target detection method provided by the embodiments of the present application can be applied in the field of cloud technology. The target detection is performed on the to-be-processed data on a cloud platform to obtain an initial detection result; when a first region corresponding to the initial detection result intersects with a preset region, point cloud data corresponding to the first region is acquired, and a target point cloud is determined from the point cloud data, the target point cloud being a point cloud located inside the first region; a projection coordinate range of the target point cloud projected to a first edge of the first region is determined, and the projection coordinate range is updated based on the first edge to obtain an updated projection coordinate range, the first edge being the longest edge among edges of the first region, and the optimal projection coordinate range can be determined in combination with the point cloud inside the first region corresponding to the initial detection result and the longest edge, so as to optimize the adjustment strategy of the target detection result by using the optimal projection coordinate range, and then the first region is adjusted based on the updated projection coordinate range to obtain an updated first region, and the target detection result is determined based on the updated first region. In this way, the initial detection result is dynamically adjusted to obtain the target detection result by using the first edge of the first region and the projection coordinate range of the target point cloud projected to the first edge, the target detection result with a rotation angle error is adjusted, and the situation that the target detection result is larger than the actual point cloud range can be avoided.

[0139] In the following, the exemplary application of the target detection method provided by the embodiments of the present application in the three-dimensional target detection scenario of automatic driving will be described.

[0140] With the rapid development of autonomous driving and robotics, the accuracy and real-time requirements of environmental perception are increasingly high. Although traditional two-dimensional target detection can identify objects and their position information in images, objects have three-dimensional shapes and poses in the real three-dimensional world, so two-dimensional detection cannot meet the high-precision and comprehensive needs, while three-dimensional target detection can provide more accurate and comprehensive object position and shape information.

[0141] Three-dimensional target detection is an important research direction in the field of computer vision. Three-dimensional target detection is a technology for detecting and locating objects in three-dimensional space. It acquires three-dimensional point cloud data or image information through different sensors (such as radar, camera, etc.), and then identifies and locates target objects in the environment. Three-dimensional target detection technology is widely used in autonomous driving, robot navigation, augmented reality, etc., providing more accurate and comprehensive environmental perception capabilities.

[0142] The occlusion between objects, as well as the truncation of objects by image edges and the small number of pixels occupied by small targets in images, can affect the accuracy of three-dimensional target detection, and the learning of the rotation angle of objects also faces great difficulties. In scenarios with high real-time requirements such as autonomous driving, how to ensure detection accuracy and improve computational efficiency is a problem that needs to be solved. In related technologies, the target detection frame obtained by three-dimensional target detection has inaccurate rotation angle, or the target detection frame is larger than the actual point cloud range, which will affect subsequent intelligent navigation.

[0143] Embodiments of the present application address the problems in related technologies and propose a target detection method. Compared with related technologies, the following improvements are included:

[0144] For the target detection frame within the preset range, the ground-filtered point cloud is obtained, and the point cloud located inside the target detection frame (the target point cloud in the above embodiment) is determined. The internal point cloud is projected onto the long side (the first side in the above embodiment) of the target detection frame, and the long side is reduced to the range where the point cloud exists, thereby reducing the negative impact caused by inaccurate rotation angle of the target detection frame.

[0145] Next, the process of three-dimensional target detection is described. A bird's eye view fusion algorithm can be used for three-dimensional target detection. Input data is collected by cameras and laser radars, etc. The input data includes camera images and laser radar point clouds. The camera image is an RGB image from multiple perspectives (such as front, back, left, and right), providing rich semantic information. The laser radar point cloud is point cloud data in three-dimensional space, containing the geometric shape and position information of the object. The camera image and the laser radar point cloud are preprocessed, including coordinate transformation, normalization, etc. to ensure adaptation to the representation space of the bird's eye view. Specifically, distortion correction can be performed on the camera image, and filtering, downsampling, etc. can be performed on the laser radar point cloud.

[0146] Deep learning models such as Convolutional Neural Network (CNN) or self-attention model are used to extract image features from camera images. Point cloud features of the laser radar point cloud are extracted through a point cloud processing network. The features from different sensors are projected into a Bird's Eye View (BEV) space, specifically, the image features are projected on a ray in the BEV space and dispersed along the ray into multiple discrete points to form an image feature point cloud. The point cloud features are directly projected into the BEV space by the laser radar, and the image feature point cloud and the point cloud features are fused into unified BEV features using a BEV pooling operation.

[0147] A convolution-based BEV encoder is used to preprocess the unified BEV features to alleviate local misalignment between different features. A task-specific head (such as a detection head) is applied on the fused BEV features to perform three-dimensional object detection, and an object detection box is output, which includes center coordinates (C x ,C y ,C z ), size (width, length, height), rotation angle γ, category information (such as person, car, truck, etc.), and confidence.

[0148] In the case of limited road width, the deflection of the object detection box may cause the subsequent navigation module to fail to plan a suitable path. In the embodiments of the present application, the range of the point cloud within the object detection box is obtained to further reduce the length of the object detection box. For example, referring to Figure 4 , Figure 4 is a schematic diagram of an object detection box provided by the embodiments of the present application. The object detection box 401 includes eight vertices v0-v7.

[0149] The vertex coordinates of the vertex v0 are R*(C x -w / 2,C y -l / 2,C z -h / 2).

[0150] The vertex coordinates of the vertex v1 are R*(C x -w / 2,C y -l / 2,C z +h / 2).

[0151] The vertex coordinates of the vertex v2 are R*(C x -w / 2,C y +l / 2,C z -h / 2).

[0152] The vertex coordinates of the vertex v3 are R*(C x -w / 2,Cy + l / 2, C z + h / 2).

[0153] The vertex coordinates of the vertex v4 are R*(C x + l / 2, C y - l / 2, C z + h / 2).

[0154] The vertex coordinates of the vertex v5 are R*(C x + l / 2, C y - l / 2, C z + h / 2).

[0155] The vertex coordinates of the vertex v6 are R*(C x + l / 2, C y - l / 2, C z + h / 2).

[0156] The vertex coordinates of the vertex v7 are R*(C x + l / 2, C y - l / 2, C z + h / 2).

[0157] Wherein, * represents matrix multiplication, R represents a rotation matrix, and is represented as Wherein, γ represents a rotation angle of the target detection frame, w represents a width of the target detection frame, l represents a length of the target detection frame, and h represents a height of the target detection frame.

[0158] The four vertices v0, v2, v4 and v6 at the bottom of the target detection frame form a first region, when the first region has an intersection with a preset region, a point cloud after ground filtering corresponding to the target detection frame and the same timestamp is acquired, wherein the point cloud is acquired by a laser radar sensor, and a ground point cloud in the point cloud is excluded by using a ground filtering algorithm, so as to focus on the point cloud of the target detection frame itself. The preset region can be a region extending a preset distance forward, backward, left and right (such as a region of -5-5 meters left and right, -10-10 meters forward and backward) with the laser radar as the center. For example, refer to Figure 5 , Figure 5 is a schematic diagram of the first region and the preset region provided by the embodiment of the application. The first region 501 has an intersection with the preset region 502.

[0159] Each point cloud in the point cloud after ground filtering is traversed to determine a target point cloud located in the first region. The coordinates of each point cloud are represented as (P x , P y , P z ), when P z is less than the depth coordinate z of the vertex v0, or P zIf it is greater than the depth coordinate z of vertex v4, the point cloud is not inside the first area. Otherwise, the horizontal coordinate and the vertical coordinate of the updated point cloud are determined by the following formula (1):

[0160]

[0161] Wherein, P′x represents the horizontal coordinate of the updated point cloud, and P′y represents the vertical coordinate of the updated point cloud.

[0162] When the absolute value of P′x is less than or equal to w / 2, and the absolute value of P′y is less than or equal to l / 2, the point cloud is inside the target detection box.

[0163] In this embodiment of the present application, when adjusting the long side of the first region (corresponding to the first side in other embodiments), the height of the target detection frame is not considered, and the long side of the first region is determined only based on the four vertices v0, v2, v4, and v6. When the length of the first region is less than or equal to the width, the width of the first region is the long side. In this case, the starting and ending coordinates of the long side are:

[0164] lineStart={(v 0x +v 2x ) / 2,(v 0y +v 2y ) / 2}

[0165] lineEnd={(v 6x +v 4x ) / 2,(v 6y +v 4y ) / 2}

[0166] Among them, lineStart represents the starting coordinate of the long side, lineEnd represents the ending coordinate of the long side, and v 0x Represents the horizontal coordinate of vertex v0, v 0y Represents the vertical coordinate of vertex v0, v 2x Represents the horizontal coordinate of vertex v2, v 2y Represents the vertical coordinate of vertex v2, v 4x Represents the horizontal coordinate of vertex v4, v 4y Indicates the vertical coordinate of vertex v4, v 6x Indicates the horizontal coordinate of vertex v6, v 6y Represents the vertical coordinate of vertex v6.

[0167] When the length of the first region is greater than the width, the length of the first region is the long side. The starting and ending coordinates of the long side are:

[0168] lineStart={(v 0x +v 2x ) / 2,(v0y + v 4y ) / 2

[0169] lineEnd = {(v 2x + v 6x ) / 2, (x 2y + v 6y ) / 2}

[0170] Then the target point cloud is projected onto the long side, and the projection coordinates of the target point cloud can be determined by the following formula (2):

[0171]

[0172] Wherein, x represents the projection horizontal coordinate of the target point cloud, y represents the projection vertical coordinate of the target point cloud, lineStart x represents the horizontal coordinate of the starting point of the long side, lineStart y represents the vertical coordinate of the starting point of the long side, lineEnd x represents the horizontal coordinate of the end point of the long side, lineEnd y represents the vertical coordinate of the end point of the long side, u represents the projection ratio ((P x -lineStart x )*dx+(P y -lineStart y )*dy) / (dx*dx+dy*dy), dx represents the first vector component (lineEnd x -lineStart x ), and dy represents the second vector component (lineEnd y -lineStart y ).

[0173] The minimum projection horizontal coordinate, the minimum projection vertical coordinate, the maximum projection horizontal coordinate and the maximum projection vertical coordinate are determined from the projection coordinates of the plurality of target point clouds, and the minimum projection vertical coordinate and the maximum projection vertical coordinate are updated, and the updated minimum projection vertical coordinate and the updated maximum projection vertical coordinate can be determined by the following formula (3):

[0174]

[0175] Wherein, y min represents the updated minimum projection vertical coordinate, y max represents the updated maximum projection vertical coordinate, x min represents the minimum projection horizontal coordinate, x max represents the maximum projection horizontal coordinate, and K represents the slope of the long side, which is (lineEnd ylineStart y ) lineEnd x lineStart x ).

[0176] The center coordinate of the first region and the long side of the first region are updated by using the minimum projection abscissa, the updated minimum projection ordinate, the maximum projection abscissa and the updated maximum projection ordinate.

[0177] According to an example, the updated center coordinate of the first region can be determined by the following formula (4):

[0178]

[0179] Wherein, C'x represents the updated center abscissa of the first region, and C'y represents the updated center ordinate of the first region.

[0180] According to an example, the length of the long side of the updated first region can be determined by the following formula (5):

[0181] dist = sqrt((x max -x min ) 2 +(y max -y min ) 2 ) (5)

[0182] Wherein, dist represents the length of the long side of the updated first region.

[0183] At this time, when the length of the first region is less than or equal to the width, the width of the first region is adjusted by using the length of the long side of the updated first region. When the length of the first region is greater than the width, the length of the first region is adjusted by using the length of the long side of the updated first region to obtain the updated first region. According to an example, refer to Figure 6 , Figure 6 is a schematic diagram for adjusting the first region provided by the embodiment of the application. The initial first region 601 is adjusted to obtain the updated first region 602.

[0184] Finally, the target detection frame is reconstructed based on the updated first region to determine the final target detection result.

[0185] In the above three-dimensional target detection scene of automatic driving, by retaining the point cloud inside the target detection frame (not containing the ground point cloud), the internal point cloud is projected onto the long side of the target detection frame, the long side is reduced to the range where the point cloud exists, which can reduce the situation that the target detection frame occupies the road due to angle deviation, and avoid affecting the subsequent navigation of automatic driving.

[0186] The following continues to illustrate an example structure of the target detection apparatus 455 provided by the embodiments of the present application, which is implemented as a software module. In some embodiments, as shown in FIG. 4, the software module stored in the target detection apparatus 455 of the memory 450 can include: a first detection module 4551 configured to perform target detection on the to-be-processed data to obtain an initial detection result; a first determination module 4552 configured to, when the first region corresponding to the initial detection result intersects with a preset region, acquire point cloud data corresponding to the first region, and determine a target point cloud from the point cloud data, the target point cloud being a point cloud located inside the first region; a point cloud projection module 4553 configured to determine a projection coordinate range of the target point cloud projected to a first edge of the first region, and update the projection coordinate range based on the first edge to obtain an updated projection coordinate range, the first edge being a longest edge among edges of the first region; a region adjustment module 4554 configured to adjust the first region based on the updated projection coordinate range to obtain an updated first region; and a second determination module 4555 configured to determine a target detection result based on the updated first region. Figure 2

[0187] In some embodiments, the first determination module 4552 is further configured to determine a first vertex coordinate, a second vertex coordinate, a center coordinate, a rotation angle, a length, and a width of the first region, the first vertex coordinate including a first depth coordinate, the second vertex coordinate including a second depth coordinate, and the center coordinate including a center horizontal coordinate and a center vertical coordinate; for each point cloud in the point cloud data, when a depth coordinate of the point cloud is greater than or equal to the first depth coordinate and less than or equal to the second depth coordinate, update a horizontal coordinate of the point cloud and a vertical coordinate of the point cloud based on the rotation angle, the center horizontal coordinate, and the center vertical coordinate, to obtain an updated horizontal coordinate of the point cloud and an updated vertical coordinate of the point cloud, respectively; determine a length threshold based on the length, and determine a width threshold based on the width; and determine the point cloud as the target point cloud when an absolute value of the updated horizontal coordinate is less than or equal to the width threshold and an absolute value of the updated vertical coordinate is less than or equal to the length threshold.

[0188] In some embodiments, the first determination module 4552 is further configured to determine a rotation matrix corresponding to the rotation angle; perform translation conversion on the horizontal coordinate of the point cloud and the vertical coordinate of the point cloud based on the center horizontal coordinate and the center vertical coordinate, to obtain a first translation amount and a second translation amount; and perform rotation conversion on the first translation amount and the second translation amount based on the rotation matrix, to obtain the updated horizontal coordinate of the point cloud and the updated vertical coordinate of the point cloud.

[0189] ​In some embodiments, the projection coordinate range includes a minimum projection horizontal coordinate, a minimum projection vertical coordinate, a maximum projection horizontal coordinate, and a maximum projection vertical coordinate, the point cloud projection module 4553 is further configured to determine a start point coordinate and an end point coordinate of the first side, and determine a direction vector of the first side based on the start point coordinate and the end point coordinate; for each target point cloud, determine a projection ratio of the target point cloud projected onto the first side based on the point cloud coordinate of the target point cloud, the start point coordinate, and the direction vector; determine a projection coordinate of the target point cloud based on the start point coordinate, the projection ratio, and the direction vector, the projection coordinate including a projection horizontal coordinate and a projection vertical coordinate; determine the minimum projection horizontal coordinate as a minimum value of the projection horizontal coordinates of the plurality of target point clouds, and determine the minimum projection vertical coordinate as a minimum value of the projection vertical coordinates of the plurality of target point clouds; determine the maximum projection horizontal coordinate as a maximum value of the projection horizontal coordinates of the plurality of target point clouds, and determine the maximum projection vertical coordinate as a maximum value of the projection vertical coordinates of the plurality of target point clouds.

[0190] In some embodiments, the start point coordinate includes a first horizontal coordinate and a first vertical coordinate, the end point coordinate includes a second horizontal coordinate and a second vertical coordinate, and the point cloud projection module 4553 is further configured to determine a first vector component as a difference between the second horizontal coordinate and the first horizontal coordinate, determine a second vector component as a difference between the second vertical coordinate and the first vertical coordinate, and construct the direction vector of the first side using the first vector component and the second vector component.

[0191] In some embodiments, the point cloud projection module 4553 is further configured to determine a displacement vector from the start point of the first side to the target point cloud based on the point cloud coordinate of the target point cloud and the start point coordinate, perform dot product processing on the displacement vector and the direction vector to obtain a dot product result, and perform normalization processing on the dot product result based on the direction vector to obtain the projection ratio of the target point cloud projected onto the first side.

[0192] In some embodiments, the point cloud projection module 4553 is further configured to scale the first vector component using the projection ratio to obtain a first displacement amount, superimpose the first horizontal coordinate and the first displacement amount to obtain the projection horizontal coordinate of the target point cloud, scale the second vector component using the projection ratio to obtain a second displacement amount, and superimpose the first vertical coordinate and the second displacement amount to obtain the projection vertical coordinate of the target point cloud.

[0193] In some embodiments, the updated projection coordinate range includes a minimum projection horizontal coordinate, an updated minimum projection vertical coordinate, a maximum projection horizontal coordinate, and an updated maximum projection vertical coordinate, and the point cloud projection module 4553 is further configured to determine a ratio of the second vector component to the first vector component as a slope of the first side; determine, based on the slope and the start point coordinate, a first projection vertical coordinate that maps the minimum projection horizontal coordinate to a straight line on which the first side is located, and determine the first projection vertical coordinate as the updated minimum projection vertical coordinate; and determine, based on the slope and the start point coordinate, a second projection vertical coordinate that maps the maximum projection horizontal coordinate to the straight line on which the first side is located, and determine the second projection vertical coordinate as the updated maximum projection vertical coordinate.

[0194] In some embodiments, the region adjustment module 4554 is further configured to determine a mean value of the minimum projection horizontal coordinate and the maximum projection horizontal coordinate as an updated center horizontal coordinate, and determine a mean value of the updated minimum projection vertical coordinate and the updated maximum projection vertical coordinate as an updated center vertical coordinate; determine, based on the minimum projection horizontal coordinate, the maximum projection horizontal coordinate, the updated minimum projection vertical coordinate, and the updated maximum projection vertical coordinate, a first length; and adjust the first region based on the first length, the updated center horizontal coordinate, and the updated center vertical coordinate to obtain an updated first region.

[0195] In some embodiments, the region adjustment module 4554 is further configured to obtain a region length and a region width of the first region; when the region length is less than or equal to the region width, determine the first length as an updated region width, and determine the updated first region based on the updated region width, the region length, the updated center horizontal coordinate, and the updated center vertical coordinate; and when the region length is greater than the region width, determine the first length as an updated region length, and determine the updated first region based on the updated region length, the region width, the updated center horizontal coordinate, and the updated center vertical coordinate.

[0196] An embodiment of the present application provides a computer program product, which includes computer executable instructions or computer programs stored in a computer readable storage medium. A processor of an electronic device reads the computer executable instructions or computer programs from the computer readable storage medium, and the processor executes the computer executable instructions or computer programs, so that the electronic device executes a target detection method provided by an embodiment of the present application.

[0197] An embodiment of the present application provides a computer readable storage medium storing computer executable instructions, wherein the computer executable instructions or computer programs are stored in the computer readable storage medium, and when the computer executable instructions or computer programs are executed by a processor, the processor will execute a target detection method or a control method provided by an embodiment of the present application, for example, as shown in the following table:Figure 3A The illustrated target detection method.

[0198] In some embodiments, the computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, a magnetic surface memory, an optical disk, or a CD-ROM, etc.; or can be various devices including one or any combination of the above memories.

[0199] In some embodiments, the computer-executable instructions can be in the form of programs, software, software modules, scripts, or code, written in any suitable programming language (including compiled or interpreted languages, or declarative or procedural languages), and can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0200] By way of example, the computer-executable instructions can, but need not, reside in a file system's files, can be stored in documents, for example, Hyper Text Markup Language (HTML) documents, in one or more scripts stored in a file or files dedicated to the program in question, or in multiple cooperating files (e.g., files implementing one or more modules, sub programs, or code portions).

[0201] By way of example, the computer-executable instructions can be deployed to be executed on one electronic device or on multiple electronic devices that are located at one site or that are distributed across multiple sites and that are interconnected by a communication network.

[0202] The above description is merely illustrative of the embodiments of the present application and is not intended to limit the scope of the present application. Any modification, equivalent replacement, and improvement made within the spirit and scope of the present application shall fall within the scope of the present application.

Claims

1. A target detection method, characterized in that: The method comprises: Perform target detection on the data to be processed to obtain initial detection results; When the first area corresponding to the initial detection result intersects with the preset area, acquiring point cloud data corresponding to the first area, and determining a target point cloud from the point cloud data, where the target point cloud is a point cloud located inside the first area; determining a projection coordinate range of a first side of the first region projected by the target point cloud, and updating the projection coordinate range based on the first side to obtain an updated projection coordinate range, where the first side is the longest side among the sides of the first region; Adjusting the first area based on the updated projection coordinate range to obtain an updated first area; An object detection result is determined based on the updated first area.

2. The method according to claim 1, characterized in that Determining a target point cloud from the point cloud data includes: Determine first vertex coordinates, second vertex coordinates, center coordinates, a rotation angle, a length, and a width of the first area, wherein the first vertex coordinates include a first depth coordinate, the second vertex coordinates include a second depth coordinate, and the center coordinates include a central abscissa and a central ordinate; For each point cloud in the point cloud data, when the depth coordinate of the point cloud is greater than or equal to the first depth coordinate and the depth coordinate of the point cloud is less than or equal to the second depth coordinate, updating the abscissa and the ordinate of the point cloud based on the rotation angle, the central abscissa, and the central ordinate, respectively, to obtain an updated abscissa and an updated ordinate of the point cloud; determining a length threshold based on the length, and determining a width threshold based on the width; A point cloud whose absolute value of the updated horizontal coordinate is less than or equal to the width threshold and whose absolute value of the updated vertical coordinate is less than or equal to the length threshold is determined as a target point cloud.

3. The method according to claim 2, characterized in that The updating of the horizontal coordinate of the point cloud and the vertical coordinate of the point cloud based on the rotation angle, the central horizontal coordinate and the central vertical coordinate, respectively, to obtain the updated horizontal coordinate of the point cloud and the updated vertical coordinate of the point cloud, respectively, includes: Determine a rotation matrix corresponding to the rotation angle; Based on the central horizontal coordinate and the central vertical coordinate, respectively, the horizontal coordinate and the vertical coordinate of the point cloud are translated, and a first translation amount and a second translation amount are correspondingly obtained; Based on the rotation matrix, the first translation amount and the second translation amount are rotationally transformed to obtain an updated horizontal coordinate of the point cloud and an updated vertical coordinate of the point cloud.

4. The method according to claim 1, wherein The projection coordinate range includes a minimum projection abscissa, a minimum projection ordinate, a maximum projection abscissa, and a maximum projection ordinate. Determining the projection coordinate range of the target point cloud projected onto the first side of the first area includes: Determine the starting point coordinates and the ending point coordinates of the first side, and determine the direction vector of the first side based on the starting point coordinates and the ending point coordinates; For each of the target point clouds, determining a projection ratio of the target point cloud onto the first edge based on the point cloud coordinates of the target point cloud, the starting point coordinates, and the direction vector; Determine the projection coordinates of the target point cloud based on the starting point coordinates, the projection scale, and the direction vector, where the projection coordinates include a projection abscissa and a projection ordinate; Determining the minimum value among the projected horizontal coordinates of the plurality of target point clouds as the minimum projected horizontal coordinate, and determining the minimum value among the projected vertical coordinates of the plurality of target point clouds as the minimum projected vertical coordinate; The maximum value among the projected horizontal coordinates of the plurality of target point clouds is determined as the maximum projected horizontal coordinate, and the maximum value among the projected vertical coordinates of the plurality of target point clouds is determined as the maximum projected vertical coordinate.

5. The method according to claim 4, characterized in that The starting point coordinates include a first horizontal coordinate and a first vertical coordinate, the ending point coordinates include a second horizontal coordinate and a second vertical coordinate, and determining the direction vector of the first side based on the starting point coordinates and the ending point coordinates includes: Determine the difference between the second horizontal coordinate and the first horizontal coordinate as a first vector component; Determine the difference between the second ordinate and the first ordinate as a second vector component; A direction vector of the first edge is constructed using the first vector component and the second vector component.

6. The method according to claim 4, characterized in that The determining, based on the point cloud coordinates of the target point cloud, the starting point coordinates, and the direction vector, of a projection ratio of the target point cloud onto the first edge includes: Determining a displacement vector pointing from the starting point of the first side to the target point cloud based on the point cloud coordinates of the target point cloud and the starting point coordinates; Performing a dot product process on the displacement vector and the direction vector to obtain a dot product result; Based on the direction vector, the dot product result is normalized to obtain a projection ratio of the target point cloud onto the first edge.

7. The method according to claim 5, characterized in that The determining the projection coordinates of the target point cloud based on the starting point coordinates, the projection scale, and the direction vector includes: Scaling the first vector component using the projection ratio to obtain a first displacement; Superimposing the first horizontal coordinate and the first displacement to obtain a projected horizontal coordinate of the target point cloud; Scaling the second vector component using the projection ratio to obtain a second displacement; The first vertical coordinate and the second displacement are superimposed to obtain the projected vertical coordinate of the target point cloud.

8. The method according to claim 5, characterized in that The updated projection coordinate range includes a minimum projection abscissa, an updated minimum projection ordinate, a maximum projection abscissa, and an updated maximum projection ordinate. The updating of the projection coordinate range to obtain the updated projection coordinate range includes: determining the ratio of the second vector component to the first vector component as the slope of the first side; Based on the slope and the starting point coordinates, determining a first projected ordinate that maps the minimum projected abscissa to a first projected ordinate on the straight line where the first side is located, and determining the first projected ordinate as an updated minimum projected ordinate; Based on the slope and the starting point coordinates, a second projected ordinate is determined to map the maximum projected abscissa to the straight line where the first side is located, and the second projected ordinate is determined as the updated maximum projected ordinate.

9. The method according to any one of claims 1 to 8, characterized in that The adjusting the first area based on the updated projection coordinate range to obtain an updated first area includes: Determine the average of the minimum projected abscissa and the maximum projected abscissa as the updated central abscissa, and determine the average of the updated minimum projected ordinate and the updated maximum projected ordinate as the updated central ordinate; determining a first length based on the minimum projected abscissa, the maximum projected abscissa, the updated minimum projected ordinate, and the updated maximum projected ordinate; The first area is adjusted based on the first length, the updated central horizontal coordinate, and the updated central vertical coordinate to obtain an updated first area.

10. The method according to claim 9, characterized in that The adjusting the first area based on the first length, the updated central horizontal coordinate, and the updated central vertical coordinate to obtain an updated first area includes: Obtaining the region length and region width of the first region; When the region length is less than or equal to the region width, determining the first length as the updated region width, and determining an updated first region based on the updated region width, the region length, the updated central abscissa, and the updated central ordinate; When the region length is greater than the region width, the first length is determined as the updated region length, and an updated first region is determined based on the updated region length, the region width, the updated central abscissa, and the updated central ordinate.

11. A target detection device, characterized in that: The device comprises: The first detection module is used to perform target detection on the data to be processed and obtain an initial detection result; a first determining module configured to, when a first area corresponding to the initial detection result intersects a preset area, obtain point cloud data corresponding to the first area, and determine a target point cloud from the point cloud data, where the target point cloud is a point cloud located inside the first area; a point cloud projection module, configured to determine a projection coordinate range of a first side of the first region projected by the target point cloud, and update the projection coordinate range based on the first side to obtain an updated projection coordinate range, wherein the first side is the longest side among the sides of the first region; A region adjustment module, configured to adjust the first region based on the updated projection coordinate range to obtain an updated first region; The second determination module is used to determine the target detection result based on the updated first area.

12. An electronic device, characterized in that: The electronic device comprises: a memory for storing computer-executable instructions or computer programs; The processor is configured to implement the target detection method according to any one of claims 1 to 10 when executing the computer-executable instructions or computer program stored in the memory.

13. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that: When the computer executable instructions or computer program are executed by a processor, the target detection method according to any one of claims 1 to 10 is implemented.

Images