Method for measuring similarity of image under shielding condition and related equipment thereof

Abstract

The invention provides a similarity measurement method under an image shielding condition and related equipment thereof, and the method comprises the steps: obtaining two-dimensional images and three-dimensional projection images of a plurality of detection objects, the three-dimensional projection images being two-dimensional projection images of point cloud data of the detection objects, and the two-dimensional images and the three-dimensional projection images being associated and having a relative position relationship; for each of the plurality of detection objects, an initial class intersection-union ratio and a correction value between the two-dimensional image of the detection object and each three-dimensional projection image are calculated on the basis of the relative positional relationship, the initial class intersection-union ratio being obtained from the area of the two-dimensional image and the area of the two-dimensional image between the two-dimensional image and the three-dimensional projection image. The correction value is obtained according to the area of the three-dimensional projection image and the area of the two-dimensional image; target class intersection-union ratios are obtained according to the initial class intersection-union ratios and the correction values, so that multiple target class intersection-union ratios of the detection object are obtained, and the target class intersection-union ratios represent the similarity degree between the two-dimensional image and the three-dimensional projection image.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a method for measuring similarity of an image under occlusion and related equipment. Background Art

[0002] In the field of intelligent driving, vehicles perceive objects around the vehicle through on-board sensors, such as obtaining two-dimensional images of surrounding objects through image collectors and obtaining point cloud data of surrounding objects through lidar.

[0003] However, when there is mutual occlusion between surrounding objects, when the two-dimensional image of the surrounding objects obtained by the image collector and the three-dimensional image of the point cloud data of the surrounding objects obtained by the lidar are measured by the existing intersection-over-union ratio, errors may occur due to insufficient precision. Summary of the invention

[0004] The present application provides a method for measuring the similarity of an image under occlusion and related equipment, provides a new similarity measurement method, and solves the error problem caused by insufficient accuracy of similarity measurement by intersection-over-union ratio in the prior art.

[0005] The first aspect of the present application provides a method for measuring similarity of an image under occlusion, comprising:

[0006] Acquire a two-dimensional image and a three-dimensional projection image of each of a plurality of detection objects, wherein the three-dimensional projection image is a two-dimensional projection image of point cloud data of the detection object, and the two-dimensional image is associated with the three-dimensional projection image and has a relative position relationship;

[0007] For each of the plurality of detection objects, an initial class intersection and union ratio and a correction value between the two-dimensional image of the detection object and each of the three-dimensional projection images are calculated based on the corresponding relative position relationship, wherein the initial class intersection and union ratio is obtained according to the overlapping area between the two-dimensional image and the three-dimensional projection image and the area of ​​the two-dimensional image, and the correction value is obtained according to the area of ​​the three-dimensional projection image and the area of ​​the two-dimensional image;

[0008] A target class intersection and union is obtained according to the initial class intersection and union and the correction value to obtain a plurality of target class intersection and union of each of the plurality of detection objects, wherein the target class intersection and union represents the degree of similarity between the two-dimensional image and the three-dimensional projection image.

[0009] In some implementations, the calculating, based on the corresponding relative positional relationship, an initial class intersection and union ratio and a correction value between the two-dimensional image of the detection object and each of the three-dimensional projection images, wherein the initial class intersection and union ratio is obtained according to an overlapping area between the two-dimensional image and the three-dimensional projection image and an area of ​​the two-dimensional image, and the correction value is obtained according to an area of ​​the three-dimensional projection image and an area of ​​the two-dimensional image, comprises:

[0010] Acquire an overlapping portion between the two-dimensional image of the detection object and each of the three-dimensional projection images according to the relative position relationship to obtain an overlapping area of ​​the overlapping portion;

[0011] Calculating a first ratio between the overlapping area and the area of ​​the two-dimensional image of the detection object to determine the first ratio as the initial intersection-union ratio;

[0012] A second ratio between the area of ​​the two-dimensional image of the detection object and the area of ​​each of the three-dimensional projection images is calculated, and the second ratio is input into a preset formula to obtain the correction value.

[0013] In some implementations, the preset formula is:

[0014]

[0015] Wherein, x is the second ratio, μ is the population mean, and σ is the population distance standard deviation.

[0016] In some implementations, the value of μ is 1.5, and the value of σ is 3.

[0017] In some implementations, obtaining a target class intersection-and-union ratio according to the initial class intersection-and-union ratio and the correction value to obtain a plurality of target class intersection-and-union ratios of the plurality of detection objects respectively includes:

[0018] The product of the initial class intersection-and-union ratio and the correction value is calculated to obtain a plurality of target class intersection-and-union ratios of each of the plurality of detection objects.

[0019] In some implementations, the acquiring of a two-dimensional image and a three-dimensional projection image of each of the plurality of detection objects, wherein the three-dimensional projection image is a two-dimensional projection image of point cloud data of the detection object, and the two-dimensional image is associated with the three-dimensional projection image and has a relative position relationship, includes:

[0020] Acquire point cloud data of each of the plurality of detection objects according to the laser radar, and acquire initial two-dimensional images of each of the plurality of detection objects according to the image collector;

[0021] Inputting the plurality of point cloud data and the plurality of initial two-dimensional images into a deep learning model to obtain the plurality of two-dimensional images and a plurality of three-dimensional images, wherein the deep learning model comprises a Yolo model or a PointPillar model;

[0022] The multiple three-dimensional images are respectively two-dimensionally projected based on coordinate conversion to obtain the three-dimensional projection images having the relative position relationship with the corresponding two-dimensional images, thereby obtaining multiple three-dimensional projection images.

[0023] In some implementations, the coordinate conversion is determined based on respective coordinate parameters of the laser radar and the image collector.

[0024] In some implementations, the two-dimensional image and the three-dimensional projection image are detection frame images that include the detection object.

[0025] A second aspect of the present application provides an image occlusion measurement device, comprising:

[0026] A first acquisition module is used to acquire a two-dimensional image and a three-dimensional projection image of each of a plurality of detection objects, wherein the three-dimensional projection image is a two-dimensional projection image of point cloud data of the detection object, and the two-dimensional image is associated with the three-dimensional projection image and has a relative position relationship;

[0027] a first calculation module, configured to calculate, for each of the plurality of detection objects, an initial class intersection and union ratio and a correction value between the two-dimensional image of the detection object and each of the three-dimensional projection images based on the corresponding relative position relationship, wherein the initial class intersection and union ratio is obtained according to an overlapping area between the two-dimensional image and the three-dimensional projection image and an area of ​​the two-dimensional image, and the correction value is obtained according to an area of ​​the three-dimensional projection image and an area of ​​the two-dimensional image;

[0028] A second calculation module is used to obtain a target class intersection and union ratio according to the initial class intersection and union ratio and the correction value, so as to obtain a plurality of target class intersection and union ratios of each of the plurality of detection objects, wherein the target class intersection and union ratio represents the similarity between the two-dimensional image and the three-dimensional projection image;

[0029] A third aspect of the present application provides a computer-readable storage medium, comprising instructions, which, when executed on a computer, enables the computer to execute the method in any implementation of the aforementioned first aspect.

[0030] Compared with the prior art, the technical solution of the present application obtains the two-dimensional images of each of the multiple detection objects and the three-dimensional projection images after projecting the point cloud data of the detection object, and then, for each of the multiple detection objects, the initial class intersection and correction value between the two-dimensional image of the detection object and the three-dimensional projection images of other detection objects is calculated based on the relative position relationship. The initial class intersection and correction value is obtained according to the overlapping area between the two-dimensional image and the three-dimensional projection image and the area of ​​the two-dimensional image, and the correction value is obtained according to the area of ​​the three-dimensional projection image and the area of ​​the two-dimensional image. Then, the target class intersection and correction value is obtained according to the initial class intersection and correction value. Since the target class intersection and correction between the three-dimensional projection image and the two-dimensional image is obtained by converting the point cloud data into a three-dimensional projection image in a two-dimensional form, wherein the target class intersection and correction represents the similarity between the two-dimensional image and the three-dimensional projection image, therefore, compared with the use of intersection and correction in the prior art to measure the similarity, the present application provides a new method for measuring the similarity, which solves the error problem caused by the insufficient accuracy of the intersection and correction in measuring the similarity. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0032] Figure 1 It is a schematic diagram of a method for obtaining a handover ratio in the prior art;

[0033] Figure 2 It is a schematic diagram of a similarity measurement method under image occlusion in an embodiment of the present application;

[0034] Figure 3 This is a schematic diagram of a calculation process of an initial class intersection-and-union ratio and a correction value in an embodiment of the present application;

[0035] Figure 4 This is a target class intersection-and-union ratio calculation schematic diagram of an embodiment of the present application;

[0036] Figure 5 This is a schematic diagram of a process for obtaining two-dimensional images and three-dimensional projection images of multiple detection objects in an embodiment of the present application;

[0037] Figure 6 is a schematic diagram of an image occlusion measurement device according to an embodiment of the present application;

[0038] Figure 7 It is a schematic diagram of another image occlusion measurement device according to an embodiment of the present application. DETAILED DESCRIPTION

[0039] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0040] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0041] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0042] The terms "first", "second", "third" in this application are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first", "second", "third" can expressly or implicitly include at least one of the features. In the description of this application, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. In the embodiments of this application, all directional indications (such as up, down, left, right, front, back...) are only used to explain the relative position relationship, movement, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication also changes accordingly. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the steps or units listed, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices.

[0043] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0044] The present application provides a calculation method similar to the intersection-and-union ratio, hereinafter referred to as the quasi-intersection-and-union ratio, and applies it to a similarity measurement method under occlusion conditions between a two-dimensional image and a three-dimensional image of point cloud data.

[0045] The calculation method of the intersection-to-union ratio in the prior art is the ratio of the intersection area of ​​two two-dimensional images to the union area of ​​two two-dimensional images. Figure 1 , where the intersection area refers to the area corresponding to the overlapping part of two two-dimensional images, and the union area refers to all areas of the two two-dimensional images in the overlapping situation, where the overlapping situation is divided into the existence of overlapping parts and the non-existence of overlapping parts. It can be understood that when there is an overlapping part, the intersection area is less than the sum of the areas of the two two-dimensional images, and the union area exists; when there is no overlapping part, the intersection area is equal to the sum of the areas of the two two-dimensional images, and the union area does not exist.

[0046] See also Figure 2 , Figure 2 The similarity measurement method based on the intersection of classes and union ratio under image occlusion is shown, which specifically includes:

[0047] 201: Acquire a two-dimensional image and a three-dimensional projection image of each of a plurality of detection objects, wherein the three-dimensional projection image is a two-dimensional projection image of point cloud data of the detection object, and the two-dimensional image is associated with the three-dimensional projection image and has a relative position relationship;

[0048] The vehicle obtains the two-dimensional images of multiple detection objects around it through the image collector, and obtains the point cloud data of multiple detection objects around it through the lidar, and then projects the multiple point cloud data into corresponding three-dimensional projection images to obtain two-dimensional images and three-dimensional projection images that are interrelated and have a relative position relationship.

[0049] The image collector may be a vehicle-mounted camera or a vehicle-mounted camera, etc., and the specifics are not limited here.

[0050] 202: for each of the plurality of detection objects, calculating an initial class intersection and union ratio and a correction value between the two-dimensional image of the detection object and each three-dimensional projection image based on the corresponding relative position relationship, wherein the initial class intersection and union ratio is obtained according to an overlapping area between the two-dimensional image and the three-dimensional projection image and an area of ​​the two-dimensional image, and the correction value is obtained according to an area of ​​the three-dimensional projection image and an area of ​​the two-dimensional image;

[0051] For each two-dimensional image of the detected object, the initial class intersection and union ratio and correction value between it and all three-dimensional projection images are calculated based on the relative position relationship, wherein the initial class intersection and union ratio is obtained according to the overlapping area between the two-dimensional image and the three-dimensional projection image and the area of ​​the two-dimensional image, and the correction value is obtained according to the area of ​​the three-dimensional projection image and the area of ​​the two-dimensional image.

[0052] It can be understood that the initial class intersection-over-union ratio is obtained by using the overlapping area, and the overlapping area is obtained based on the relative position relationship.

[0053] Preferably, the relative position can be represented by a fused image including a two-dimensional image and a three-dimensional projection image having a relative position relationship. That is, the present application obtains an image including a two-dimensional image, a three-dimensional projection image and an overlapping portion thereof through multiple fused images.

[0054] 203: Obtaining a target class intersection-and-union ratio according to the initial class intersection-and-union ratio and the correction value to obtain multiple target class intersection-and-union ratios of the detection object, wherein the target class intersection-and-union ratio indicates the similarity between the two-dimensional image and the three-dimensional projection image.

[0055] After obtaining the corresponding initial class intersection and union ratio and correction value between the two-dimensional image of each detection object and all three-dimensional projection images, the vehicle obtains the target class intersection and union ratio between the two-dimensional image of each detection object and all three-dimensional projection images based on the class intersection and union ratio and the correction value.

[0056] It can be understood that the present application converts point cloud data into a three-dimensional projection image. Since the three-dimensional projection image and the two-dimensional projection image have a relative position relationship, the target class intersection-union ratio between the projection image and the two-dimensional image is obtained based on the relative position relationship. The method of obtaining the target class intersection-union ratio improves the method of obtaining the intersection-union ratio in the prior art, thereby providing a new method for measuring the degree of similarity and solving the error problem caused by the insufficient accuracy of the intersection-union ratio in measuring the degree of similarity in the prior art.

[0057] In some embodiments, after the vehicle determines the target class intersection and union ratio representing the similarity between the detection objects, it further determines two detection objects with the target similarity based on a preset class intersection and union ratio threshold. For example, the two detection objects with the target similarity can be determined by the Hungarian algorithm.

[0058] See also Figure 3 , Figure 3 The calculation process of determining the initial class intersection and union ratio and the correction value is shown. Now, an example of a two-dimensional image and a three-dimensional projection image is used for explanation. This example can be applied to calculate the target class intersection and union ratio of each two-dimensional image and all its three-dimensional projection images, specifically including:

[0059] 301: Acquire an overlapping portion between the two-dimensional image of the detection object and each three-dimensional projection image according to the relative position relationship to obtain an overlapping area of ​​the overlapping portion;

[0060] The vehicle obtains the overlapping parts between the two-dimensional image of each detection object and multiple three-dimensional projection images based on the relative position relationship, and calculates the overlapping area of ​​the overlapping parts.

[0061] In some cases, the overlapping portion of the two-dimensional image and the three-dimensional projection image does not exist, and the corresponding overlapping area is zero.

[0062] The relative position relationship refers to the relative position relationship between the two-dimensional image and the three-dimensional projection image in the same coordinate system. Whether the two-dimensional image and the three-dimensional projection image overlap and the size of the overlap can be identified based on the relative position relationship.

[0063] Preferably, the relative positional relationship may be represented by a fused image including a two-dimensional image and a three-dimensional projection image having the relative positional relationship.

[0064] 302: Calculate a first ratio between the overlapping area and the area of ​​the two-dimensional image of the detection object to determine the first ratio as an initial intersection-and-union ratio;

[0065] The vehicle further calculates a first ratio between the overlapping area and the area of ​​the acquired two-dimensional image of the detection object, and determines the first ratio as an initial intersection-union ratio.

[0066] 303: Calculate a second ratio between the area of ​​the two-dimensional image of the detection object and the area of ​​each three-dimensional projection image, and input the second ratio into a preset formula to obtain a correction value.

[0067] The vehicle further calculates a second ratio of the area of ​​the two-dimensional image of the detection object to the area of ​​all its three-dimensional projection images, and then inputs the second ratio into a preset formula to obtain a correction value.

[0068] It is understandable that the correction value is related to the initial class intersection and union ratio.

[0069] The default formula is:

[0070]

[0071] Wherein, x is the second ratio, μ is the mean obtained by multiple statistics, and σ is the distance standard deviation obtained by multiple statistics.

[0072] In some embodiments, μ is 1.5 and σ is 3.

[0073] See also Figure 4 , Figure 4 It shows the determination of the target class intersection and union ratio, which specifically includes:

[0074] 401: Calculate the product of the initial class intersection-union ratio and the correction value to obtain multiple target class intersection-union ratios of the detection object.

[0075] After the vehicle obtains all the mutually related initial class intersections and correction values ​​of the two-dimensional image of each detection object, the product of the mutually related initial class intersections and correction values ​​is calculated to obtain all target class intersections and corrections of the two-dimensional image of each detection object.

[0076] Based on the above formula 1, the calculation formula of the target class intersection and union ratio is:

[0077]

[0078] Wherein, S1 represents the overlapping area between the two-dimensional image and the three-dimensional projection image, S2 represents the area of ​​the two-dimensional image, and S3 represents the area of ​​the three-dimensional projection image. x representing the second ratio.

[0079] The two-dimensional image and three-dimensional projection image of this application are detection frame images including the detection object, please refer to Figure 5 , which shows the process of obtaining two-dimensional images and three-dimensional projection images of multiple detection objects, specifically including:

[0080] 501: acquiring point cloud data of each of the plurality of detection objects according to a laser radar, and acquiring initial two-dimensional images of each of the plurality of detection objects according to an image collector;

[0081] The vehicle obtains point cloud data of each of the multiple detection objects through its laser radar, and obtains initial two-dimensional images of each of the multiple detection objects through the image collector.

[0082] 502: Inputting a plurality of point cloud data and a plurality of initial two-dimensional images into a deep learning model to obtain a plurality of two-dimensional images and a plurality of three-dimensional images, wherein the deep learning model includes a Yolo model or a PointPillar model;

[0083] The vehicle inputs the acquired point cloud data and the initial two-dimensional image into the trained deep learning model, thereby obtaining a two-dimensional image and a three-dimensional image, wherein the two-dimensional image and the three-dimensional image are detection frame images including the detection object.

[0084] In some embodiments, the deep learning model includes a Yolo model or a PointPillar model.

[0085] 503: Perform two-dimensional projection on the multiple three-dimensional images based on coordinate conversion to obtain three-dimensional projection images that have a relative position relationship with the corresponding two-dimensional images, and then obtain multiple three-dimensional projection images.

[0086] Since the three-dimensional image and the two-dimensional image are obtained by two different devices, both are in the coordinate system of the corresponding device. In order to calculate the intersection and union ratio of the two, and then restore the real position relationship between the two in the physical scene, it is necessary to process the position of the two. Therefore, each three-dimensional image is projected in two dimensions based on coordinate conversion to obtain a three-dimensional projection image with a relative position relationship between each three-dimensional image and all two-dimensional images. Among them, the coordinate conversion is used to determine the relative position relationship between the three-dimensional projection image and the two-dimensional image, and the coordinate conversion is determined based on the coordinate parameters of the laser radar and the image collector.

[0087] Preferably, the three-dimensional image is projected based on coordinate conversion to obtain a fused image including the three-dimensional projection image and the two-dimensional image having a relative position relationship. Therefore, the target class intersection-union ratio between the three-dimensional projection image and the two-dimensional image can be determined through the fused image.

[0088] In some embodiments, multiple three-dimensional images are projected two-dimensionally onto corresponding two-dimensional images based on coordinate conversion to obtain a fused image, wherein the content of the fused image includes two-dimensional images and three-dimensional projection images that are associated and have a relative position relationship.

[0089] See also Figure 6 , Figure 6 An image occlusion measurement device is shown, comprising:

[0090] A first acquisition module 601 is used to acquire a 2D image and a 3D projection image of each of a plurality of detection objects, wherein the 3D projection image is a 2D projection image of the point cloud data of the detection object, and the 2D image is associated with the 3D projection image and has a relative position relationship;

[0091] A first calculation module 602 is used to calculate, for each of the multiple detection objects, an initial class intersection and union ratio and a correction value between the two-dimensional image of the detection object and each three-dimensional projection image based on the corresponding relative position relationship, wherein the initial class intersection and union ratio is obtained according to the overlapping area between the two-dimensional image and the three-dimensional projection image and the area of ​​the two-dimensional image, and the correction value is obtained according to the area of ​​the three-dimensional projection image and the area of ​​the two-dimensional image;

[0092] The second calculation module 603 is used to obtain a target class intersection-union ratio according to the initial class intersection-union ratio and the correction value, so as to obtain multiple target class intersection-union ratios of the detection object.

[0093] This application also includes an image occlusion measurement device, see Figure 7 The method comprises: a memory 701 and a processor 702; wherein the memory is used to store programs; and the processor is used to execute the programs in the memory, including executing any one of the methods in the aforementioned implementation modes;

[0094] The device also includes a bus system, which is used to connect the memory and the processor so that the memory and the processor can communicate with each other.

[0095] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0096] In some embodiments, the memory may be an internal storage unit of the cloud device, for example, a hard disk or memory of the cloud device. In other embodiments, the memory may also be an external storage device of the cloud device, for example, a plug-in hard disk equipped on the cloud device, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Further, the memory may also include both an internal storage unit of the cloud device and an external storage device. The memory is used to store an operating system, an application, a boot loader (BootLoader), data, and other programs, such as program code of a computer program, etc. The memory may also be used to temporarily store data that has been output or is to be output.

[0097] The present application also includes a computer-readable storage medium, including instructions, which, when executed on a computer, enable the computer to execute the method of any one of the aforementioned embodiments.

[0098] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of each of the above-mentioned method embodiments can be implemented.

[0099] The computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form. Computer readable media may include at least: any entity or device capable of carrying computer program code to a camera / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electric carrier signal, a telecommunication signal and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer readable media cannot be electric carrier signals and telecommunication signals.

[0100] The above are only implementation methods of the present application, and are not intended to limit the patent protection scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A similarity measurement method for images under occlusion, characterized in that: include: Acquire a two-dimensional image and a three-dimensional projection image of each of a plurality of detection objects, wherein the three-dimensional projection image is a two-dimensional projection image of point cloud data of the detection object, and the two-dimensional image is associated with the three-dimensional projection image and has a relative position relationship; For each of the plurality of detection objects, an initial class intersection and union ratio and a correction value between the two-dimensional image of the detection object and each of the three-dimensional projection images are calculated based on the corresponding relative position relationship, wherein the initial class intersection and union ratio is obtained according to the overlapping area between the two-dimensional image and the three-dimensional projection image and the area of ​​the two-dimensional image, and the correction value is obtained according to the area of ​​the three-dimensional projection image and the area of ​​the two-dimensional image; A target class intersection and union is obtained according to the initial class intersection and union and the correction value to obtain a plurality of target class intersection and union of each of the plurality of detection objects, wherein the target class intersection and union represents the degree of similarity between the two-dimensional image and the three-dimensional projection image.

2. The similarity measurement method under occlusion according to claim 1, characterized in that: The calculating of the initial intersection-and-union ratio and the correction value between the two-dimensional image of the detection object and each of the three-dimensional projection images based on the corresponding relative position relationship, wherein the initial intersection-and-union ratio is obtained according to the overlapping area between the two-dimensional image and the three-dimensional projection image and the area of ​​the two-dimensional image, and the correction value is obtained according to the area of ​​the three-dimensional projection image and the area of ​​the two-dimensional image, comprises: Acquire an overlapping portion between the two-dimensional image of the detection object and each of the three-dimensional projection images according to the relative position relationship to obtain an overlapping area of ​​the overlapping portion; Calculating a first ratio between the overlapping area and the area of ​​the two-dimensional image of the detection object to determine the first ratio as the initial intersection-union ratio; A second ratio between the area of ​​the two-dimensional image of the detection object and the area of ​​each of the three-dimensional projection images is calculated, and the second ratio is input into a preset formula to obtain the correction value.

3. The similarity measurement method under occlusion according to claim 2, characterized in that: The preset formula is: Wherein, x is the second ratio, μ is the population mean, and σ is the population distance standard deviation.

4. The similarity measurement method under occlusion according to claim 3, characterized in that: The value of μ is 1.5, and the value of σ is 3.

5. The similarity measurement method under occlusion according to any one of claims 1 to 4, characterized in that: Obtaining a target class intersection and union ratio according to the initial class intersection and union ratio and the correction value to obtain a plurality of target class intersection and union ratios of the plurality of detection objects respectively includes: The product of the initial class intersection-and-union ratio and the correction value is calculated to obtain a plurality of target class intersection-and-union ratios of each of the plurality of detection objects.

6. The similarity measurement method under occlusion according to any one of claims 1 to 4, characterized in that: The acquiring of a two-dimensional image and a three-dimensional projection image of each of the plurality of detection objects, wherein the three-dimensional projection image is a two-dimensional projection image of point cloud data of the detection object, and the two-dimensional image is associated with the three-dimensional projection image and has a relative position relationship, comprises: Acquire point cloud data of each of the plurality of detection objects according to the laser radar, and acquire initial two-dimensional images of each of the plurality of detection objects according to the image collector; Inputting the plurality of point cloud data and the plurality of initial two-dimensional images into a deep learning model to obtain the plurality of two-dimensional images and a plurality of three-dimensional images, wherein the deep learning model comprises a Yolo model or a PointPillar model; The multiple three-dimensional images are respectively two-dimensionally projected based on coordinate conversion to obtain the three-dimensional projection images having the relative position relationship with the corresponding two-dimensional images, thereby obtaining multiple three-dimensional projection images.

7. The similarity measurement method under occlusion according to claim 6, characterized in that: The coordinate conversion is determined according to the respective coordinate parameters of the laser radar and the image collector.

8. The similarity measurement method under occlusion conditions according to any one of claims 1 to 4, characterized in that: The two-dimensional image and the three-dimensional projection image are detection frame images including the detection object.

9. An image occlusion measurement device, characterized in that: include: A first acquisition module is used to acquire a two-dimensional image and a three-dimensional projection image of each of a plurality of detection objects, wherein the three-dimensional projection image is a two-dimensional projection image of point cloud data of the detection object, and the two-dimensional image is associated with the three-dimensional projection image and has a relative position relationship; a first calculation module, configured to calculate, for each of the plurality of detection objects, an initial class intersection and union ratio and a correction value between the two-dimensional image of the detection object and each of the three-dimensional projection images based on the corresponding relative position relationship, wherein the initial class intersection and union ratio is obtained according to an overlapping area between the two-dimensional image and the three-dimensional projection image and an area of ​​the two-dimensional image, and the correction value is obtained according to an area of ​​the three-dimensional projection image and an area of ​​the two-dimensional image; The second calculation module is used to obtain a target class intersection and union according to the initial class intersection and union and the correction value to obtain multiple target class intersection and union of each of the multiple detection objects, wherein the target class intersection and union represents the similarity between the two-dimensional image and the three-dimensional projection image.

10. A computer-readable storage medium comprising instructions, characterized in that: When the method is run on a computer, the computer is enabled to execute the similarity measurement method under occlusion conditions according to any one of claims 1 to 8.

Images