Image accuracy assessment methods, devices, equipment, storage media, and software products

Abstract

This application provides a method, apparatus, device, storage medium, and program product for judging image accuracy. Embodiments of this application can be applied to various scenarios such as cloud technology, artificial intelligence, smart transportation, and vehicle-mounted systems, and relate to image processing technology. The method includes: acquiring a depth map to be judged, and a reference point cloud map of the subject included in the depth map to be judged; wherein the reference point cloud map is a point cloud map obtained by scanning with an accuracy not less than an accuracy threshold; converting the depth map to be judged into a converted point cloud map; wherein the converted point cloud map records the three-dimensional coordinate information of each point of the subject reconstructed based on the depth values ​​recorded in the depth map to be judged; determining the accuracy of the depth map to be judged based on the overlap detection of the converted point cloud map and the reference point cloud map, thereby realizing image accuracy judgment. This application can improve the accuracy of image accuracy judgment.

Description

Technical Field

[0001] This application relates to image processing technology, and more particularly to an image accuracy judgment method, apparatus, device, storage medium, and program product. Background Technology

[0002] A depth map is an image that uses pixel values ​​to represent the depth of points in the scene from the image acquisition device. It reflects the depth information of objects in the scene. Figure 1 Generally, data is collected by depth cameras, which are widely used in facial payment systems and autonomous driving systems.

[0003] Before use, depth cameras need to have their acquired depth maps verified for accuracy to ensure the reliability of the depth maps acquired during actual use. However, some related technologies suffer from poor accuracy in judging the stereoscopic accuracy of depth maps, ultimately resulting in low precision in image accuracy assessment. Summary of the Invention

[0004] This application provides an image accuracy judgment method, apparatus, storage medium, and program product, which can improve the accuracy judgment of image accuracy.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides an image accuracy determination method, including:

[0007] Obtain a depth map to be judged, and a reference point cloud map of the subject included in the depth map to be judged; wherein, the reference point cloud map is a point cloud map obtained by scanning with an accuracy not less than an accuracy threshold, and the reference point cloud map records the real three-dimensional coordinate information of each point of the subject.

[0008] The depth map to be determined is converted into a point cloud map to obtain a converted point cloud map; wherein, the converted point cloud map records the three-dimensional coordinate information of each point of the photographed object reconstructed based on the depth values ​​recorded in the depth map to be determined;

[0009] By performing overlap detection on the converted point cloud map and the reference point cloud map, the accuracy of the depth map to be judged is determined, thereby realizing image accuracy judgment.

[0010] This application provides an image accuracy determination device, including:

[0011] The image acquisition module is used to acquire a depth map to be judged, and a reference point cloud map of the subject included in the depth map to be judged; wherein, the reference point cloud map is a point cloud map obtained by scanning with an accuracy not less than an accuracy threshold, and the reference point cloud map records the real three-dimensional coordinate information of each point of the subject.

[0012] The image conversion module is used to convert the depth map to be determined into a point cloud map to obtain a converted point cloud map; wherein, the converted point cloud map records the three-dimensional coordinate information of each point of the shooting object reconstructed based on the depth values ​​recorded in the depth map to be determined;

[0013] The accuracy judgment module is used to determine the accuracy of the depth map to be judged based on the overlap detection of the converted point cloud map and the reference point cloud map, thereby realizing image accuracy judgment.

[0014] In some embodiments of this application, the accuracy judgment module is further configured to generate multiple matching point pairs by registering the converted point cloud map with the reference point cloud map; and to obtain the accuracy of the depth map to be judged based on the overlap calculation of the multiple matching point pairs respectively.

[0015] In some embodiments of this application, the accuracy judgment module is further configured to perform distance calculation on at least two points contained in each of the plurality of matching point pairs to obtain multiple point pair distances; calculate the overlap between the converted point cloud map and the reference point cloud map based on the multiple point pair distances; and convert the overlap into the accuracy of the depth map to be judged.

[0016] In some embodiments of this application, the accuracy judgment module is further configured to filter out, from the plurality of point-to-point distances, the first distances corresponding to the N points closest to the origin in the transformed point cloud map, and the second distances corresponding to the N points farthest from the origin; and determine the difference between the mean of the N first distances and the mean of the N second distances as the overlap between the transformed point cloud map and the reference point cloud map; wherein, N is a positive integer.

[0017] In some embodiments of this application, the accuracy judgment module is further configured to filter out the third distances corresponding to M points in the target area of ​​the converted point cloud map from the plurality of point-to-point distances; calculate the Gaussian distribution variance of the M third distances to obtain the overlap between the converted point cloud map and the reference point cloud map; wherein M is a positive integer.

[0018] In some embodiments of this application, the image accuracy determination device further includes: a parameter determination module, used to determine the reconstructed shape parameters of the photographed object based on the converted point cloud map, and to determine the reference shape parameters of the photographed object based on the reference point cloud map;

[0019] The accuracy determination module is further configured to determine the accuracy of the depth map to be determined by the difference between the reconstructed shape parameters and the reference shape parameters.

[0020] In some embodiments of this application, the transformed point cloud map includes: multiple point cloud maps; the accuracy judgment module is further configured to: for each point in each of the multiple point cloud maps, select the candidate point with the smallest distance from the reference point cloud map; generate a transformed point cloud map corresponding to each point cloud map based on the rigid body transformation between each point in each point cloud map and the corresponding candidate point; select the matching point with the smallest distance to each point in the transformed point cloud map corresponding to each point cloud map from the reference point cloud map, and generate multiple transformed point pairs using each point in the transformed point cloud map and the corresponding matching point; fuse the transformed point cloud maps corresponding to each point cloud map to obtain a fused point cloud map, and determine the multiple matching point pairs from the multiple transformed point pairs for each point in the fused point cloud map.

[0021] In some embodiments of this application, the transformation parameters include: rotation parameters and translation parameters; the accuracy judgment module is further configured to determine the rotation parameters and the translation parameters by performing a rigid body transformation with the minimum distance on each point of each point cloud map and the corresponding candidate point; and to perform the transformation corresponding to the rotation parameters and the translation parameters on each point cloud map to obtain the transformed point cloud map corresponding to each point cloud map.

[0022] In some embodiments of this application, the accuracy judgment module is further configured to: filter out a target transformed point cloud map from the transformed point cloud maps corresponding to each point cloud map; calculate the normal difference and spatial distance between each point in the target transformed point cloud map and each point in other transformed point cloud maps for each point in the target transformed point cloud map; wherein, the other transformed point cloud maps refer to transformed point cloud maps other than the target transformed point cloud map in the transformed point cloud maps corresponding to each point cloud map; based on the normal difference and the spatial distance, filter out points to be fused from the other transformed point cloud maps for each point in the target transformed point cloud map; and perform weighted fusion of each point in the target transformed point cloud map and the corresponding point to be fused to obtain the fused point cloud map.

[0023] In some embodiments of this application, the accuracy judgment module is further configured to: determine a target matching point from the reference point cloud for each point in the target transformed point cloud; and determine a fusion matching point from the reference point cloud for the point to be fused; determine a first fusion weight for each point in the target transformed point cloud using the distance between each point in the target transformed point cloud and the target matching point; determine a second fusion weight for the point to be fused using the distance between the point to be fused and the fusion matching point; and perform weighted fusion on each point in the target transformed point cloud and the point to be fused based on the first fusion weight and the second fusion weight to obtain the fused point cloud.

[0024] In some embodiments of this application, the depth map to be determined includes: a plurality of consecutive depth maps; the image conversion module is further configured to downsample the plurality of depth maps respectively to obtain a plurality of downsampled depth maps; to perform image conversion on the plurality of downsampled depth maps and the plurality of depth maps to obtain a plurality of point cloud maps; and to determine the plurality of point cloud maps as the converted point cloud maps.

[0025] This application provides an electronic device for judging image accuracy, comprising:

[0026] Memory, used to store executable instructions;

[0027] The processor, when executing executable instructions stored in the memory, implements the image accuracy judgment method provided in the embodiments of this application.

[0028] This application provides a computer-readable storage medium storing executable instructions, which, when executed by a processor, implement the image accuracy judgment method provided in this application.

[0029] This application provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the image accuracy judgment method provided in this application.

[0030] The embodiments of this application have the following beneficial effects: the electronic device can convert the depth map to be judged into a point cloud map, and by calculating the overlap between the converted point cloud map and the reference point cloud map, it can clarify the difference between the three-dimensional coordinate information of each point of the photographed object reconstructed based on the depth value to be judged and the actual three-dimensional coordinate information of each point of the photographed object. Thus, it can determine whether the spatial coordinates of the object determined based on the depth map are consistent with the actual spatial coordinates of the object, thereby improving the judgment effect of the stereo accuracy of the depth map and improving the judgment accuracy of the image. Attached Figure Description

[0031] Figure 1This is a schematic diagram of a speckle structured light imaging system;

[0032] Figure 2 This is a schematic diagram of a point cloud map;

[0033] Figure 3 This is a schematic diagram of the architecture of the image accuracy judgment system provided in the embodiments of this application;

[0034] Figure 4 This is provided by the embodiments of this application. Figure 3 A schematic diagram of the terminal structure in the diagram;

[0035] Figure 5 This is a flowchart illustrating the image accuracy determination method provided in the embodiments of this application. Figure 1 ;

[0036] Figure 6 This is a schematic diagram of the reference point cloud map provided in the embodiments of this application;

[0037] Figure 7 This is a schematic diagram illustrating the conversion of the camera coordinate system to the pixel coordinate system provided in an embodiment of this application;

[0038] Figure 8 This is a schematic diagram of the conversion relationship provided in the embodiments of this application;

[0039] Figure 9 This is a schematic diagram of the camera extrinsic parameters provided in the embodiments of this application;

[0040] Figure 10 This is a flowchart illustrating the image accuracy determination method provided in the embodiments of this application. Figure 2 ;

[0041] Figure 11 This is a schematic diagram of reference points for determining overlap, provided in an embodiment of this application.

[0042] Figure 12 This is a flowchart illustrating the image accuracy determination method provided in the embodiments of this application. Figure 3 ;

[0043] Figure 13 This is a flowchart illustrating the image accuracy determination method provided in the embodiments of this application. Figure 4 ;

[0044] Figure 14 This is a schematic diagram illustrating the principle of determining the accuracy of the depth map captured by the 3D camera in the face payment system, as provided in the embodiments of this application. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0046] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0047] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0049] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0050] 1) A depth camera is a camera capable of capturing the depth of field in a shooting space. Compared to ordinary color cameras, depth cameras can record the distance between objects and the camera, thereby obtaining the three-dimensional coordinates of each point in the image. These three-dimensional coordinates can then be used to reconstruct the real scene and enable applications such as scene modeling.

[0051] 2) A color image is obtained by capturing natural light and forming an image using a color sensor. A color image is represented by a two-dimensional matrix of red, green, and blue triplets (three primary colors). Each value of the three primary colors is between 0 and 255. 0 indicates that the corresponding primary color does not appear in the pixel, while 255 indicates that the corresponding primary color has reached its maximum value in the pixel.

[0052] 3) A depth map is an image that uses pixel values ​​to represent the depth from the image acquisition device to various points in the scene being captured. Depth maps can be obtained by acquiring speckle-structured infrared light from an infrared sensor and then analyzing the speckle. A depth map is an image or image channel that contains depth information from the surface of a common object to the viewpoint. Each pixel in the depth map represents the vertical distance between the depth camera plane and the plane of the object being photographed. It should be noted that although the depth map also contains depth information of objects, the x and y coordinates of the objects are pixel coordinates.

[0053] 4) An infrared image is an infrared image generated by an infrared sensor collecting infrared light.

[0054] 5) Speckle structured light refers to a lattice of light arranged according to a certain structural pattern, projected by an infrared speckle projector.

[0055] 6) Speckle structured light imaging is a common imaging method for depth cameras. In speckle structured light imaging, an infrared laser projector projects speckle structured light onto the object surface, which is then collected by an infrared sensor. Based on the principle of triangulation, the three-dimensional (3D) coordinate information of the object surface can be reconstructed to obtain a depth map.

[0056] For example, Figure 1 This is a schematic diagram of a speckle structured light imaging system. The speckle structured light imaging system 1-1 scans a face 1-2 to obtain a depth map of the face. The speckle structured light imaging system 1-1 includes an infrared speckle projector 1-11 and an infrared sensor 1-12. The infrared speckle projector 1-11 projects a structured light pattern onto the face 1-2, and the infrared sensor 1-12 collects the structured light pattern reflected from the face surface. This allows the spatial information of the face surface to be calculated through the deformation of the structured light pattern, thus obtaining a depth map.

[0057] 7) Point cloud map: Records the three-dimensional (3D) coordinate information of an object in the real world. Point cloud maps are calculated using depth maps and camera parameters. They can be displayed in a 3D rendering engine to reflect the 3D positional relationships of different points.

[0058] For example, Figure 2 This is a schematic diagram of a point cloud. Point cloud 2-1 represents the 3D coordinate information of a hand in the real world.

[0059] 8) Camera intrinsic parameters are parameters used to describe the transformation relationship between the 3D coordinates of a real-world object and the pixel coordinates of the image when the image is captured on the camera sensor. Depth maps and point cloud maps can be converted to each other using camera parameters.

[0060] 9) The camera coordinate system refers to a coordinate system with the camera's optical center as the origin, the optical axis as the z-axis, and the x and y axes parallel to the pixel plane of the camera image. It should be noted that the 3D coordinates obtained through the transformation of the depth map and camera intrinsic parameters are coordinates in the camera coordinate system.

[0061] 10) Camera extrinsic parameters are parameters used to describe the transformation relationship between other 3D coordinate systems and the camera coordinate system. When there are multiple cameras, when the coordinates of an object point in one camera's coordinate system are transformed to another camera's coordinate system using rotation and translation matrices, the rotation and translation matrices are the extrinsic parameters between these different cameras. Thus, camera extrinsic parameters describe the transformation relationship between different camera coordinate systems.

[0062] 11) Liveness detection is the process of determining whether an object detected using a depth camera is a living person. For example, in facial recognition payment, it is usually determined whether the person scanning their face is a real person, a photo, or a silicone model. Generally, depth maps can be used to determine whether the person scanning their face is a photo, while the brightness of infrared images can be used to determine whether the person scanning their face is a silicone model.

[0063] 12) Comparison recognition, which is the process of identifying which user is using facial recognition for payment. Generally, features are extracted from color images, feature similarity is compared, and then depth images are used to assist in comparing the three-dimensional feature similarity to obtain the recognition result.

[0064] A depth map is an image that uses pixel values ​​to represent the depth of points in the scene from the image acquisition device. It reflects the depth information of objects in the scene. Figure 1 Generally, data is collected by a depth camera, which is widely used in liveness detection and comparison recognition in facial payment systems, as well as in autonomous driving systems.

[0065] Before using a depth camera, the accuracy of the depth maps it acquires must be verified to ensure the reliability of the depth maps acquired during actual use.

[0066] In related technologies, the depth of the horizontal plane reflected in the depth map is usually compared with the actual measured depth of the horizontal plane. The accuracy of the depth map is determined by the difference between the depth distance reflected in the depth map and the actual distance. However, this method can accurately determine the distance from the object to the camera plane, i.e., it is suitable for judging the accuracy of the horizontal plane in the depth map. It is difficult to determine whether the spatial coordinates of the object determined by the depth map match the actual spatial coordinates of the object. Therefore, it is not suitable for judging the stereo accuracy of the depth map, resulting in poor judgment of the stereo accuracy of the depth map and ultimately low accuracy of the image accuracy assessment.

[0067] This application provides an image accuracy judgment method, apparatus, device, storage medium, and program product, which can improve the accuracy judgment of images. The following describes exemplary applications of the electronic device provided in this application for image accuracy judgment. The electronic device provided in this application can be implemented as various types of user terminals such as laptops, tablets, desktop computers, and mobile devices, or as a server. The following will describe exemplary applications when the electronic device is implemented as a terminal.

[0068] See Figure 3 , Figure 3 This is a schematic diagram of the architecture of the image accuracy assessment system provided in this application embodiment. To support an image accuracy assessment application, in Figure 3 In the image accuracy judgment system 100 shown, the terminal 400 is connected to the depth camera 200 and the high-precision scanner 500 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0069] Terminal 400 is used to acquire a depth map to be judged from depth camera 200 and a reference point cloud map of the subject contained in the depth map to be judged from high-precision scanner 500. The reference point cloud map is a point cloud map with an accuracy not less than an accuracy threshold obtained by high-precision scanner 500, and records the real three-dimensional coordinate information of each point of the subject. The point cloud map to be judged is converted to obtain a converted point cloud map, which records the three-dimensional coordinate information of the subject reconstructed based on the depth values ​​recorded in the depth map to be judged. The accuracy of the depth map to be judged is determined by performing overlap detection on the point cloud map to be judged and the reference point cloud map, thereby realizing image accuracy judgment.

[0070] In some embodiments, the terminal 400 may be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smartwatch, smart home appliance, vehicle terminal, etc., but is not limited thereto. The terminal 400 and the depth camera 200 and high-precision scanner 500 may be directly or indirectly connected via wired or wireless communication, and no limitation is imposed in this embodiment of the invention.

[0071] See Figure 4 , Figure 4 This is provided by the embodiments of this application. Figure 3 A schematic diagram of the terminal structure in the diagram. Figure 4The terminal 400 shown includes at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. The various components in the terminal 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 4 The general labeled all buses as Bus System 440.

[0072] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0073] User interface 430 includes one or more output devices 431 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0074] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0075] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0076] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0077] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0078] The network communication module 452 is used to reach other computing devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including: Bluetooth, Wi-Fi, and Universal Serial Bus (USB), etc.

[0079] Presentation module 453 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 431 (e.g., a display screen, a speaker, etc.) associated with user interface 430;

[0080] The input processing module 454 is used to detect and translate one or more user inputs or interactions from one or more input devices 432.

[0081] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 4 An image accuracy determination device 455 stored in memory 450 is shown. This device can be software in the form of programs or plug-ins, and includes the following software modules: an image acquisition module 4551, an image conversion module 4552, an accuracy determination module 4553, and a parameter determination module 4554. These modules are logically connected and can therefore be arbitrarily combined or further separated according to their implemented functions. The functions of each module will be described below.

[0082] In other embodiments, the image accuracy determination device provided in this application can be implemented in hardware. As an example, the image accuracy determination device provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the image accuracy determination method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0083] For example, this application provides an electronic device for image accuracy determination, comprising:

[0084] Memory, used to store executable instructions;

[0085] The processor, when executing executable instructions stored in the memory, implements the image accuracy judgment method provided in the embodiments of this application.

[0086] In some embodiments, an electronic device can implement the image accuracy determination method provided in this application by running a computer program. For example, the computer program can be a native program or software module in an operating system; it can be a native application (APP), i.e., a program that needs to be installed in the operating system to run; it can also be a small program, i.e., a program that only needs to be downloaded to a browser environment to run; or it can be a small program that can be embedded in any APP. In short, the above-mentioned computer program can be any form of application, module, or plugin.

[0087] The image accuracy judgment method provided in this application can be applied to various scenarios such as cloud technology, artificial intelligence, smart transportation, and vehicle-mounted systems. Below, we will describe the image accuracy judgment method provided in this application by combining exemplary applications and implementations of the electronic device for image accuracy judgment provided in this application.

[0088] See Figure 5 , Figure 5 This is a flowchart illustrating the image accuracy determination method provided in the embodiments of this application. Figure 1 , will combine Figure 5 The steps shown are explained.

[0089] S101. Obtain the depth map to be judged, and the reference point cloud map of the shooting object contained in the depth map to be judged.

[0090] This application's embodiments are implemented in scenarios involving the determination of the accuracy of depth maps. For example, the accuracy of depth maps captured by depth cameras in facial payment systems or autonomous driving systems is determined to clarify whether these depth cameras are accurate. Alternatively, the accuracy of any depth map can be determined to ascertain whether it needs to be re-acquired. Electronic devices can obtain the depth map to be determined and the reference point cloud map from their own storage space, download them from the network, or directly access the connected depth camera and high-precision scanner for scanning point cloud maps. The depth camera captures an image of the subject to obtain the depth map to be determined, and the high-precision scanner scans the subject to obtain the reference point cloud map.

[0091] It should be noted that, in order to accurately reflect the three-dimensional coordinate information of each point on the subject, the reference point cloud map must be a point cloud map obtained through scanning with an accuracy not less than a certain accuracy threshold. In other words, the reference point cloud map is considered an accurate point cloud map of the subject, thus recording the true three-dimensional coordinate information of each point on the subject. Understandably, the subject in the depth map to be judged and the reference point cloud map must be the same; only in this way can the accuracy of the depth map to be judged be accurately determined.

[0092] It is understood that the subject of the photograph can be a special silicone head mold, or it can be a human face, common objects, etc. This application does not limit it.

[0093] In some embodiments, the depth map to be determined may include multiple depth maps, which may be obtained by continuously capturing images of the subject with a depth camera. These multiple depth maps may be obtained from the same shooting angle or from different shooting angles. Similarly, the reference point cloud map may also include multiple reference maps, which may be obtained by scanning the subject from the same angle or from different angles; this application does not impose any limitations on this.

[0094] For example, Figure 6 This is a schematic diagram of the reference point cloud map provided in an embodiment of this application. Image 6-1 shows the photographed object 6-11. The electronic device can use a high-precision scanner to scan the photographed object 6-11 from both the side and front angles to obtain the side reference point. Figure 6-2 and positive benchmark Figure 6-3 ,。 Side reference Figure 6-2 and positive benchmark Figure 6-3 It is a baseline cloud map.

[0095] S102. Convert the depth map to be judged into a point cloud map to obtain the converted point cloud map.

[0096] The electronic device acquires the camera parameters corresponding to the depth map to be judged, and then transforms the pixels in the depth map to be judged from the pixel coordinate system to the three-dimensional camera coordinate system based on the camera parameters. That is, it determines the physical coordinates of each pixel in the depth map in the real world, thereby obtaining the transformed point cloud map.

[0097] In other words, the converted point cloud map records the three-dimensional coordinate information of each point of the photographed object reconstructed based on the depth values ​​recorded in the depth map to be judged. The accuracy of the depth map to be judged directly determines the accuracy of this three-dimensional coordinate information. Therefore, by jointly analyzing it with the three-dimensional coordinate information reflected in the reference point cloud map, it is possible to determine whether the depth map to be judged is accurate.

[0098] It's understandable that the camera parameters corresponding to the depth map to be determined refer to the intrinsic camera parameters of the depth camera that captured the depth map. The process by which electronic devices convert points in the camera coordinate system to points in the pixel coordinate system using these intrinsic parameters can be understood through… Figure 7 To illustrate.

[0099] Figure 7 This is a schematic diagram illustrating the conversion of the camera coordinate system to the pixel coordinate system provided in an embodiment of this application. Figure 7 In this system, Oc-XcYcZc is the camera coordinate system, with the optical center Oc as its origin and the unit being m; o-xy is the image coordinate system, with the optical center being the midpoint of the image and the unit being m; uv is the pixel coordinate system, with the origin being the upper left corner of the image and the unit being pixels; P(Xc, Yc, Zc) is a point in the world coordinate system; p(x, y) is the image point of point P in the image, with coordinates (x, y) in the image coordinate system (oxy) and (u, v) in the pixel coordinate system (ouv); and f is the focal length. The transformation from the camera coordinate system to the pixel coordinate system can be expressed as equation (1):

[0100]

[0101] Where (u0, v0) represents the coordinates of the origin of the pixel coordinate system, f x =f / dx, f y =f / dy.

[0102] It should be noted that camera intrinsic parameters reflect the transformation relationship between the 3D coordinate information of the depth camera during imaging and the pixel coordinates after imaging. For example, Figure 8 This is a schematic diagram illustrating the conversion relationship provided in an embodiment of this application. See also... Figure 8 The perpendicular distance from the optical center O8-11 on the camera plane 8-1 (coordinate system Oxyz) to the imaging plane 8-2 (O'-x'-y'-z) is the focal length 8-3. The projection of any point P in three-dimensional space onto the imaging plane 8-2 through the optical center 8-11 is P'. Based on this, similar triangles 8-4 can be obtained, such as the triangle formed by OAP and the triangle formed by OB-P'. There is a transformation relationship between x' and f in similar triangles 8-4 and the X and Z coordinates of the real point P.

[0103] Furthermore, the depth camera can be a stereo camera. When generating a depth map using a stereo camera, it is first necessary to transform the images captured by the two cameras into the same camera coordinate system, that is, to determine which two points in the left and right cameras correspond. Camera extrinsic parameters describe the transformation relationship between the two camera coordinate systems.

[0104] For example, Figure 9This is a schematic diagram of the camera extrinsic parameters provided in an embodiment of this application. In the binocular camera, the center of the left camera is c0, the center of the right camera is c1, and P is any point in space. The projections of P onto the image planes corresponding to c0 and c1 are x0 and x1, respectively. The intersection points e0 and e1 of the line connecting c0 and c1 with image planes 9-1 and 9-2 are poles, and l0 and l1 are epipolar lines. c0, c1, and P form the epipolar plane 9-3. It can be seen that the camera coordinate system of the left camera and the camera coordinate system of the right camera can be transformed using a rotation matrix R and a translation matrix t, i.e., (R, t). The rotation matrix R and the translation matrix t are the camera extrinsic parameters.

[0105] It should be noted that when the depth map to be judged includes multiple depth maps, the electronic device will convert all the depth maps into point cloud maps, thus the converted point cloud map includes multiple point cloud maps.

[0106] S103. Based on the overlap detection of the transformed point cloud map and the reference point cloud map, the accuracy of the depth map to be judged is determined, and the image accuracy judgment is realized.

[0107] The degree of overlap between the point cloud image to be judged and the reference point cloud image can indicate the difference between the 3D coordinate information of the photographed object reconstructed from the depth map to be judged and the actual 3D coordinate information of the photographed object, thus reflecting the stereo accuracy of the depth map to be judged. Therefore, in this embodiment, the electronic device calculates the degree of overlap between the point cloud image to be judged and the reference point cloud image, and then determines the accuracy of the depth map to be judged based on the degree of overlap. In this way, the stereo accuracy of the depth map to be judged can be accurately judged, thereby enabling a more precise determination of the accuracy of the depth map.

[0108] It should be noted that electronic devices can directly determine the accuracy of the depth map to be judged by the degree of overlap between the converted point cloud map and the reference point cloud map. They can also compare the degree of overlap between the converted point cloud map and the reference point cloud map with the thresholds corresponding to multiple overlap levels, and determine the accuracy of the depth map to be judged by the preset accuracy of the overlap level to which the degree of overlap is hit. This application does not limit this.

[0109] Understandably, when the converted point cloud map includes multiple point cloud maps, the electronic device can fuse the multiple point cloud maps and then perform overlap detection on the fusion result with the multiple reference maps contained in the reference point cloud map. Alternatively, it can first perform overlap detection on each point cloud map with the reference map at the same angle as it in the multiple reference maps to obtain multiple overlap values, and then perform weighted fusion on the multiple overlap values ​​to obtain the accuracy of the depth map to be judged.

[0110] In this embodiment, compared to the related art method of determining image accuracy based on the depth of the horizontal plane reflected by the depth map and the actual depth of the measured horizontal plane, the depth map to be judged can be converted into a point cloud map. By calculating the overlap between the converted point cloud map and the reference point cloud map, the difference between the three-dimensional coordinate information of each point of the photographed object reconstructed based on the depth value to be judged and the actual three-dimensional coordinate information of each point of the photographed object can be clarified. This allows for the determination of whether the spatial coordinates of the object determined based on the depth map match the actual spatial coordinates of the object, improving the judgment effect of the stereo accuracy of the depth map, and ultimately improving the accuracy of the image accuracy judgment. Furthermore, since the conversion of the depth map into a point cloud map requires the help of camera parameters, calculating the overlap between the converted point cloud map and the reference point cloud map can also help determine the accuracy of the camera parameters of the depth camera.

[0111] based on Figure 5 See Figure 10 , Figure 10 This is a flowchart illustrating the image accuracy determination method provided in the embodiments of this application. Figure 2 In some embodiments of this application, the accuracy of the depth map to be judged is determined based on the overlap detection of the transformed point cloud map and the reference point cloud map. The specific implementation process of S103 may include: S1031-S1032, as follows:

[0112] S1031. By registering the transformed point cloud map with the reference point cloud map, multiple matching point pairs are generated.

[0113] Electronic devices register a transformed point cloud map with a reference point cloud map. For each point in the transformed point cloud map, a corresponding matching point is found in the reference point cloud map. Multiple matching point pairs are generated by pairing each point with its matching point. Each point in a matching point pair represents the reconstructed 3D coordinate information of a point in the real world (the point in the transformed point cloud map) and the actual 3D coordinate information of the point in the real world (the point in the reference point cloud map).

[0114] S1032. Based on the overlap calculation of multiple matching point pairs, the accuracy of the depth map to be judged is obtained.

[0115] The electronic device calculates the degree of overlap for each pair of matching points, and then analyzes the degree of overlap of each pair of matching points to clarify the difference between the converted point cloud map and the reference point cloud map. This allows for an accurate analysis of the accuracy of the depth map to be judged.

[0116] It is understood that electronic devices can determine the degree of overlap of each matching point pair by calculating the spatial distance between each point in each matching point pair, or by calculating the projected positions of the two points in each matching point pair, or by other feasible methods, which are not limited herein.

[0117] In this embodiment of the application, the electronic device registers the converted point cloud map with the reference point cloud map to obtain multiple matching point pairs. Then, it calculates the degree of overlap in the dimension of the matching point pairs, that is, it judges the accuracy of the point dimension for the depth map to be judged. As a result, the accuracy judgment dimension is more detailed and the accuracy is more precise.

[0118] In some embodiments of this application, the accuracy of the depth map to be judged is obtained based on the overlap calculation of multiple matching point pairs, i.e., the specific implementation process of S1032 may include: S1032a-S1032c, as follows:

[0119] S1032a. Calculate the distance between at least two points contained in each of the multiple matching point pairs to obtain the distance between the multiple point pairs.

[0120] In this embodiment, each matching point pair contains at least two points. The electronic device calculates the distance between the at least two points in each matching point pair and uses the calculated distance as the point-to-point distance for each matching point pair. When the electronic device completes the distance calculation for multiple matching point pairs, it obtains multiple point-to-point distances.

[0121] Understandably, when each matching point pair contains only two points, the electronic device determines the point-to-point distance as the spatial distance between these two points. When each matching point pair contains more than two points, the electronic device calculates the distance between each pair of these points and then uses the average of these distances as the final point-to-point distance.

[0122] S1032b. Based on the distances between multiple point pairs, determine the degree of overlap between the transformed point cloud map and the reference point cloud map.

[0123] Electronic devices analyze the distances between multiple point pairs to clarify the reconstructed 3D coordinates of each point in the real world and the distance between the reconstructed 3D coordinates and the actual 3D coordinates. Based on this distance, the difference between the converted point cloud map and the reference point cloud map can be determined, and the degree of overlap can be obtained.

[0124] In some embodiments, the electronic device may directly determine the minimum or maximum distance of multiple point pairs as the degree of overlap, or it may average the distances of multiple point pairs and use the average distance as the degree of overlap, or it may filter out the point pairs corresponding to points in specific regions of the point cloud map from multiple point pairs, and analyze the point pairs corresponding to these specific regions to obtain the degree of overlap, etc. This application does not limit these embodiments.

[0125] S1032c: Convert the overlap ratio into the accuracy of the depth map to be judged.

[0126] Electronic devices can directly determine the overlap ratio as the accuracy of the depth map to be judged, or they can perform conversion processing such as rounding, halving, or percentage calculation on the overlap ratio to obtain the accuracy of the depth map to be judged.

[0127] For example, when the overlap is 0.823, the electronic device can round it down to determine the accuracy of the depth map to be judged as 0.8, etc., which is not limited in this application.

[0128] In this embodiment of the application, the electronic device calculates the point-to-point distance of the matching point pairs and analyzes the point-to-point distance to obtain the accuracy of the depth map to be judged, thereby enabling the measurement of the accuracy of the depth map to be judged in the dimension of spatial distance, and the obtained accuracy is more precise.

[0129] In some embodiments of this application, the overlap between the transformed point cloud map and the reference point cloud map is calculated based on the distances between multiple point pairs. The specific implementation process of S1032b may include: S201-S202, as follows:

[0130] S201. From multiple point-to-point distances, select the first distances corresponding to the N points closest to the origin in the transformed point cloud map, and the second distances corresponding to the N points farthest from the origin.

[0131] The electronic device first selects N points from the points contained in the transformed point cloud map that are closest to the origin of the camera coordinate system, and N points that are furthest from the origin, as reference points for determining the overlap. Each point in the transformed point cloud map has its own matching point pair, and there is a correspondence between the matching point pairs and the distances between them. Thus, the electronic device can filter out the first distances corresponding to the N points closest to the origin from multiple point pair distances, thus obtaining N first distances, and filter out the second distances corresponding to the N points farthest from the origin, thus obtaining N second distances. These N first distances and N second distances are then used to analyze the error distribution of point pairs in different regions of the transformed point cloud map.

[0132] It is understood that N is a positive integer. The value of N can be pre-set, such as 100, 200, etc., or it can be calculated based on the total number of points contained in the converted point cloud map and the point selection ratio automatically generated by the electronic device. The point selection ratio refers to the proportion of the points to be extracted to the total number of points contained in the converted point cloud map, such as 30%, 50%, etc. The point selection ratio can be randomly generated by the electronic device, or it can be automatically selected by the electronic device based on time, the total number of matching point pairs, etc.

[0133] For example, Figure 11 This is a schematic diagram of reference points provided in an embodiment of this application for determining the degree of overlap. Figure 11 In the process, black solid dots represent each point 11-1 in the transformed point cloud map. The electronic device selects the 30% of points 11-3 (containing 2 points) that are closest to the origin 11-2 and the 30% of points 11-4 (containing 3 points) that are farthest from the origin 11-2 as reference points for the overlap judgment, and extracts the point-to-point distances corresponding to these points.

[0134] S202. The difference between the mean of N first distances and the mean of N second distances is determined as the overlap between the transformed point cloud map and the reference point cloud map.

[0135] The electronic device calculates the mean of N first distances, and simultaneously calculates the mean of N second distances. Then, the electronic device calculates the difference between the mean of the N first distances and the mean of the N second distances; the calculated difference represents the accuracy between the converted point cloud map and the baseline point cloud map.

[0136] It is understood that electronic devices can obtain the difference result by subtracting the mean of N first distances from the mean of N second distances, or by comparing the mean of N first distances with the mean of N second distances; this application does not limit this.

[0137] In this embodiment of the application, the electronic device can determine the degree of overlap between the converted point cloud map and the reference point cloud map by calculating the error of points in different regions of the converted point cloud map. This not only makes the method of calculating the degree of overlap by point-to-point distance more diverse, but also makes the degree of overlap more accurate by comparing the errors of different regions.

[0138] In some embodiments of this application, the overlap between the transformed point cloud map and the reference point cloud map is calculated based on the distances between multiple point pairs. The specific implementation process of S1032b may include: S203-S204, as follows:

[0139] S203. From multiple point-to-point distances, select the third distances corresponding to M points in the target area of ​​the transformed point cloud map.

[0140] The electronic device filters out M points in the target area of ​​the filtering and conversion point cloud map as reference points for judging the degree of overlap. Then, from multiple point pair distances, it obtains the point pair distance corresponding to the matching point pair to which each of the M points belongs, and records the point pair distance as the third distance. Thus, the electronic device can obtain M third distances.

[0141] It is understood that the target area can be any area in the transformed point cloud map, such as the 30% area closest to the origin of the camera coordinate system, or the area with the densest points in the transformed point cloud map. This application does not limit this.

[0142] S204. Calculate the Gaussian distribution variance of the M third distances to obtain the overlap between the transformed point cloud map and the reference point cloud map.

[0143] The electronic device fits a Gaussian distribution to M third distances, calculates the variance of the fitted Gaussian distribution, and determines the degree of overlap by the calculated variance, where M is a positive integer.

[0144] In this embodiment of the application, the electronic device can calculate the degree of overlap based on the statistical analysis of the point-to-point distances between points in the same area. This not only makes the methods for calculating the degree of overlap more diverse, but also makes the degree of overlap more accurate.

[0145] based on Figure 5 See Figure 12 , Figure 12 This is a flowchart illustrating the image accuracy determination method provided in the embodiments of this application. Figure 3 In some embodiments of this application, after converting the depth map to be determined into a point cloud map to obtain the converted point cloud map, i.e., after S102, the method may further include: S104-S105, as follows:

[0146] S104. Determine the reconstructed shape parameters of the photographed object based on the transformed point cloud map, and determine the reference shape parameters of the photographed object based on the reference point cloud map.

[0147] The electronic device measures the radius, side length, height, and other shape-related parameters of the photographed object based on the converted point cloud map, and records these parameters as the reconstructed shape parameters. Simultaneously, the electronic device also measures the radius, side length, height, and other parameters of the photographed object based on the reference point cloud map. Since the reference point cloud map is accurate, the reference shape parameters obtained from it are also accurate.

[0148] S105. By reconstructing the difference between the shape parameters and the reference shape parameters, the accuracy of the depth map to be judged is determined.

[0149] The electronic device compares the reconstructed shape parameters with the reference shape parameters to determine the differences between them, such as the difference in radius or height. Then, the electronic device further analyzes and processes the differences, for example, by using the reciprocal of the difference in radius or height as the accuracy, or by determining the difference between the percentage of the difference and 1 as the accuracy, etc., which are not limited herein.

[0150] Furthermore, the subject of the photograph can be required to be a standard-shaped object, such as a cube, cylinder, or sphere, in order to accurately obtain the depth map to be determined.

[0151] In this embodiment of the application, the electronic device can also determine the accuracy of the depth map to be judged based on the shape measurement of the photographed object, thereby making the method of judging accuracy more diversified.

[0152] based on Figure 10 See Figure 13 , Figure 13 This is a flowchart illustrating the image accuracy determination method provided in the embodiments of this application. Figure 4 In some embodiments of this application, the converted point cloud map includes: multiple point cloud maps, and multiple matching point pairs are generated by registering the converted point cloud map with the reference point cloud map. Specifically, the implementation process of S1031 may include: S1031a-S1031d, as follows:

[0153] S1031a. For each point in each of the multiple point cloud maps, select the candidate point with the smallest distance from the reference point cloud map.

[0154] The electronic device calculates the distance between each point in each point cloud map and each point in the reference point cloud map. Then, from the reference point cloud map, it selects the point with the smallest distance to each point in each point cloud map and determines that point as the candidate point corresponding to each point.

[0155] In other words, for each point in each point cloud map, the electronic device indexes candidate points in the baseline point cloud map, with the goal of minimizing the distance.

[0156] Understandably, electronic devices can calculate the spatial distance between the 3D coordinates of each point in each point cloud map and the 3D coordinates of the reference point cloud map, thus obtaining the distance between each point and each point in the reference point cloud map. The formula for spatial distance is shown in equation (2):

[0157]

[0158] Where (x1,y1,z1) represents the three-dimensional coordinate information of each point in each point cloud map, (x2,y2,z2) represents the three-dimensional coordinate information of each point in the reference point cloud map, and E is the calculated distance.

[0159] S1031b: Based on the rigid body transformation between each point in each point cloud map and the corresponding candidate point, generate a transformed cloud map corresponding to each point cloud map.

[0160] The electronic device solves for the transformation parameters of each point in each point cloud map and its corresponding candidate points when performing rigid body transformations. Then, it uses these transformation parameters to transform each point cloud map to obtain the transformed point cloud map corresponding to each point cloud map.

[0161] It should be noted that the transformed point cloud map in this application embodiment refers to the transformed point cloud map whose average distance to each point cloud map (i.e., the average distance of each point) is less than a preset threshold. Before obtaining the transformed point cloud map, the device may perform multiple iterative transformations on each point cloud map until the distance threshold requirement is reached, and then the point cloud map obtained by the last iterative transformation is determined as the transformed point cloud map.

[0162] In other words, the electronic device first transforms each point cloud map using transformation parameters to obtain a new point cloud map. Then, it calculates the average distance between the new point cloud map and each of the original point cloud maps and determines whether the average distance is less than a distance threshold. If it is less than the distance threshold, the new point cloud map is used as the transformed point cloud map. If it is not less than the distance threshold, the new point cloud map will continue to be transformed until a transformed point cloud map that meets the distance threshold requirement is obtained.

[0163] S1031c: From the baseline point cloud map, select the matching point that has the smallest distance to each point in the transformed point cloud map corresponding to each point in the base point cloud map, and generate multiple transformed point pairs using each point in the transformed point cloud map and the corresponding matching point.

[0164] For each point in the transformed point cloud map, the electronic device searches for the point with the smallest distance from the reference point cloud map, and determines the found point as the matching point. Then, the electronic device pairs each point in the transformed point cloud map with its corresponding matching point to generate multiple transformed point pairs.

[0165] S1031d. Merge the transformed point cloud maps corresponding to each point cloud map to obtain a merged point cloud map, and for each point in the merged point cloud map, determine multiple matching point pairs from multiple transformed point pairs.

[0166] Each point cloud map's transformed point cloud map might be a point cloud map from different angles of the subject, or multiple point cloud maps from the same angle of the subject. To improve the quality of the subject's point cloud map, the electronic device fuses the transformed point cloud maps corresponding to each point cloud map to obtain a global point cloud map of the subject, or a higher-quality point cloud map from the same angle. The resulting point cloud map is called the fused point cloud map. During fusion, the electronic device may merge some different points into a single point or discard some points. Thus, each point in the fused point cloud map is a subset of the points in the transformed point cloud maps corresponding to all the point cloud maps. Based on each point in the fused point cloud map, the electronic device determines its corresponding point pair from the transformed point pairs, thereby obtaining multiple matching point pairs.

[0167] Understandably, when an electronic device merges different points into a single point, it can perform fusion on the corresponding transform point pairs to which these points belong, and the resulting merged point pairs are the matching point pairs. When an electronic device obtains a merged point cloud map by discarding some points, it can determine the matching point pairs only by identifying the transform point pairs to which the remaining points belong.

[0168] In this embodiment of the application, the electronic device can first filter candidate points for each point in each point cloud map, then transform each point cloud map, and then continue to filter matching points, thereby more accurately determining the matching point corresponding to each point. Finally, the transformed point cloud maps corresponding to each point cloud map are fused to obtain a point cloud map of better quality. Through the matching point pairs formed by the points in the point cloud map of better quality and the points in the reference point cloud map, the degree of overlap can be calculated more accurately in the subsequent process.

[0169] In some embodiments of this application, the transformation parameters include rotation parameters and translation parameters. In this case, based on the rigid body transformation between each point in each point cloud map and the corresponding candidate point, a transformed point cloud map corresponding to each point cloud map is generated. That is, the specific implementation process of S1031b may include: S301-S302, as follows:

[0170] S301. By performing a rigid body transformation with the minimum distance on each point in each point cloud map and its corresponding candidate point, the rotation and translation parameters are determined.

[0171] S302. Perform the transformation corresponding to the rotation and translation parameters for each point cloud map to obtain the transformed point cloud map corresponding to each point cloud map.

[0172] The electronic device solves for the minimum rigid body transformation for each point in each point cloud map and its corresponding candidate point, and obtains the rotation and translation parameters that minimize the distance between each point and its corresponding candidate point when performing rigid body transformation. Then, each point cloud map is rotated according to the rotation parameters and translated according to the translation parameters to obtain the corresponding transformed point cloud map.

[0173] Understandably, the rotation parameter represents the amount of rotation required for each point cloud map, while the translation parameter represents the amount of translation required for each point cloud map.

[0174] In this embodiment of the application, the electronic device can perform corresponding transformations on each point cloud map according to the obtained rotation and translation parameters, so that each point cloud map becomes a transformed point cloud map after rigid body transformation.

[0175] In some embodiments of this application, the transformed point cloud maps corresponding to each point cloud map are fused to obtain a fused point cloud map. The specific implementation process of S1031d may include: S303-S306, as follows:

[0176] S303. Select the target transformed point cloud map from the transformed point cloud maps corresponding to each point cloud map.

[0177] The electronic device can select any one of the transformed point cloud maps corresponding to each point cloud map as the target transformed point cloud map, or it can determine the transformed point cloud map corresponding to the first point cloud map as the target transformed point cloud map. This application does not limit this.

[0178] S304. For each point in the target transformed point cloud map, calculate the normal difference and spatial distance between it and each point in other transformed point cloud maps.

[0179] Among them, other transformed point cloud maps refer to the transformed point cloud maps other than the target transformed point cloud map in the transformed point cloud map corresponding to each point cloud map.

[0180] In other words, the electronic device calculates the normal difference and spatial distance between each point in the fused reference point cloud map and each point in other transformed point cloud maps (excluding the target transformed point cloud map). The normal difference refers to the difference between the normal vectors.

[0181] S305. Based on the normal difference and spatial distance, for each point in the target transformed point cloud map, select the points to be fused from the other transformed point cloud maps.

[0182] The electronic device filters out points in other transformed point cloud maps whose normal difference from each point in the target transformed point cloud map is less than the difference threshold and whose spatial distance is less than the distance threshold. The filtered points are those that are close enough to each point in the target transformed point cloud map and can be used for fusion to obtain the points to be fused.

[0183] It should be noted that in other transformed point cloud maps, if the normal difference between a point and a point in the target transformed point cloud map is not less than the difference threshold, or the spatial distance is not less than the distance threshold, the electronic device can discard the point during the fusion process. In this case, the electronic device can directly determine the point as a point in the fused point cloud map, or it can discard the point, that is, if no corresponding point to be fused can be found, it will not participate in the fusion process.

[0184] S306. Weighted fusion of each point in the target transformed point cloud map and the corresponding point to be fused is performed to obtain the fused point cloud map.

[0185] The electronic device calculates a fusion weight for each point in the target transformed point cloud map, and then uses this weight to perform weighted fusion of each point in the target transformed point cloud map with the corresponding point to be fused. The resulting fusion result is the fused point cloud map.

[0186] In some embodiments of this application, each point of the target transformation point cloud map and its corresponding point to be fused are weighted and fused to obtain a fused point cloud map. The specific implementation process of S306 may include: S3061-S3064, as follows:

[0187] S3061. For each point in the target transformation point cloud map, determine the target matching point from the reference point cloud map, and for the point to be fused, determine the fusion matching point from the reference point cloud map.

[0188] In the aforementioned steps, the electronic device determined a matching point from the reference point cloud map for each point in the transformed point cloud map. Since the target transformed point cloud map and other transformed point cloud maps are both transformed point cloud maps, each point in the target transformed point cloud map also has a corresponding matching point. The electronic device records the matching point corresponding to each point in the target transformed point cloud map as the target matching point, and records the matching points corresponding to the points to be fused in other transformed point cloud maps as fusion matching points.

[0189] S3062. Using the distance between each point in the target transformation point cloud map and the target matching point, determine the first fusion weight of each point in the target transformation point cloud map.

[0190] S3063. Determine the second fusion weight of the point to be fused by using the distance between the point to be fused and the fusion matching point.

[0191] The electronic device calculates the distance between each point in the target transformed point cloud map and its corresponding target matching point to obtain the first matching point distance. It then calculates the distance between the point to be fused and the fusion matching point to obtain the second matching point distance. Next, the electronic device performs an inverse proportional fitting process on the absolute value of the first matching point distance, such as taking the reciprocal (subtracting from 1 if the distance is less than 1, and taking the reciprocal if the distance is greater than 1), to obtain the first fusion weight. Similarly, the electronic device determines the second fusion weight based on the distance between the point to be fused and the fusion matching point.

[0192] In other words, the first fusion weight and the distance to the first matching point, and the second fusion weight and the distance to the second matching point are inversely proportional; the smaller the distance, the larger the corresponding fusion weight.

[0193] It is understood that in some embodiments, the electronic device may execute S3063 first and then S3062, or it may execute S3062 and S3063 simultaneously. This application does not limit this.

[0194] S3064. Based on the first fusion weight and the second fusion weight, each point of the target transformed point cloud map and the point to be fused are weighted and fused to obtain the fused point cloud map.

[0195] The electronic device adds a first fusion weight to the three-dimensional coordinate information of each point in the target transformed point cloud map, and adds a second fusion weight to the three-dimensional coordinate information of the point to be fused. Then, the two weighted three-dimensional coordinate information are added together, which realizes the weighted fusion of each point in the target transformed point cloud map and the corresponding point to be fused, thereby obtaining the fused point cloud map.

[0196] In this embodiment of the application, the electronic device can determine the fusion weight that is inversely proportional to the distance by the distance between each point of the target transformed point cloud map and the corresponding target matching point, as well as the distance between the point to be fused and the fusion matching point. The fusion weight is used to perform weighted fusion, so that the points that are more similar to the real points in the real world play a greater role in the fusion, thereby improving the accuracy of the fused point cloud map.

[0197] In some embodiments of this application, the depth map to be judged includes: multiple consecutive depth maps. A point cloud map is converted from the depth map to be judged to obtain a converted point cloud map. The specific implementation process of S102 may include: S1021-S1023, as follows:

[0198] S1021. Downsample multiple depth maps to obtain multiple downsampled depth maps.

[0199] Electronic devices can reduce the size of multiple depth maps by a factor of one or several by means of mean downsampling or maximum downsampling, and then define the reduced depth map as the downsampled depth map, thus obtaining multiple downsampled depth maps.

[0200] It is understandable that the number of downsampled depth maps can be equal to the number of depth maps, meaning that the electronic device performs downsampling only once for each depth map, thus obtaining only one corresponding downsampled depth map for each depth map; the number of downsampled depth maps can also be greater than the number of depth maps, meaning that the electronic device performs downsampling several times for each depth map, thus obtaining several downsampled depth maps for one depth map.

[0201] S1022. Perform image conversion on multiple downsampled depth maps and multiple depth maps to obtain multiple point cloud maps.

[0202] S1023. Select multiple point cloud maps as the conversion point cloud map.

[0203] Since depth maps with different resolutions may have different characteristics, in order to comprehensively analyze the accuracy of the depth map to be judged, the electronic device will convert multiple downsampled depth maps and multiple depth maps into point cloud maps, thereby obtaining multiple point cloud maps. Finally, the multiple point cloud maps are used to form a converted point cloud map to determine the degree of overlap with the reference depth map.

[0204] In this embodiment, the electronic device can obtain depth maps at different resolutions by downsampling, and convert the depth maps at different resolutions into point cloud maps. This allows for overlap detection of the point cloud maps corresponding to the depth maps at different resolutions during subsequent overlap detection with the reference point cloud map, thereby improving the comprehensiveness of overlap detection and increasing the accuracy of image accuracy judgment.

[0205] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0206] This embodiment of the application is implemented by judging the accuracy of the depth map captured by the 3D camera in the face payment system. The 3D camera used can be a speckle imaging system or a color binocular imaging system.

[0207] Figure 14 This is a schematic diagram illustrating the principle of determining the accuracy of the depth map captured by the 3D camera in the facial payment system, as provided in this application embodiment. The terminal (electronic device) first processes the captured depth... Figure 14-1 (Depth map to be determined) is transformed using camera intrinsic parameter 14-2 to obtain point cloud. Figure 14-3 (Convert point cloud image), and generate standard point clouds using a high-precision scanner for objects such as human face head models (subjects). Figure 14-4 (Baseline point cloud map), comparing the standard point cloud map with the depth data captured by the 3D camera. Figure 14-1 The generated point cloud Figure 14-3 The overlap of the point cloud is detected (14-5) (overlap detection is performed on the transformed depth map and the reference depth map) to determine the reconstruction accuracy of the 3D camera, i.e., the accuracy of the depth map (14-6).

[0208] Furthermore, the specific steps of this process are as follows:

[0209] Step 1: Convert the depth map to a point cloud map (convert the depth map to be judged into a point cloud map to obtain the converted point cloud map). The coordinate transformation relationship between the depth map and the point cloud map can be calculated using the camera intrinsic parameters. The formula for the conversion can be found in Equation (1).

[0210] Step 2: Perform mean downsampling on the depth map (downsample multiple depth maps separately), reduce the size by half, and then recalculate the 3D points and normal vectors to convert them into point cloud maps (multiple downsampled depth maps, as well as multiple depth maps, are image converted to obtain multiple point cloud maps).

[0211] Step 3: Use a high-precision scanner to scan objects such as human head models to generate high-precision standard point cloud maps (the benchmark point cloud map is a point cloud map obtained by scanning with an accuracy not less than the accuracy threshold).

[0212] Step 4: Determine the corresponding point pairs between the point cloud image generated from the depth map captured by the 3D camera and the high-precision standard point cloud image, i.e., register the point cloud image with the standard point cloud image (by registering the transformed point cloud image with the reference point cloud image, multiple matching point pairs are generated). This process includes the following steps:

[0213] Step 1: Assume that the point cloud map generated based on the depth map captured by the 3D camera is X2, and the high-precision standard point cloud map is X1. Calculate the nearest point in X1 for each point in X2 (for each point in each point cloud map of multiple point cloud maps, select the candidate point with the smallest distance from the reference point cloud map). This calculation process can be achieved by minimizing equation (2).

[0214] Step 2: Solve for the rigid body transformation that minimizes the average distance between the point pairs in Step 1 (perform the minimum distance rigid body transformation between each point in the point cloud and its corresponding candidate point) to obtain the translation and rotation parameters.

[0215] Step 3: Apply the translation and rotation parameters obtained in step 2 to X2 to obtain a new set of transformation points.

[0216] Step 4: If the average distance between the new set of transformed points and the reference set, i.e., X2, is less than a given threshold, then stop the iteration; otherwise, the new set of transformed points is used as the new X2 and the iteration continues until the given threshold requirement is met, and the transformed X2 (transformed point cloud map) is obtained.

[0217] Step 5: Calculate the nearest point in X1 for each point of the transformed X2, that is, determine the final point pair (multiple transformed point pairs).

[0218] At this point, the registration process between the point cloud map and the standard point cloud map is complete.

[0219] Step 5: The point cloud generated from the depth map captured by the 3D camera is fused into the global model of the scene. For example, 25 consecutive depth maps are captured, and the point cloud corresponding to each depth map is fused. Specifically, during fusion, the 25 transformed X2 frames are fused (the transformed point cloud corresponding to each point cloud is fused). At this point, points with small spatial distance and normal vector differences are fused (based on the normal vector difference and spatial distance, for each point in the target transformed point cloud, points to be fused are selected from other transformed point cloud maps for fusion). The fusion weights can refer to the distance values ​​from the standard point cloud (the first fusion weight is determined by the distance between each point in the target transformed point cloud and the target matching point, and the second fusion weight is determined by the distance between the point to be fused and the fusion matching point). Generally, the smaller the distance value, the greater the weight.

[0220] Step Six: Point Cloud Overlap Calculation. The terminal calculates the overlap between the fused point cloud image obtained in Step Five and the corresponding points in the standard point cloud image (for each point in the fused point cloud image, multiple matching point pairs are determined, and the overlap is calculated for each of the multiple matching point pairs to obtain the accuracy of the depth map to be judged). Methods for judging overlap may include the following:

[0221] 1) Find the minimum error between point pairs to obtain the degree of overlap, which is the minimum distance between point pairs.

[0222] 2) Find the maximum error between point pairs to obtain the degree of overlap, which is the maximum distance between point pairs.

[0223] 3) Calculate the average error between point pairs to obtain the degree of overlap, which is the average distance between point pairs.

[0224] 4) Obtain the overlap by calculating the difference in the error distribution of point pairs in different regions. For example, take the 30% of point clouds closest to the origin (the N point pairs closest to the origin) and the 30% of point pairs farthest from the origin (the N point pairs farthest from the origin), and calculate the difference in the average distance between these point pairs (the difference between the mean of the N first distances and the mean of the N second distances is determined as the overlap).

[0225] 5) Determine the degree of overlap by considering the error fluctuations of point pairs in the same area. For example, take the 50% of points closest to the origin (M points in the target area of ​​the converted point cloud map), fit a Gaussian distribution of the point pair distances, and calculate the variance of the Gaussian distribution (calculate the variance of the Gaussian distribution for the M third distances) to obtain the degree of overlap.

[0226] In addition to judging by the degree of overlap, the terminal can also use standard shapes to measure the point pair error to obtain the degree of overlap. Standard shapes include, but are not limited to, cubes, cylinders, spheres, etc. By generating a point cloud map based on the depth map captured by the 3D camera, the radius, side length, height and other data are measured (reconstructed shape parameters). These parameters are compared with the data measured from the reference point cloud map (reference shape parameters) to directly obtain the accuracy of the depth map captured by the 3D camera.

[0227] The above method allows for the determination of the stereo accuracy of the depth map by comparing the overlap between the point cloud map generated from the depth map captured by the 3D camera (i.e., the measured point cloud map) and the standard point cloud map obtained by the high-precision scanner (i.e., the actual point cloud map), thus improving the precision of depth map accuracy assessment. It can also be used to determine the accuracy of camera intrinsic and extrinsic parameters.

[0228] The following description continues to illustrate the exemplary structure of the image accuracy judgment device 455 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 4 As shown, the software module stored in the image accuracy determination device 455 in the memory 450 may include:

[0229] Image acquisition module 4551 is used to acquire a depth map to be judged and a reference point cloud map of the shooting object contained in the depth map to be judged; wherein, the reference point cloud map is a point cloud map obtained by scanning with an accuracy not less than an accuracy threshold, and the reference point cloud map records the real three-dimensional coordinate information of each point of the shooting object.

[0230] Image conversion module 4552 is used to convert the depth map to be determined into a point cloud map to obtain a converted point cloud map; wherein, the converted point cloud map records the three-dimensional coordinate information of each point of the shooting object reconstructed based on the depth values ​​recorded in the depth map to be determined;

[0231] The accuracy judgment module 4553 is used to determine the accuracy of the depth map to be judged based on the overlap detection of the converted point cloud map and the reference point cloud map, thereby realizing image accuracy judgment.

[0232] In some embodiments of this application, the accuracy judgment module 4553 is further configured to generate multiple matching point pairs by registering the converted point cloud map with the reference point cloud map; and to obtain the accuracy of the depth map to be judged based on the overlap calculation of the multiple matching point pairs respectively.

[0233] In some embodiments of this application, the accuracy judgment module 4553 is further configured to perform distance calculation on at least two points contained in each of the plurality of matching point pairs to obtain the distance between the plurality of point pairs; calculate the overlap between the converted point cloud map and the reference point cloud map based on the plurality of point pair distances; and convert the overlap into the accuracy of the depth map to be judged.

[0234] In some embodiments of this application, the accuracy judgment module 4553 is further configured to filter out, from the plurality of point-to-point distances, the first distances corresponding to the N points closest to the origin in the transformed point cloud map and the second distances corresponding to the N points farthest from the origin; and determine the difference between the mean of the N first distances and the mean of the N second distances as the overlap between the transformed point cloud map and the reference point cloud map; wherein, N is a positive integer.

[0235] In some embodiments of this application, the accuracy judgment module 4553 is further configured to filter out the third distances corresponding to M points in the target area of ​​the converted point cloud map from the plurality of point-to-point distances; calculate the Gaussian distribution variance of the M third distances to obtain the overlap between the converted point cloud map and the reference point cloud map; wherein M is a positive integer.

[0236] In some embodiments of this application, the image accuracy determination device 455 further includes: a parameter determination module 4554, used to determine the reconstructed shape parameters of the photographed object based on the converted point cloud map, and to determine the reference shape parameters of the photographed object based on the reference point cloud map;

[0237] The accuracy determination module 4553 is further configured to determine the accuracy of the depth map to be determined by the difference between the reconstructed shape parameters and the reference shape parameters.

[0238] In some embodiments of this application, the transformed point cloud map includes: multiple point cloud maps; the accuracy judgment module 4553 is further configured to: for each point in each of the multiple point cloud maps, select the candidate point with the smallest distance from the reference point cloud map; generate a transformed point cloud map corresponding to each point cloud map based on the rigid body transformation between each point in each point cloud map and the corresponding candidate point; select the matching point with the smallest distance to each point in the transformed point cloud map corresponding to each point cloud map from the reference point cloud map, and generate multiple transformed point pairs using each point in the transformed point cloud map and the corresponding matching point; fuse the transformed point cloud maps corresponding to each point cloud map to obtain a fused point cloud map, and determine the multiple matching point pairs from the multiple transformed point pairs for each point in the fused point cloud map.

[0239] In some embodiments of this application, the transformation parameters include: rotation parameters and translation parameters; the accuracy judgment module 4553 is further configured to determine the rotation parameters and the translation parameters by performing a rigid body transformation with the minimum distance on each point of each point cloud map and the corresponding candidate point; and to perform the transformation corresponding to the rotation parameters and the translation parameters on each point cloud map to obtain the transformed point cloud map corresponding to each point cloud map.

[0240] In some embodiments of this application, the accuracy judgment module 4553 is further configured to: filter out a target transformed point cloud map from the transformed point cloud maps corresponding to each point cloud map; calculate the normal difference and spatial distance between each point in the target transformed point cloud map and each point in other transformed point cloud maps for each point in the target transformed point cloud map; wherein, the other transformed point cloud maps refer to the transformed point cloud maps other than the target transformed point cloud map in the transformed point cloud maps corresponding to each point cloud map; based on the normal difference and the spatial distance, filter out points to be fused from the other transformed point cloud maps for each point in the target transformed point cloud map; and perform weighted fusion of each point in the target transformed point cloud map and the corresponding point to be fused to obtain the fused point cloud map.

[0241] In some embodiments of this application, the accuracy judgment module 4553 is further configured to: determine a target matching point from the reference point cloud for each point in the target transformed point cloud; and determine a fusion matching point from the reference point cloud for the point to be fused; determine a first fusion weight for each point in the target transformed point cloud using the distance between each point in the target transformed point cloud and the target matching point; determine a second fusion weight for the point to be fused using the distance between the point to be fused and the fusion matching point; and perform weighted fusion on each point in the target transformed point cloud and the point to be fused based on the first fusion weight and the second fusion weight to obtain the fused point cloud.

[0242] In some embodiments of this application, the depth map to be determined includes: a plurality of consecutive depth maps; the image conversion module 4552 is further configured to downsample the plurality of depth maps respectively to obtain a plurality of downsampled depth maps; to perform image conversion on the plurality of downsampled depth maps and the plurality of depth maps to obtain a plurality of point cloud maps; and to determine the plurality of point cloud maps as the converted point cloud maps.

[0243] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. The processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the image accuracy determination method described in this application.

[0244] This application provides a computer-readable storage medium storing executable instructions. When these executable instructions are executed by a processor, they cause the processor to execute the image accuracy determination method provided in this application. For example, ... Figure 5 The method for judging image accuracy is shown.

[0245] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0246] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0247] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0248] As an example, executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0249] In summary, through the embodiments of this application, the depth map to be judged can be converted into a point cloud map. By calculating the overlap between the converted point cloud map and the reference point cloud map, the difference between the three-dimensional coordinate information of each point of the photographed object reconstructed based on the depth value to be judged and the actual three-dimensional coordinate information of each point can be clearly identified. This improves the judgment effect of the stereo accuracy of the depth map, and ultimately enhances the judgment accuracy of the image. Furthermore, since the conversion of the depth map into a point cloud map requires the aid of camera parameters, calculating the overlap between the converted point cloud map and the reference point cloud map can also help determine the accuracy of the depth camera's camera parameters.

[0250] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A method for judging image accuracy, characterized in that, The image accuracy determination method includes: Obtain a depth map to be judged, and a reference point cloud map of the subject included in the depth map to be judged; wherein, the reference point cloud map is a point cloud map obtained by scanning with an accuracy not less than an accuracy threshold, and the reference point cloud map records the real three-dimensional coordinate information of each point of the subject. The depth map to be determined is converted into a point cloud map to obtain a converted point cloud map; wherein, the converted point cloud map records the three-dimensional coordinate information of each point of the photographed object reconstructed based on the depth values ​​recorded in the depth map to be determined; By registering the transformed point cloud map with the reference point cloud map, multiple matching point pairs are generated; For each pair of matching points, the distance between at least two points is calculated to obtain multiple point pair distances; Based on the distances between the multiple point pairs, the overlap between the transformed point cloud map and the reference point cloud map is calculated; The overlap is converted into the accuracy of the depth map to be judged, thereby realizing the image accuracy judgment.

2. The method according to claim 1, characterized in that, The step of calculating the overlap between the transformed point cloud map and the reference point cloud map based on the distances between the multiple point pairs includes: From the multiple point-to-point distances, the first distances corresponding to the N points closest to the origin in the transformed point cloud map and the second distances corresponding to the N points farthest from the origin are selected respectively. The difference between the mean of N first distances and the mean of N second distances is determined as the overlap between the transformed point cloud map and the reference point cloud map; where N is a positive integer.

3. The method according to claim 1, characterized in that, The step of calculating the overlap between the transformed point cloud map and the reference point cloud map based on the distances between the multiple point pairs includes: From the multiple point-to-point distances, select the third distances corresponding to M points located in the target area of ​​the transformed point cloud map; The Gaussian distribution variance is calculated for the M third distances to obtain the overlap between the transformed point cloud map and the reference point cloud map; where M is a positive integer.

4. The method according to any one of claims 1 to 3, characterized in that, After converting the depth map to a point cloud to obtain the converted point cloud, the method further includes: The reconstructed shape parameters of the photographed object are determined based on the transformed point cloud map, and the reference shape parameters of the photographed object are determined based on the reference point cloud map. The accuracy of the depth map to be judged is determined by the difference between the reconstructed shape parameters and the reference shape parameters.

5. The method according to claim 1, characterized in that, The transformed point cloud map includes: multiple point cloud maps; the step of registering the transformed point cloud map with the reference point cloud map to generate multiple matching point pairs includes: For each point in each of the multiple point cloud maps, the candidate point with the smallest distance is selected from the reference point cloud map; Based on the rigid body transformation between each point and the corresponding candidate point in each point cloud map, a transformed point cloud map corresponding to each point cloud map is generated. From the baseline point cloud map, select the matching point that has the smallest distance to each point in the transformed point cloud map corresponding to each point in the point cloud map, and generate multiple transformed point pairs using each point in the transformed point cloud map and the corresponding matching point; The transformed point cloud maps corresponding to each point cloud map are fused to obtain a fused point cloud map, and for each point in the fused point cloud map, the multiple matching point pairs are determined from the multiple transformed point pairs.

6. The method according to claim 5, characterized in that, The transformation parameters include: rotation parameters and translation parameters; the step of generating a transformed point cloud map corresponding to each point cloud map based on the rigid body transformation between each point in each point cloud map and the corresponding candidate point includes: The rotation parameters and translation parameters are determined by performing a rigid body transformation with the minimum distance on each point in each point cloud and the corresponding candidate point. For each point cloud image, the transformations corresponding to the rotation parameters and translation parameters are performed to obtain the transformed point cloud image corresponding to each point cloud image.

7. The method according to claim 5 or 6, characterized in that, The step of fusing the transformed point cloud maps corresponding to each point cloud map to obtain a fused point cloud map includes: From the transformed point cloud maps corresponding to each point cloud map, the target transformed point cloud map is selected; For each point in the target transformed point cloud map, the normal difference and spatial distance between the point and each point in other transformed point cloud maps are calculated; wherein, the other transformed point cloud maps refer to the transformed point cloud maps in the transformed point cloud maps corresponding to each point cloud map other than the target transformed point cloud map; Based on the normal difference and the spatial distance, for each point in the target transformed point cloud map, points to be fused are selected from the other transformed point cloud maps; The target transformed point cloud map is weighted and fused with the corresponding point to be fused to obtain the fused point cloud map.

8. The method according to claim 7, characterized in that, The step of weightedly fusing each point of the target transformed point cloud map with the corresponding point to be fused to obtain the fused point cloud map includes: For each point in the target transformation point cloud map, a target matching point is determined from the reference point cloud map; and for the point to be fused, a fusion matching point is determined from the reference point cloud map. The first fusion weight of each point in the target transformation point cloud map is determined by using the distance between each point in the target transformation point cloud map and the target matching point. The second fusion weight of the point to be fused is determined by using the distance between the point to be fused and the fusion matching point; Based on the first fusion weight and the second fusion weight, each point of the target transformed point cloud map and the point to be fused are weighted and fused to obtain the fused point cloud map.

9. The method according to any one of claims 1 to 3, 5 or 6, characterized in that, The depth map to be determined includes: multiple consecutive depth maps; the conversion of the depth map to be determined into a point cloud map to obtain a converted point cloud map includes: The multiple depth maps are downsampled to obtain multiple downsampled depth maps; The multiple downsampled depth maps and the multiple depth maps are all converted into multiple point cloud maps; The multiple point cloud maps are determined as the transformed point cloud map.

10. An image accuracy judging device, characterized in that, The image accuracy determination device includes: The image acquisition module is used to acquire a depth map to be judged, and a reference point cloud map of the subject included in the depth map to be judged; wherein, the reference point cloud map is a point cloud map obtained by scanning with an accuracy not less than an accuracy threshold, and the reference point cloud map records the real three-dimensional coordinate information of each point of the subject. The image conversion module is used to convert the depth map to be determined into a point cloud map to obtain a converted point cloud map; wherein, the converted point cloud map records the three-dimensional coordinate information of each point of the shooting object reconstructed based on the depth values ​​recorded in the depth map to be determined; The accuracy judgment module is used to generate multiple matching point pairs by registering the transformed point cloud map with the reference point cloud map; to calculate the distance between at least two points contained in each matching point pair to obtain multiple point pair distances; to calculate the overlap between the transformed point cloud map and the reference point cloud map based on the multiple point pair distances; and to convert the overlap into the accuracy of the depth map to be judged, thereby realizing image accuracy judgment.

11. The apparatus according to claim 10, characterized in that, The accuracy judgment module is further configured to, from the plurality of point-to-point distances, respectively select the first distances corresponding to the N points closest to the origin in the transformed point cloud map, and the second distances corresponding to the N points farthest from the origin; and determine the difference between the mean of the N first distances and the mean of the N second distances as the overlap between the transformed point cloud map and the reference point cloud map; wherein, N is a positive integer.

12. The apparatus according to claim 10, characterized in that, The accuracy judgment module is further configured to filter out the third distances corresponding to M points in the target area of ​​the converted point cloud map from the multiple point-to-point distances; calculate the Gaussian distribution variance of the M third distances to obtain the overlap between the converted point cloud map and the reference point cloud map; where M is a positive integer.

13. An electronic device for judging image accuracy, characterized in that, The electronic device includes: Memory, used to store executable instructions; The processor, when executing executable instructions stored in the memory, implements the image accuracy determination method according to any one of claims 1 to 9.

14. A computer-readable storage medium storing executable instructions, characterized in that, When the executable instructions are executed by the processor, they implement the image accuracy determination method according to any one of claims 1 to 9.

15. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by the processor, they implement the image accuracy determination method according to any one of claims 1 to 9.

Images