Image reconstruction method and device, electronic equipment and storage medium

Abstract

The present disclosure provides an image reconstruction method, device, electronic equipment and storage medium, the image reconstruction method comprising: acquiring a compressed image of a detection object with respect to a projection plane; processing at least one of two-dimensional position information, texture information and depth information included in the compressed image based on a conversion relationship to obtain three-dimensional information, the conversion relationship being determined according to extreme position information of the two-dimensional position information with respect to the projection plane or a projection angle of the two-dimensional position information with respect to the projection plane; and generating point cloud data of the detection object according to a plurality of three-dimensional information.

Description

Technical Field

[0001] This disclosure relates to the fields of image processing, point cloud processing, and industrial technology, and more specifically, to an image reconstruction method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the development of industrial technology, 3D model data is increasingly being used in various industrial fields. However, due to the limitations of graphics card processing power and network bandwidth, 3D model data is difficult to transmit in real time, making it difficult to effectively apply 3D model data in industrial fields.

[0003] In one example, the ability to transmit 3D model data can be improved by increasing investment in hardware. However, this method not only increases the cost of use but also fails to solve the problem of effectively applying 3D model data in industrial applications.

[0004] In another example, one could consider compressing the 3D model data before transmission and then reconstructing the compressed image data upon use to improve the application capabilities of 3D model data in industrial fields. Image compression refers to the process of reducing the size of 3D model data, while image reconstruction refers to the process of recovering the 3D model data from the compressed image data.

[0005] Therefore, how to effectively reconstruct compressed image data is an urgent problem to be solved. Summary of the Invention

[0006] In view of this, the present disclosure provides an image reconstruction method, apparatus, electronic device, and storage medium.

[0007] According to one aspect of this disclosure, an image reconstruction method is provided, comprising: acquiring a compressed image of a detected object relative to a projection plane; processing at least one of two-dimensional position information, texture information, and depth information included in the compressed image based on a transformation relationship to obtain three-dimensional information, wherein the transformation relationship is determined based on the extreme position information of the two-dimensional position information relative to the projection plane or the projection angle of the two-dimensional position information relative to the projection plane; and generating point cloud data of the detected object based on the plurality of three-dimensional information.

[0008] According to another aspect of this disclosure, an image reconstruction apparatus is provided, comprising: a second acquisition module for acquiring a compressed image of a detected object relative to a projection plane; a second processing module for processing at least one of two-dimensional position information, texture information, and depth information included in the compressed image based on a transformation relationship to obtain three-dimensional information, wherein the transformation relationship is determined based on the extreme position information of the two-dimensional position information relative to the projection plane or the projection angle of the two-dimensional position information relative to the projection plane; and a second generation module for generating point cloud data of the detected object based on the plurality of three-dimensional information.

[0009] According to another aspect of this disclosure, an electronic device is provided, comprising: one or more processors; and a memory for storing one or more instructions, wherein, when executed by the one or more processors, the one or more processors cause the one or more processors to perform the method as described in this disclosure.

[0010] According to another aspect of this disclosure, a computer-readable storage medium is provided having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the methods described in this disclosure.

[0011] According to another aspect of this disclosure, a computer program product is provided, which includes computer-executable instructions that, when executed, are used to perform the methods described in this disclosure. Attached Figure Description

[0012] The above and other objects, features and advantages of this disclosure will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0013] Figure 1 This illustration schematically shows a system architecture to which image reconstruction methods can be applied according to embodiments of the present disclosure;

[0014] Figure 2 A flowchart illustrating an image reconstruction method according to an embodiment of the present disclosure is shown schematically.

[0015] Figure 3 This illustration schematically shows an example diagram of an image reconstruction process according to an embodiment of the present disclosure;

[0016] Figure 4 This illustration schematically shows an example diagram of an image reconstruction process according to another embodiment of the present disclosure;

[0017] Figure 5 A block diagram of an image reconstruction apparatus according to an embodiment of the present disclosure is schematically shown; and

[0018] Figure 6A block diagram of an electronic device suitable for implementing an image reconstruction method according to an embodiment of the present disclosure is shown schematically. Detailed Implementation

[0019] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0020] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0021] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0022] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0023] In the technical solution of this invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with relevant laws, regulations, and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse.

[0024] Therefore, this disclosure provides an image reconstruction method, apparatus, electronic device, and storage medium, which can be applied to the fields of image processing, point cloud processing, and industrial technology. The image reconstruction method includes: acquiring a compressed image of a detected object relative to a projection plane; processing at least one of two-dimensional position information, texture information, and depth information included in the compressed image based on a transformation relationship to obtain three-dimensional information, wherein the transformation relationship is determined based on the extreme position information of the point cloud data relative to the projection plane, or the projection angle of the three-dimensional information relative to the projection plane; and generating point cloud data of the detected object based on multiple three-dimensional information sets.

[0025] Figure 1 The illustration schematically depicts a system architecture to which image reconstruction methods can be applied according to embodiments of the present disclosure. It should be noted that... Figure 1 The examples shown are merely examples of system architectures that can be applied to the embodiments of this disclosure, in order to help those skilled in the art understand the technical content of this disclosure, but do not mean that the embodiments of this disclosure cannot be used in other devices, systems, environments or scenarios.

[0026] like Figure 1 As shown, the system architecture 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between different devices.

[0027] It should be noted that the image reconstruction method provided in this embodiment can generally be executed by the server 105. Accordingly, the image reconstruction apparatus provided in this embodiment can generally be located in the server 105.

[0028] Alternatively, the image reconstruction method provided in this embodiment of the present disclosure can also be executed by the first terminal device 101, the second terminal device 102, or the third terminal device 103. Correspondingly, the image reconstruction apparatus provided in this embodiment of the present disclosure can also be disposed in the first terminal device 101, the second terminal device 102, or the third terminal device 103.

[0029] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0030] It should be noted that the sequence numbers of the operations in the following methods are for descriptive purposes only and should not be considered as indicating the execution order of the operations. Unless explicitly stated otherwise, the method does not need to be executed in the exact order shown.

[0031] The system architecture for applying image reconstruction methods provided in this disclosure has been described above. The following will use... Figure 2The image reconstruction process of this disclosure will be further illustrated as an example.

[0032] Figure 2 A flowchart illustrating an image reconstruction method according to an embodiment of the present disclosure is shown schematically.

[0033] like Figure 2 As shown, the image reconstruction method 200 includes operations S210 to S230.

[0034] In operation S210, a compressed image of the detected object relative to the projection plane is acquired.

[0035] In operation S220, based on the transformation relationship, at least one of the two-dimensional position information, texture information and depth information included in the compressed image is processed to obtain three-dimensional information. The transformation relationship is determined according to the limit position information of the two-dimensional position information relative to the projection plane, or the projection angle of the two-dimensional position information relative to the projection plane.

[0036] When operating the S230, point cloud data of the detected object is generated based on multiple 3D information.

[0037] Image reconstruction can refer to the inverse process of image compression. It can also refer to the process of reconstructing point cloud data in three-dimensional space from a compressed image in a two-dimensional plane. The object to be detected refers to the target or object of interest during image compression.

[0038] Compressed images refer to images obtained by compressing 3D information used to represent a detected object in three-dimensional space onto a two-dimensional plane in order to reduce the file size of the 3D information or speed up transmission. Compressed images can be in Portable Network Graphics (PNG) or Bitmap (BMP) formats.

[0039] Two-dimensional position information can characterize the position of each point of the detected object in a two-dimensional plane relative to a reference origin. For example, two-dimensional position information can include X and Y coordinates, where the X coordinate describes the point's position in the horizontal direction and the Y coordinate describes the point's position in the vertical direction. Depth information can characterize the distance between each point and the viewpoint or a reference plane.

[0040] In one example, the transformation relationship can be determined based on the limiting position information of the two-dimensional position information relative to the projection plane. The limiting position information can include the maximum and minimum coordinate values ​​of the two-dimensional position information on the two coordinate axes of the projection plane, and in the positive and negative directions perpendicular to the projection plane. In this case, based on the limiting position information, at least one of the two-dimensional position information, texture information, and depth information can be processed to transform points in the two-dimensional plane into points in three-dimensional space, obtaining three-dimensional information.

[0041] In another example, the transformation relationship can be determined based on the projection angle of the 2D position information relative to the projection plane. The projection angle refers to the angle at which the 2D position information is projected onto the projection plane, i.e., the angle between a point and the projection plane. The projection angle is obtained by projecting the 2D position information onto a 2D plane and calculating the angle between the projected vector and the normal vector of the projection plane. In this case, at least one of the 2D position information, texture information, and depth information can be processed based on the projection angle to transform a point in the 2D plane into a point in 3D space, thus obtaining 3D information.

[0042] After obtaining multiple 3D information points, point cloud data of the detected object can be generated. Point cloud data refers to data composed of a series of 3D information points on the surface of the target or object. Point cloud data characterizes the external shape, structure, and geometric information of the detected object. Point cloud data includes multiple points. 3D information can include 3D position information and texture information. 3D position information can characterize the position of each point in 3D space relative to a reference origin. For example, 3D position information can include X, Y, and Z coordinates.

[0043] The shape of the object to be inspected can be configured according to actual business needs and is not limited here. For example, the shape of the object to be inspected can include at least one of the following: sphere, cube, cylinder, cone, polyhedron, and irregular shape, etc. In one example, the object to be inspected as a sphere can include a workpiece with a spherical or approximately spherical shape, which can be a complete sphere or a partial sphere, such as a spherical surface and a hemisphere, etc.

[0044] According to embodiments of this disclosure, by combining transformation relationships to process at least one of the two-dimensional position information, texture information, and depth information of the detected object in the projection plane, the accuracy of the three-dimensional information can be improved. In this process, since the transformation relationship is determined based on the extreme position information of the point cloud data relative to the projection plane, or the projection angle of the three-dimensional information relative to the projection plane, it can adapt to the needs of different scenarios, improving the flexibility and applicability of the two-dimensional position information transformation. Based on this, by generating the point cloud data of the detected object based on the three-dimensional information, the reconstruction effect of the point cloud data can be improved.

[0045] In this embodiment of the disclosure, the transformation relationship can be determined based on the extreme position information of the two-dimensional position information relative to the projection plane, which will be described below in conjunction with... Figure 3 The process of image reconstruction is further explained based on the transformation relationship determined by the extreme position information of the two-dimensional position information relative to the projection plane.

[0046] Figure 3 The illustration shows an example schematic diagram of an image reconstruction process according to an embodiment of the present disclosure.

[0047] In 300, taking a sphere as the detection object, XOY as the projection plane, and the compressed image including texture and depth images as an example, the image reconstruction process is explained.

[0048] like Figure 3 As shown, the point cloud data can be divided into two hemispheres based on the XOY projection plane, namely hemisphere 301 along the positive Z-axis and hemisphere 302 along the negative Z-axis.

[0049] In the texture image 303 and depth image 304 corresponding to the negative Z-axis direction, to obtain the corresponding point D1 in three-dimensional space for any point D0 in texture image 303 and depth image 304, the two-dimensional position information (ix4, iy4) of point D0 can be mapped to three-dimensional space to obtain the x4 and y4 of the three-dimensional position information of point D1; and the z4 of the three-dimensional position information of D1 can be obtained based on the pixel value of point D0 in depth image 304, thus obtaining the three-dimensional position information (x4, y4, z4) of point D1. The method of determining the three-dimensional position information based on the two-dimensional position information can be configured according to actual business needs and is not limited here. For example, the determination method can include at least one of the following: depth sensor method, multi-view geometry method, and deep learning method, etc.

[0050] Depth sensor methods utilize depth sensors, such as laser scanners, stereo cameras, or time-of-flight cameras, to measure the distance from the surface of the object to the depth sensor, thereby obtaining three-dimensional position information. Multi-view geometry methods involve capturing multiple images of the same object from different angles and using triangulation to analyze the two-dimensional position information of each image to determine its three-dimensional position. Deep learning methods utilize depth information models to process two-dimensional position information, extracting depth cues from captured images to obtain three-dimensional position information.

[0051] For example, in an application scenario where the object to be detected is a sphere, the texture image 303 and the depth image 304 may include a set of two-dimensional position information of the circle, and correspondingly, the point cloud data may include a set of three-dimensional position information of the sphere obtained by using a point cloud acquisition device.

[0052] In one example, the extreme position information may include the maximum and minimum coordinate values ​​of the point cloud data on each of the two coordinate axes of the projection plane, and the maximum and minimum coordinate values ​​of the point cloud data along the positive and negative directions perpendicular to the projection plane. For each of the two coordinate axes of the projection plane in the positive direction and along the positive direction perpendicular to the projection plane, the maximum coordinate value represents the farthest point position of the point cloud data in that positive direction, while the minimum coordinate value represents the nearest point position of the point cloud data in that positive direction. Similarly, for each of the two coordinate axes of the projection plane in the negative direction and along the negative direction perpendicular to the projection plane, the maximum coordinate value represents the nearest point position of the point cloud data in that negative direction, while the minimum coordinate value represents the farthest point position of the point cloud data in that negative direction.

[0053] For example, in applications where the object to be detected is a sphere, the coordinate value corresponding to the projection plane can refer to the coordinate value of the point corresponding to the diameter of the circle or semicircle obtained by projecting the point cloud data onto the projection plane.

[0054] Based on this, a fourth ratio can be determined using the two-dimensional position information and the maximum and minimum coordinate values ​​corresponding to the projection plane. A fifth ratio can be determined based on the depth information and the maximum and minimum coordinate values ​​perpendicular to the projection plane. Finally, three-dimensional position information can be determined based on the target image resolution, the fourth ratio, and the fifth ratio.

[0055] It should be noted that the reconstruction level can be adjusted by changing the target image resolution. For example, when high-precision details need to be preserved, a high reconstruction level can be selected, and the target image resolution can be set to a higher resolution. Alternatively, when real-time display of point cloud data is required, a low reconstruction level can be selected, and the target image resolution can be set to a lower resolution.

[0056] In a specific example, the process of processing two-dimensional position information based on limit position information to obtain three-dimensional position information can be shown in equations (1) and (2) below.

[0057] （1） （2）

[0058] in, and The coordinate values ​​corresponding to the projection plane in the two-dimensional position information. The minimum coordinate value of the point cloud data on the X-axis. The maximum coordinate value of the point cloud data on the X-axis. The minimum coordinate value of the point cloud data on the Y-axis. The maximum coordinate value of the point cloud data on the Y-axis. Characterizes the width information in the resolution of the target image. Characterize the height information in the target image resolution. and It represents two-dimensional position information.

[0059] In another example, the process of processing depth information based on extreme position information to obtain three-dimensional position information can be shown in equations (3) and (4) below.

[0060] （3） （4）

[0061] in, The pixel value that represents the two-dimensional location information. The minimum coordinate value of the point cloud data in the negative direction perpendicular to the projection plane. The maximum coordinate value of the point cloud data in the negative direction perpendicular to the projection plane. Characterizes the three-dimensional position information in the negative direction perpendicular to the projection plane. The minimum coordinate value of the point cloud data in the positive direction perpendicular to the projection plane. The maximum coordinate value of the point cloud data in the positive direction perpendicular to the projection plane. It represents the three-dimensional position information in the positive direction perpendicular to the projection plane.

[0062] In another specific example, the two-dimensional position information can be processed based on the projection angle of the three-dimensional position information relative to the projection plane to obtain the three-dimensional position information. This process can be shown in equations (5) to (9) below.

[0063] （5） （6） （7） （8） （9）

[0064] in, Characterizing the first pitch angle, The first azimuth angle is represented. It should be noted that some formulas provided in this disclosure are given with the detection object being a sphere as an example. However, the shape of the detection object to which the image reconstruction method of this disclosure is applicable can be configured according to actual business needs, and is not limited here.

[0065] According to embodiments of this disclosure, a fourth ratio is determined by utilizing two-dimensional position information and the maximum and minimum coordinate values ​​corresponding to the projection plane, and a fifth ratio is determined based on depth information and the maximum and minimum coordinate values ​​perpendicular to the projection plane, enabling accurate determination of three-dimensional position information at different spatial locations. Furthermore, three-dimensional position information is determined based on the target image resolution, the fourth ratio, and the fifth ratio. Since the target image resolution can be flexibly adjusted according to actual needs, the detected object can be rendered more clearly and accurately at the required resolution, thereby improving the reconstruction effect of point cloud data.

[0066] In this embodiment of the disclosure, the transformation relationship can be determined based on the projection angle of the two-dimensional position information relative to the projection plane, which will be described below in conjunction with... Figure 4 The process of image reconstruction is further explained based on the transformation relationship determined by the projection angle of the two-dimensional position information relative to the projection plane.

[0067] Figure 4 The illustration shows an example schematic diagram of an image reconstruction process according to another embodiment of the present disclosure.

[0068] In 400, taking a sphere as the detection object and XOY as the projection plane as an example, the image reconstruction process is explained.

[0069] like Figure 4 As shown, for the two-dimensional position information (ix5, iy5) of any point E0 in the compressed image 401, the two-dimensional position information can be reconstructed in three dimensions to obtain the three-dimensional position information (x5, y5, z5) of the corresponding point E1.

[0070] Based on this, a sixth ratio can be determined according to the two-dimensional position information and a preset angle range. Three-dimensional position information is then determined based on the target image resolution and the projection angle determined by the sixth ratio. The preset angle range characterizes the viewing angle of the point cloud data acquisition device, i.e., the observation angle or field of view of the acquisition device when acquiring point cloud data. The projection angle may include a second pitch angle. Second azimuth angle .

[0071] In one example, the projection angle may include a second pitch angle. Second azimuth angle Second pitch angle It can refer to the angle between the positive Z-axis direction and the position of the 3D position information in 3D space and the XOY plane, usually varying between -π / 2 and π / 2, the second pitch angle. It can be used to describe the height or depth of a three-dimensional position relative to the XOY plane. Second azimuth angle. It can refer to the angle between the projection of the 3D position information onto the XOY plane from the positive Z-axis direction and the positive Z-axis direction, typically varying between 0 and 2π, the second azimuth angle. It can be used to describe the position of three-dimensional position information on a horizontal plane.

[0072] For example, in an application scenario where the object to be detected is a sphere, the compressed image 401 may include a set of two-dimensional position information of a circle, and correspondingly, the point cloud data may include a set of three-dimensional position information of the sphere obtained by a point cloud acquisition device. In a specific example, the process of processing the two-dimensional position information to obtain the three-dimensional position information according to the projection angle of the three-dimensional position information relative to the projection plane can be shown in the following equations (10) to (14).

[0073] （10） （11） （12） （13） （14）

[0074] in, Characterizing the second pitch angle, Representing the second azimuth angle, and Representing two-dimensional position information, Characterizes the width information in the resolution of the target image. The height information in the target image resolution is represented by a preset angle range corresponding to the second pitch angle. The preset angle range corresponding to the second azimuth angle is .

[0075] According to embodiments of this disclosure, by combining two-dimensional position information, projection angle, and a preset angle range, the conversion of two-dimensional to three-dimensional position information can be performed more accurately, thereby improving the accuracy of the subsequently obtained three-dimensional position information. Furthermore, since the target image resolution can be flexibly adjusted according to actual needs, the detected object can be rendered more clearly and accurately at the required resolution, thus helping to improve the reconstruction effect of point cloud data.

[0076] Figure 5 A block diagram of an image reconstruction apparatus according to an embodiment of the present disclosure is shown schematically.

[0077] like Figure 5 As shown, the image reconstruction apparatus 500 may include a second acquisition module 510, a second processing module 520, and a second generation module 530.

[0078] The second acquisition module 510 is used to acquire a compressed image of the detection object relative to the projection plane.

[0079] The second processing module 520 is used to process at least one of the two-dimensional position information, texture information and depth information included in the compressed image based on the transformation relationship to obtain three-dimensional information. The transformation relationship is determined according to the limit position information of the two-dimensional position information relative to the projection plane or the projection angle of the two-dimensional position information relative to the projection plane.

[0080] The second generation module 530 is used to generate point cloud data of the detected object based on multiple 3D information.

[0081] Any one or more of the modules according to embodiments of this disclosure, or at least part of the functionality of any one or more of them, can be implemented in one module. Any one or more of the modules according to embodiments of this disclosure can be implemented by dividing them into multiple modules. Any one or more of the modules according to embodiments of this disclosure can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, one or more of the modules according to embodiments of this disclosure can be at least partially implemented as computer program modules, which, when run, can perform corresponding functions.

[0082] It should be noted that the image reconstruction apparatus part in the embodiments of this disclosure corresponds to the image reconstruction method part in the embodiments of this disclosure. The specific description of the image reconstruction apparatus part is referred to in the image reconstruction method part, and will not be repeated here.

[0083] Figure 6 A block diagram of an electronic device suitable for implementing an image reconstruction method according to an embodiment of the present disclosure is shown schematically. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0084] like Figure 6 As shown, a computer electronic device 600 according to an embodiment of the present disclosure includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage portion 608 into a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0085] RAM 603 stores various programs and data required for the operation of electronic device 600. Processor 601, ROM 602, and RAM 603 are interconnected via bus 604.

[0086] According to embodiments of this disclosure, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to a bus 604. The electronic device 600 may also include one or more of the following components connected to the input / output (I / O) interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 610 as needed so that computer programs read from it can be installed into the storage section 608 as needed.

[0087] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the image reconstruction method according to the embodiments of this disclosure.

[0088] In this disclosure, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0089] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods provided in the embodiments of this disclosure. When the computer program product is run on an electronic device, the program code enables the electronic device to implement the image reconstruction methods provided in the embodiments of this disclosure.

[0090] When the computer program is executed by the processor 601, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0091] According to embodiments of this disclosure, program code for executing computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages.

[0092] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. It should also be noted that in some alternative implementations, the functions indicated in the boxes may occur in a different order than those shown in the drawings.

[0093] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. An image reconstruction method, comprising: Obtain a compressed image of the detected object relative to the projection plane, the compressed image including a texture image and a depth image; Based on the transformation relationship, at least one of the two-dimensional position information, texture information, and depth information included in the compressed image is processed to obtain three-dimensional information, wherein the three-dimensional information includes three-dimensional position information. The transformation relationship is used to convert the two-dimensional position information in the texture image and the depth information corresponding to the two-dimensional position information in the depth image into the three-dimensional position information. The transformation relationship is determined based on the extreme position information of the two-dimensional position information relative to the projection plane, or the transformation relationship is determined based on a preset angle range, the target image resolution, and the projection angle of the two-dimensional position information relative to the projection plane. The extreme position information includes the maximum and minimum coordinate values ​​of the two-dimensional position information on each of the two coordinate axes of the projection plane and the maximum and minimum coordinate values ​​of the two-dimensional position information along the positive and negative directions perpendicular to the projection plane. The preset angle range characterizes the viewing angle of the point cloud data acquisition device. as well as Based on the three-dimensional information, point cloud data of the detected object is generated.

2. The method according to claim 1, wherein, The three-dimensional information also includes texture information; Based on the transformation relationship, at least one of the two-dimensional position information, texture information, and depth information included in the compressed image is processed to obtain three-dimensional information, including: Based on the extreme position information, the two-dimensional position information or the depth information is processed to obtain the three-dimensional position information; and The texture information of the three-dimensional position information is determined based on the texture information of the two-dimensional position information.

3. The method according to claim 2, wherein, The process of processing the two-dimensional position information or the depth information based on the extreme position information to obtain the three-dimensional position information includes: For each of the two-dimensional location information, The fourth ratio is determined based on the two-dimensional position information, the maximum coordinate value and the minimum coordinate value corresponding to the projection plane; Based on the depth information, the maximum and minimum coordinate values ​​perpendicular to the projection plane, a fifth ratio is determined; and The three-dimensional position information is determined based on the target image resolution, the fourth ratio, and the fifth ratio.

4. The method according to claim 1, wherein, The three-dimensional information includes three-dimensional position information and texture information; Based on the transformation relationship, at least one of the two-dimensional position information, texture information, and depth information included in the compressed image is processed to obtain three-dimensional information, including: Based on the projection angle, the two-dimensional position information or the depth information is processed to obtain the three-dimensional position information; and The texture information of the three-dimensional position information is determined based on the texture information of the two-dimensional position information.

5. The method according to claim 4, wherein, The process of processing the two-dimensional position information or the depth information based on the projection angle to obtain the three-dimensional position information includes: For each of the two-dimensional location information, Based on the two-dimensional position information and the preset angle range, determine the sixth ratio; and The three-dimensional position information is determined based on the target image resolution and the projection angle determined based on the sixth ratio.

6. The method according to claim 1, wherein, The object being detected is a sphere.

7. An image reconstruction apparatus, comprising: The second acquisition module is used to acquire a compressed image of the detection object relative to the projection plane, the compressed image including a texture image and a depth image; The second processing module is used to process at least one of the two-dimensional position information, texture information, and depth information included in the compressed image based on a transformation relationship to obtain three-dimensional information. The three-dimensional information includes three-dimensional position information. The transformation relationship is used to convert the two-dimensional position information in the texture image and the depth information corresponding to the two-dimensional position information in the depth image into the three-dimensional position information. The transformation relationship is determined based on the extreme position information of the two-dimensional position information relative to the projection plane, or the transformation relationship is determined based on a preset angle range, the target image resolution, and the projection angle of the two-dimensional position information relative to the projection plane. The extreme position information includes the maximum and minimum coordinate values ​​of the two-dimensional position information on each of the two coordinate axes of the projection plane and the maximum and minimum coordinate values ​​of the two-dimensional position information along the positive and negative directions perpendicular to the projection plane. The preset angle range characterizes the viewing angle of the point cloud data acquisition device. as well as The second generation module is used to generate point cloud data of the detected object based on the three-dimensional information.

8. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 6.

Images