Image generation method, electronic device and computer readable storage medium

Abstract

Embodiments of the present application relate to the technical field of machine vision, and disclose an image generation method, an electronic device and a computer readable storage medium. The method comprises: selecting a center point of a plurality of emitters on a target VCSEL as a reference point, establishing a first mapping relationship according to the reference point and a preset diffraction point corresponding to the reference point; mapping each of the emitters on the VCSEL into an original speckle in an image coordinate system according to a preset second mapping relationship, the second mapping relationship being a mapping relationship between a VCSEL coordinate system and the image coordinate system; mapping each of the original speckles into a corresponding diffraction speckle according to the first mapping relationship, and generating a GT image according to each of the diffraction speckles. The image generation method provided by the embodiments of the present application can efficiently and quickly generate a scientific and reasonable GT image, thereby providing guidance feedback for the production and improvement of a speckle projector.

Description

Technical Field

[0001] This application relates to the field of machine vision technology, and in particular to an image generation method, an electronic device, and a computer-readable storage medium. Background Technology

[0002] Currently, the most dynamic technology branch in the field of machine vision technology is depth perception technology, and speckle structured light technology is an important component of depth perception technology. A speckle projector is a device that can project speckle structured light patterns onto a target object, and then capture the speckle image with an infrared lens.

[0003] A speckle projector typically consists of a vertical cavity surface emitting laser (VCSEL), a collimating lens group, and diffractive optical elements (DOEs). Several emitters are formed on the surface of the VCSEL through oxidation. The positions of these emitters are determined by the speckle structured light pattern. After the laser emitted by these emitters is collimated by the collimating lens group, the collimated laser enters the DOE and is diffracted by the DOE to form a speckle pattern of m×n diffraction orders. Since the diffraction angles between different diffraction orders are not linearly related, the diffracted speckle pattern will exhibit pincushion distortion.

[0004] The inventors of this application have discovered that different types of DOEs have different diffraction properties, and even different DOEs of the same type have certain differences in diffraction properties. If a systematic improvement and upgrade of speckle projectors is to be carried out, it is crucial to use scientific and reasonable ground truth (GT) images. However, the industry usually only selects a standard DOE as the DOE of the speckle projector and generates GT images through the projection of the speckle projector and the capture of the infrared lens. Such GT images are not scientific, accurate and reasonable enough. Summary of the Invention

[0005] The purpose of this application is to provide an image generation method, electronic device, and computer-readable storage medium that can efficiently and quickly generate scientific and reasonable GT images, providing guidance and feedback for the production and improvement of speckle projectors.

[0006] To address the aforementioned technical problems, embodiments of this application provide an image generation method comprising the following steps: selecting the center points of several transmitters on a VCSEL as reference points; establishing a first mapping relationship based on the reference points and preset diffraction points corresponding to the reference points; mapping each transmitter on the VCSEL to a raw speckle pattern in an image coordinate system according to a preset second mapping relationship, wherein the second mapping relationship is a mapping relationship between the VCSEL coordinate system and the image coordinate system; mapping each raw speckle pattern to a corresponding diffraction speckle pattern according to the first mapping relationship; and generating a GT image based on each diffraction speckle pattern.

[0007] Embodiments of this application also provide an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the image generation method described above.

[0008] Embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the image generation method described above.

[0009] The image generation method, electronic device, and computer-readable storage medium provided in the embodiments of this application first select the center points of several transmitters on a VCSEL as reference points. Based on the reference points and preset diffraction points corresponding to the reference points, a first mapping relationship is established. Then, based on a preset second mapping relationship, i.e., the mapping relationship between the VCSEL coordinate system and the image coordinate system, all transmitters on the VCSEL are mapped to the original speckle patterns in the image coordinate system. Finally, based on the established first mapping relationship, each original speckle pattern is mapped to the corresponding diffraction speckle pattern, and a ground truth (GT) image is generated based on each diffraction speckle. Considering that any Design of Objects (DOE) may have certain defects due to manufacturing issues, obtaining GT images based on a selected standard DOE is not scientific, accurate, or reasonable. The embodiments of this application use simulated diffraction, without using a real DOE for diffraction. Instead, a diffraction relationship is established based on the stable center points of the transmitters, and combined with the coordinate system mapping relationship, the speckle pattern emitted by the VCSEL is mapped to the original speckle patterns in the image coordinate system, and then simulated diffraction is used to generate diffraction speckles. This efficiently and quickly generates scientific and reasonable GT images, providing guidance and feedback for the production and improvement of speckle projectors.

[0010] Furthermore, the step of mapping each original speckle to a corresponding diffraction speckle according to the first mapping relationship, and generating a GT image based on each diffraction speckle, includes: traversing each original speckle, sequentially taking each original speckle in the current original speckle as a diffraction point, and determining the diffracted coordinates of the diffraction point according to the first mapping relationship and the coordinates of the diffraction point; obtaining the rounded coordinates according to a preset rounding rule and the diffracted coordinates of the diffraction point; allocating the brightness value of the diffraction point to a point on the rounded coordinates and points adjacent to the point on the rounded coordinates according to the diffracted coordinates and the rounded coordinates; wherein, the point on the rounded coordinates and the points adjacent to the point on the rounded coordinates are all diffraction speckles; after the traversal of each original speckle is completed, The brightness values ​​assigned to the diffraction speckles are accumulated to determine the final brightness value of the diffraction speckles. A ground truth (GT) image is generated based on each diffraction speckle and its final brightness value. Considering that the speckle pattern will undergo a certain pincushion distortion during actual diffraction by DOE, the original circular speckle will be stretched into a near-elliptical shape due to the pincushion distortion. Therefore, in the process of generating the GT image by simulating diffraction in this application, this pincushion distortion must also be simulated. This application simulates this distortion from two aspects: coordinates and brightness values. Not only are the coordinates of the points to be diffracted diffracted according to the first mapping relationship, but the brightness values ​​of the points to be diffracted are also redistributed to the corresponding regions, so that the generated GT image is more scientific and reasonable, and thus provides better guidance and feedback for the production and improvement of speckle projectors.

[0011] Furthermore, the brightness value of the diffracted point is obtained through the following steps: determining the center point of the original speckle where the diffracted point is located, and obtaining the coordinates of the diffracted point, the coordinates of the center point, and the radius of the original speckle; calculating the distance between the diffracted point and the center point based on the coordinates of the diffracted point and the center point; determining the brightness value of the diffracted point based on a preset adjustment coefficient, the distance between the diffracted point and the center point, and the radius of the original speckle; wherein, the greater the distance, the smaller the brightness value of the diffracted point, and the distance includes the difference in the horizontal and vertical coordinates between the diffracted point and the center point. Considering that the brightness of the original speckle is proportional to its energy, that is, the brightness of the original speckle also satisfies a Gaussian distribution, with the center point having the highest brightness and the distance from the center point decreasing in brightness, this application, based on the distance between the diffracted point and the center point of the original speckle where the diffracted point is located, plus the radius of the original speckle as a constraint and a preset adjustment coefficient as an adjustment, can scientifically and reasonably determine the brightness value of each original speckle in the original speckle.

[0012] Furthermore, the second mapping relationship is expressed by the following formula: u=ax ×x+b x v=a y ×y+b y Where (x,y) are the coordinates of a point in the VCSEL coordinate system, (u,v) are the coordinates of a point in the image coordinate system, and a x b x a y and b y The second mapping coefficient is a preset value, which is determined according to a preset resolution, namely the resolution of the GT image. The mapping coefficient in the second mapping relationship of this application is determined based on the resolution of the GT image to be generated, thereby making the image generation method of this application applicable to various scenarios that require GT images, and enhancing the universality of the GT image generation process.

[0013] Furthermore, the diffraction points corresponding to the reference point include diffraction points of each diffraction order corresponding to the reference point. Establishing a first mapping relationship based on the reference point and the preset diffraction points corresponding to the reference point includes: using the reference point as a first target point, and randomly selecting a diffraction point from the diffraction points of each diffraction order corresponding to the reference point as a second target point; establishing an equation corresponding to the reference point based on the coordinates of the first target point and the second target point; wherein the equation contains several unknown first mapping coefficients; simultaneously solving the equations corresponding to each reference point to obtain an overdetermined system of equations, and performing least-squares solution on the overdetermined system of equations to obtain several solved first mapping parameters; wherein the number of equations in the overdetermined system of equations is greater than the number of first mapping coefficients; determining the first mapping relationship based on the several solved first mapping parameters. This application determines the first mapping coefficients in the first mapping relationship by establishing an overdetermined system of equations and solving the least-squares solution, which can obtain a more scientific, accurate, and reasonable first mapping relationship. Such a first mapping relationship is more realistic and more stable. Attached Figure Description

[0014] One or more embodiments are illustrated by way of example with reference to the accompanying drawings, and these illustrative descriptions do not constitute a limitation on the embodiments.

[0015] Figure 1 This is a flowchart of an image generation method provided in one embodiment of this application; Figure 2 This is a schematic diagram of a primitive speckle pattern provided in one embodiment of this application; Figure 3 This is a schematic diagram of a diffraction speckle pattern provided in one embodiment of this application; Figure 4This is a flowchart of an embodiment of the present application, in which each original speckle is mapped to a corresponding diffraction speckle according to a first mapping relationship, and a GT image is generated according to each diffraction speckle. Figure 5 This is a GT image provided in one embodiment of this application; Figure 6 This is a flowchart of obtaining the brightness value of the diffraction point in one embodiment of this application; Figure 7 This is a flowchart illustrating, in one embodiment of the present application, the establishment of a first mapping relationship based on a reference point and a preset diffraction point corresponding to the reference point; Figure 8 This is a schematic diagram of the structure of an electronic device provided in another embodiment of this application. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the various embodiments of this application will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been provided in the various embodiments of this application to help readers better understand this application. However, the technical solutions claimed in this application can be implemented even without these technical details and various changes and modifications based on the following embodiments. The division of the various embodiments below is for the convenience of description and should not constitute any limitation on the specific implementation of this application. The various embodiments can be combined with and referenced by each other without contradiction.

[0017] One embodiment of this application relates to an image generation method applied to an electronic device, wherein the electronic device can be a terminal or a server. In this embodiment and the following embodiments, the electronic device is described using a server as an example. The implementation details of the image generation method of this embodiment are described in detail below. The following implementation details are provided only for ease of understanding and are not necessary for implementing this solution.

[0018] The specific process of the image generation method in this embodiment can be described as follows: Figure 1 As shown, it includes: Step 101: Select the center points of several transmitters on the VCSEL as reference points, and establish the first mapping relationship based on the reference points and the preset diffraction points corresponding to the reference points.

[0019] Specifically, hundreds or even thousands of emitters are formed on the surface of a VCSEL through oxidation. The server first randomly selects several emitters from a number of emitters on the VCSEL, finds the center point of these emitters, and uses the center point of these emitters as the reference point. The emitters on the VCSEL are generally circular, so the center point of the emitter is the center of the circle of the emitter. After the server determines the reference point, it can establish a first mapping relationship based on the reference point and the preset diffraction points corresponding to the reference point. The preset diffraction points corresponding to the reference point are predetermined before the production of this model and this batch of speckle projectors.

[0020] In the specific implementation, the server first establishes a VCSEL coordinate system. The center of the VCSEL can be used as the origin of the VCSEL coordinate system, or the lower left corner of the VCSEL can be used as the origin of the VCSEL coordinate system. While selecting the center points of several transmitters as reference points, the server can obtain the coordinates of these reference points. The preset diffraction points corresponding to the reference points also include the coordinates of these diffraction points. The server establishes the first mapping relationship based on the coordinates of the reference points and the coordinates of the diffraction points corresponding to the reference points.

[0021] In one example, the pre-defined diffraction points corresponding to the reference point include the diffraction points of each diffraction order corresponding to the reference point. For example, if there are 7×11 diffraction orders, then there are 77 diffraction points of each diffraction order corresponding to the reference point.

[0022] In one example, the first mapping relationship is the perspective transformation matrix. The server can traverse the selected reference points and solve for the perspective transformation matrix based on the coordinates of each reference point and the coordinates of the diffraction points corresponding to each reference point.

[0023] In one example, the first mapping relationship can be implemented by the first mapping model. The server uses the coordinates of the reference point as training samples and the coordinates of the diffraction points corresponding to the reference point as labels to iteratively train the first mapping model until the first mapping model meets the preset convergence condition, thus obtaining the trained first mapping model. The server uses the trained first mapping model as the first mapping relationship.

[0024] Step 102: According to the preset second mapping relationship, each emitter on the VCSEL is mapped to the original speckle in the image coordinate system. The second mapping relationship is the mapping relationship between the VCSEL coordinate system and the image coordinate system.

[0025] Specifically, after the server establishes the VCSEL coordinate system and determines the coordinates of each point in each transmitter on the VCSEL, it can map the VCSEL coordinate system to the image coordinate system according to the preset second mapping relationship. That is, all transmitters on the VCSEL are mapped to the original speckle in the image coordinate system. The preset second mapping relationship can be set by those skilled in the art according to actual needs.

[0026] In the specific implementation, the server traverses each emitter on the VCSEL and maps the coordinates of each point in the current emitter in the VCSEL coordinate system to the coordinates in the image coordinate system according to the preset second mapping relationship. That is, the current emitter is mapped to the corresponding original speckle. Each point in the emitter is the original speckle in the corresponding original speckle. Since the emitters on the VCSEL are generally circular, the original speckle corresponding to the emitter is also circular. Furthermore, the size of each emitter on the VCSEL is the same, so the size of each mapped original speckle is also the same, that is, the radius of each original speckle is equal.

[0027] In one example, the original speckle pattern mapped according to the second mapping relation can be as follows: Figure 2 As shown.

[0028] In one example, the server can choose to map the global raw speckle image according to the second mapping relationship, that is, to map out all the raw speckles.

[0029] In another example, the server can choose to map a local original speckle image according to a second mapping relationship. The local original speckle image includes the zero-order diffraction region and the region corresponding to the diffraction orders around the zero-order diffraction region. In other words, only a part of the original speckle needs to be mapped.

[0030] In one example, the second mapping relationship is represented by the following formula: u=a x ×x+b x v=a y ×y+b y In the formula, (x,y) are the coordinates of a point in the VCSEL coordinate system, (u,v) are the coordinates of a point in the image coordinate system, and a x b x a y and b y The second mapping coefficients are preset and are determined based on a preset resolution, which is the resolution of the GT image to be generated. The resolution of the GT image to be generated can be an 8-bit image, a 16-bit image, etc. Since the mapping coefficients in the second mapping relationship are determined based on the resolution of the GT image to be generated, the image generation method of this embodiment is applicable to many types of scenarios that require GT images, thus enhancing the universality of the GT image generation process.

[0031] In one example, the server can execute step 101 first and then step 102, or it can execute step 102 first and then step 101, or it can execute steps 101 and 102 simultaneously.

[0032] Step 103: According to the first mapping relationship, each original speckle is mapped to the corresponding diffraction speckle, and a GT image is generated based on each diffraction speckle.

[0033] Specifically, after the server maps each transmitter on the VCSEL to the original speckle in the image coordinate system, it can then map each original speckle to the corresponding diffraction speckle according to the first mapping relationship, that is, perform simulated diffraction, and the server generates a GT image based on each diffraction speckle.

[0034] In one example, the diffraction speckle pattern mapped according to the first mapping relation can be as follows: Figure 3 As shown.

[0035] In one example, the server iterates through each original speckle and maps the coordinates of the original speckle in the current original speckle to the coordinates after simulated diffraction according to the first mapping relationship. The point on the coordinates after simulated diffraction is the diffraction speckle corresponding to the original speckle. After all the original speckles have been traversed, the GT image can be obtained.

[0036] In this embodiment, compared to the technical solution of selecting a standard DOE as the speckle projector and generating a GT image through the projection of the speckle projector and the capture of the infrared lens, the server first selects the center points of several transmitters on the VCSEL as reference points. Based on the reference points and preset diffraction points corresponding to the reference points, a first mapping relationship is established. Then, based on a preset second mapping relationship—that is, the mapping relationship between the VCSEL coordinate system and the image coordinate system—all transmitters on the VCSEL are mapped to the original speckle patterns in the image coordinate system. Finally, based on the established first mapping relationship, each original speckle pattern is mapped to its corresponding diffraction pattern. The speckle pattern is generated, and a ground truth (GT) image is produced based on each speckle. Considering that any design object (DOE) may have certain defects due to manufacturing issues, it is not scientific, accurate, or reasonable to obtain a GT image based on a selected standard DOE. The embodiments of this application adopt a simulated diffraction method. Instead of using a real DOE for diffraction, a diffraction relationship is established based on the center point of a stable emitter. Combined with the coordinate system mapping relationship, the speckle pattern emitted by the VCSEL is mapped to the original speckle in the image coordinate system. Then, simulated diffraction is used to generate diffraction speckle, thereby efficiently and quickly generating a scientific and reasonable GT image, providing guidance and feedback for the production and improvement of speckle projectors.

[0037] In one embodiment, the server maps each original speckle pattern to a corresponding diffraction speckle pattern according to a first mapping relationship, and generates a ground truth (GT) image based on each diffraction speckle pattern. This can be achieved through methods such as... Figure 4 The steps shown are implemented as follows: Step 201: Traverse each original speckle pattern, and take each original speckle in the current original speckle pattern as the diffraction point in turn. Then, determine the coordinates of the diffraction point after diffraction based on the first mapping relationship and the coordinates of the diffraction point.

[0038] Specifically, after determining each original speckle, the server can traverse each original speckle and take each original speckle in the current original speckle as the diffraction point in turn, and determine the coordinates of the diffraction point after diffraction based on the first mapping relationship and the coordinates of the diffraction point.

[0039] In one example, let the coordinates of the point to be diffracted be (u, v), and the first mapping relationship be f. x and f y Then the coordinates of the point to be diffracted can be denoted as (R, C), where R = f x (u,v), C=f y (u,v).

[0040] Step 202: Obtain the rounded coordinates according to the preset rounding rules and the coordinates of the diffracted point.

[0041] In the specific implementation, the coordinates of the diffracted point determined according to the first mapping relationship and the coordinates of the point to be diffracted can not be integers. The GT image to be generated is based on 1 pixel as the smallest unit. That is to say, the coordinates of the diffracted point determined are sub-pixel coordinates, which are sub-pixel points. Therefore, the server needs to determine the coordinates of the diffracted point, which is to obtain integer pixel coordinates. The preset rounding rule can be set by those skilled in the art according to actual needs.

[0042] In one example, the preset rounding rules can be rounding up, rounding down, or rounding to the nearest integer.

[0043] In one example, the coordinates of the diffracted point are denoted as (R,C), and the rounded coordinates can be denoted as (r,c).

[0044] In one example, the coordinates of the diffracted point are (5.3, 2.2). The default rounding rule is to round to the nearest integer, so the rounded coordinates are (5, 2).

[0045] Step 203: Based on the diffracted coordinates and the rounded coordinates of the point to be diffracted, assign the brightness value of the point to be diffracted to the point on the rounded coordinates and the point adjacent to the point on the rounded coordinates.

[0046] Specifically, after determining the rounded coordinates, the server can assign the brightness value of the point to be diffracted to the point on the rounded coordinates and the points adjacent to the point on the rounded coordinates, based on the diffracted coordinates and the rounded coordinates. The point on the rounded coordinates and the points adjacent to the point on the rounded coordinates are actually diffraction speckles.

[0047] In practical implementation, due to the diffraction characteristics of DOE, after diffraction, not only will the coordinates of the diffracted point be stretched, but the brightness value of the diffracted point will also be "stretched". Therefore, it is necessary to assign the brightness value of the diffracted point to each diffracted speckle in a certain area in order to better simulate the actual diffraction process of DOE and ensure that the brightness value of the original speckle is not lost due to the first mapping relationship.

[0048] In one example, the server can assign the brightness value of the diffraction point to nine diffraction spots within a 3px × 3px window centered on the point at the rounded coordinates, according to a preset brightness allocation formula.

[0049] In one example, the server can use the following formula to assign the brightness value of the diffracted point to the point at the rounded coordinates and the points adjacent to the point at the rounded coordinates:

[0050] In the formula, (R,C) are the coordinates of the point to be diffracted after diffraction, (r,c) are the coordinates after rounding, P is the brightness value of the point to be diffracted, P(r,c) is the brightness value assigned to the point on the rounded coordinates, P(r+1,c) is the brightness value assigned to the point adjacent to the right of the point on the rounded coordinates, P(r,c+1) is the brightness value assigned to the point adjacent above the point on the rounded coordinates, and P(r+1,c+1) is the brightness value assigned to the point adjacent to the upper right of the point on the rounded coordinates.

[0051] Step 204: After the original speckle traversal is completed, the brightness values ​​assigned to the diffraction speckles are accumulated to determine the final brightness value of the diffraction speckles, and a GT image is generated based on each diffraction speckle and its final brightness value.

[0052] In practice, each diffraction speckle may actually be assigned partial brightness values ​​of multiple diffraction points. Therefore, after the server completes the traversal of each original speckle, it needs to accumulate the brightness values ​​assigned to the diffraction speckles to determine the final brightness value of the diffraction speckles, and generate a GT image based on each diffraction speckle and its final brightness value.

[0053] In one example, the GT image generated by the server can be like... Figure 5 As shown.

[0054] In this embodiment, considering that the speckle pattern will undergo a certain pincushion distortion during the actual diffraction process performed by DOE, the original circular speckle will be stretched into a near-elliptical shape due to the pincushion distortion. Therefore, in the process of generating GT images by simulating diffraction in this application, this pincushion distortion must also be simulated. This application simulates this distortion from two aspects: coordinates and brightness values. Not only will the coordinates of the points to be diffracted be diffracted according to the first mapping relationship, but the brightness values ​​of the points to be diffracted will also be redistributed to the corresponding regions, so that the generated GT image is more scientific and reasonable, and thus provides better guidance and feedback for the production and improvement of speckle projectors.

[0055] In one embodiment, the brightness value of the diffraction point can be determined by the server via, for example... Figure 6 The steps shown are as follows: Step 301: Determine the center point of the original speckle pattern where the diffraction point is located, and obtain the coordinates of the diffraction point, the coordinates of the center point, and the radius of the original speckle pattern.

[0056] Specifically, the server can determine the center point of the original speckle where the diffraction point is located, and obtain the coordinates of the diffraction point, the coordinates of the center point of the original speckle where the diffraction point is located, and the radius of the original speckle where the diffraction point is located.

[0057] Step 302: Calculate the distance between the diffraction point and the center point based on the coordinates of the diffraction point and the center point.

[0058] Specifically, after the server obtains the coordinates of the point to be diffracted, the coordinates of the center point of the original speckle where the point to be diffracted is located, and the radius of the original speckle where the point to be diffracted is located, it can calculate the distance between the point to be diffracted and the center point of the original speckle where the point to be diffracted is located, based on the coordinates of the point to be diffracted and the coordinates of the center point of the original speckle where the point to be diffracted is located. The distance between the point to be diffracted and the center point of the original speckle where the point to be diffracted is located includes the difference in the horizontal coordinate and the difference in the vertical coordinate between the point to be diffracted and the center point of the original speckle where the point to be diffracted is located.

[0059] Step 303: Determine the brightness value of the diffraction point based on the preset adjustment coefficient, the distance between the diffraction point and the center point, and the radius of the original speckle.

[0060] Specifically, after the server calculates the distance between the diffraction point and the center point of the original speckle where the diffraction point is located, it can determine the brightness value of the diffraction point based on the preset adjustment coefficient, the distance between the diffraction point and the center point of the original speckle where the diffraction point is located, and the radius of the original speckle where the diffraction point is located.

[0061] In one example, the server can determine the brightness value of the diffraction point using the following formula, based on a preset adjustment coefficient, the distance between the diffraction point and the center point of the original speckle containing the diffraction point, and the radius of the original speckle containing the diffraction point:

[0062] In the formula, P0 is the preset adjustment coefficient, s is the abscissa of the center point of the original speckle where the diffraction point is located, t is the ordinate of the center point of the original speckle where the diffraction point is located, u is the abscissa of the diffraction point, v is the ordinate of the diffraction point, w is the radius of the original speckle where the diffraction point is located, (su) is the difference in abscissa between the center point of the diffraction point and the center point of the original speckle where the diffraction point is located, and (tv) is the difference in ordinate between the center point of the diffraction point and the center point of the original speckle where the diffraction point is located.

[0063] In this embodiment, considering that the brightness of the original speckle is proportional to its energy, that is, the brightness of the original speckle also follows a Gaussian distribution, with the highest brightness at the center point and the lower brightness as it moves further away from the center point, this application uses the distance between the diffraction point and the center point of the original speckle where the diffraction point is located, plus the radius of the original speckle as a constraint and a preset adjustment coefficient as an adjustment, to scientifically and reasonably determine the brightness value of each original speckle in the original speckle.

[0064] In one embodiment, the diffraction points corresponding to the reference point include diffraction points of each diffraction order corresponding to the reference point. The server establishes a first mapping relationship based on the reference point and preset diffraction points corresponding to the reference point, which can be achieved through methods such as... Figure 7 The steps shown are implemented as follows: Step 401: Take the reference point as the first target point, and randomly select a diffraction point from the diffraction points of each diffraction order corresponding to the reference point as the second target point.

[0065] Step 402: Based on the coordinates of the first target point and the second target point, establish the equation corresponding to the reference point. The equation contains several unknown first mapping coefficients.

[0066] In the specific implementation, after the server selects the center points of several transmitters on the VCSEL as reference points, it can use the reference points as the first target point and randomly select a diffraction point from the diffraction points of each diffraction order corresponding to the reference point as the second target point. If the preset diffraction order is 7×11 diffraction orders, then there are 77 diffraction points corresponding to one reference point. However, in order to enhance the stability and scientific nature of the determined first mapping relationship, the first mapping relationship needs to be determined based on several reference points. Therefore, for each reference point, only one diffraction point corresponding to the reference point needs to be selected. The server then establishes the equation corresponding to the reference point based on the coordinates of the first target point and the coordinates of the second target point. The established equation contains several unknown first mapping coefficients.

[0067] In one example, the diffraction orders are m×n, where m and n are both odd numbers greater than 1. The first mapping relationship is a bivariate D-order function. The server establishes the equation corresponding to the reference point based on the coordinates of the first and second target points, which can be expressed by the following formula:

[0068]

[0069]

[0070] In the formula, (x i ,y i (x) represents the coordinates of the first target point. i g,h ,y i g,h ) represents the coordinates of the second target point, q dx,dy This is the first mapping coefficient.

[0071] In one example, the first mapping relationship is a bivariate fourth-order function, so the first mapping coefficients total 30.

[0072] Step 403: Combine the equations corresponding to each reference point to obtain an overdetermined system of equations. Solve the overdetermined system of equations by least squares to obtain several first mapping parameters.

[0073] Step 404: Determine the first mapping relationship based on several solved first mapping parameters.

[0074] In the specific implementation, after the server establishes the equations corresponding to each reference point, it can combine the equations corresponding to each reference point to obtain an overdetermined system of equations. The server then performs least squares solution on the overdetermined system of equations to obtain several first mapping parameters. Finally, based on the several first mapping parameters, the server determines the first mapping relationship. The number of equations in the overdetermined system of equations is greater than the number of first mapping coefficients.

[0075] In one example, there are a total of 30 first mapping coefficients. These 30 first mapping coefficients actually include f. x The 15 first mapping coefficients on the side and f y The server can solve for these 30 first mapping coefficients by selecting 15 reference points, and by selecting more than 16 reference points, it can establish an overdetermined system of equations and perform least squares solution, thus solving for these 30 first mapping coefficients more scientifically, reasonably and accurately.

[0076] In this embodiment, by establishing an overdetermined system of equations and finding the least squares solution, the first mapping coefficients in the first mapping relationship can be determined, which can obtain a more scientific, accurate, and reasonable first mapping relationship. Such a first mapping relationship is more realistic and more stable.

[0077] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this patent.

[0078] Another embodiment of this application relates to an electronic device, such as... Figure 8 As shown, it includes: at least one processor 501; and a memory 502 communicatively connected to the at least one processor 501; wherein the memory 502 stores instructions executable by the at least one processor 501, the instructions being executed by the at least one processor 501 to enable the at least one processor 501 to perform the image generation methods in the above embodiments.

[0079] The memory and processor are connected via a bus, which can include any number of interconnecting buses and bridges, connecting various circuits of one or more processors and memories. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and will not be described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over the wireless medium via an antenna, which further receives data and transmits it to the processor.

[0080] The processor manages the bus and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory is used to store data used by the processor during operation.

[0081] Another embodiment of this application relates to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the method embodiments described above.

[0082] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0083] Those skilled in the art will understand that the above embodiments are specific embodiments for implementing this application, and in practical applications, various changes can be made to them in form and detail without departing from the spirit and scope of this application.

Claims

1. An image generation method, characterized in that, include: The center points of several emitters on a vertical cavity surface-emitting laser (VCSEL) are selected as reference points. A first mapping relationship is established based on the reference points and the preset diffraction points corresponding to the reference points. According to the preset second mapping relationship, each of the transmitters on the VCSEL is mapped to the original speckle in the image coordinate system. The second mapping relationship is the mapping relationship between the VCSEL coordinate system and the image coordinate system. Based on the first mapping relationship, each of the original speckles is mapped to the corresponding diffraction speckles, and a real GT image is generated based on each of the diffraction speckles. The step of establishing a first mapping relationship based on the reference point and a preset diffraction point corresponding to the reference point includes: The equations containing unknown first mapping coefficients for each reference point are established based on the coordinates of the reference point and the coordinates of the diffraction points of each diffraction order corresponding to the reference point. By combining the equations corresponding to each of the aforementioned reference points, an overdetermined system of equations is obtained. The overdetermined system of equations is then solved by least squares to obtain several first mapping parameters. The step of mapping each of the original speckles to a corresponding diffraction speckle according to the first mapping relationship, and generating a GT image based on each of the diffraction speckles, includes: Traverse each of the original speckle patterns, and sequentially take each original speckle pattern in the current original speckle pattern as the diffraction point, and determine the coordinates of the diffraction point after diffraction according to the first mapping relationship and the coordinates of the diffraction point. Based on the preset rounding rules and the coordinates of the point to be diffracted after diffraction, the rounded coordinates are obtained; Based on the diffracted coordinates of the point to be diffracted and the rounded coordinates, the brightness value of the point to be diffracted is assigned to the point on the rounded coordinates and the points adjacent to the point on the rounded coordinates; wherein, the point on the rounded coordinates and the points adjacent to the point on the rounded coordinates are all diffraction speckles. After each of the original speckle patterns is traversed, the brightness values ​​assigned to the diffraction speckles are accumulated to determine the final brightness value of the diffraction speckles, and a GT image is generated based on each of the diffraction speckles and the final brightness value of each of the diffraction speckles. The brightness value of the diffraction point is determined using the following formula, based on a preset adjustment coefficient, the distance between the diffraction point and the center point, and the radius of the original speckle: Wherein, P0 is the preset adjustment coefficient, s is the abscissa of the center point, t is the ordinate of the center point, u is the abscissa of the point to be diffracted, v is the ordinate of the point to be diffracted, w is the radius of the original speckle, (su) is the difference in abscissa between the point to be diffracted and the center point, and (tv) is the difference in ordinate between the point to be diffracted and the center point. Wherein, the diffraction points corresponding to the reference point include diffraction points of each diffraction order corresponding to the reference point, and the step of establishing a first mapping relationship based on the reference point and the preset diffraction points corresponding to the reference point includes: The reference point is taken as the first target point, and a diffraction point is randomly selected from the diffraction points of each diffraction order corresponding to the reference point as the second target point. Based on the coordinates of the first target point and the second target point, an equation corresponding to the reference point is established; wherein, the equation contains several unknown first mapping coefficients; By combining the equations corresponding to each of the aforementioned reference points, an overdetermined system of equations is obtained. The overdetermined system of equations is then solved by least squares to obtain several first mapping parameters. The number of equations in the overdetermined system of equations is greater than the number of first mapping coefficients. The first mapping relationship is determined based on the several solved first mapping parameters; Wherein, the diffraction order is m×n, where m and n are both odd numbers greater than 1, the first mapping relationship is a bivariate D-order function, and the equation corresponding to the reference point is established based on the coordinates of the first target point and the coordinates of the second target point, expressed by the following formula: Among them, (x i ,y i (X) represents the coordinates of the first target point. i g,h ,Y i g,h ) represents the coordinates of the second target point, q dx,dy is the first mapping coefficient.

2. The image generation method according to claim 1, characterized in that, The brightness value of the diffracted point is assigned to the point at the rounded coordinates and the points adjacent to the point at the rounded coordinates using the following formula: Where (R,C) are the coordinates of the point to be diffracted after diffraction, (r,c) are the rounded coordinates, P is the brightness value of the point to be diffracted, P(r,c) is the brightness value assigned to the point at the rounded coordinates, P(r+1,c) is the brightness value assigned to the point to the right of the point at the rounded coordinates, P(r,c+1) is the brightness value assigned to the point above the point at the rounded coordinates, and P(r+1,c+1) is the brightness value assigned to the point to the upper right of the point at the rounded coordinates.

3. The image generation method according to claim 1, characterized in that, The brightness value of the diffraction point is obtained through the following steps: Determine the center point of the original speckle pattern where the diffraction point is located, and obtain the coordinates of the diffraction point, the coordinates of the center point, and the radius of the original speckle pattern; Calculate the distance between the diffraction point and the center point based on the coordinates of the point to be diffracted and the center point. The brightness value of the diffracted point is determined based on a preset adjustment coefficient, the distance between the diffracted point and the center point, and the radius of the original speckle. The greater the distance, the smaller the brightness value of the diffracted point. The distance includes the difference in the horizontal coordinate and the difference in the vertical coordinate between the diffracted point and the center point.

4. The image generation method according to claim 1, characterized in that, The second mapping relationship is represented by the following formula: u=a x ×x+b x ,v=a y ×y+b y Where (x,y) are the coordinates of a point in the VCSEL coordinate system, (u,v) are the coordinates of a point in the image coordinate system, and a x b x a y and b y The second mapping coefficient is a preset value, which is determined according to a preset resolution, namely the resolution of the GT image.

5. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the image generation method as described in any one of claims 1 to 4.

6. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the image generation method according to any one of claims 1 to 4.

Images