Structured light system modeling and calibration method and device based on laser galvanometer

Abstract

The embodiment of the invention discloses a modeling and calibration method and device for a structured light system based on a laser galvanometer. A specific embodiment of the method comprises the following steps: acquiring a first checkerboard image set; according to the first checkerboard image set, determining first initial internal reference information, second initial internal reference information corresponding to the projection device, and initial rotation information and initial translation information between the shooting device and the projection device; obtaining a second checkerboard image set; acquiring an energy generation function; and according to the second checkerboard image set, determining an optimal solution under the condition that a function value corresponding to the energy generation function is minimized so as to generate the first internal reference information, the second internal reference information, the initial rotation information, the translation information and a linear mapping matrix which are used as calibration parameter information of the structured light system based on the laser galvanometer. According to the embodiment, based on the laser galvanometer, high-precision calibration can be carried out on the structured light system, so that the three-dimensional object model information of the target object can be accurately determined subsequently.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a modeling and calibration method and device for a structured light system based on a laser galvanometer. Background Art

[0002] At present, structured light 3D reconstruction technology is widely used in intelligent manufacturing, industrial detection, environmental perception and other fields as a low-cost, high-precision and high-robust 3D reconstruction method. For the calibration of structured light systems, the usual method is: the existing structured light systems based on laser galvanometers are mostly modeled using phase height models, and the mapping relationship between the phase value at a single pixel and the coordinates of a three-dimensional point in space is established through polynomial fitting.

[0003] However, when using the above method to determine value information, the following technical problems often occur:

[0004] First, the calibration process of the entire structured light system requires the introduction of an additional 3D reconstruction system to obtain point cloud coordinates, which increases the complexity of the calibration system and the calibration process. Second, during the calibration process, the position and posture of the calibration plate are required to be high. In order to obtain differentiated calibration data, the position and posture of the calibration plate must be set additionally.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0006] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0007] Some embodiments of the present disclosure propose a modeling and calibration method and device for a structured light system based on a laser galvanometer to solve one or more of the technical problems mentioned in the above background technology section.

[0008] In a first aspect, some embodiments of the present disclosure provide a modeling and calibration method for a structured light system based on a laser galvanometer, comprising: obtaining a first checkerboard image set for different checkerboard postures under projection by a projection device based on a laser galvanometer, wherein the hardware device corresponding to the structured light system based on the laser galvanometer comprises: the above-mentioned projection device based on the laser galvanometer and a shooting device, wherein the above-mentioned projection device based on the laser galvanometer is a device that converts the projector plane to a virtual picture plane corresponding to a pinhole imaging model through a linear mapping matrix to achieve projection; according to the above-mentioned first checkerboard image set, determining the first initial internal parameter information corresponding to the above-mentioned shooting device, the second initial internal parameter information corresponding to the above-mentioned projection device, and the initial internal parameter information between the above-mentioned shooting device and the above-mentioned projection device rotation information and initial translation information; obtaining a second chessboard image set for different encoded images under projection by a projection device, wherein the second chessboard image set is generated based on a target chessboard posture; obtaining an energy generation function, wherein the initial parameter information corresponding to the energy generation function includes: the first initial intrinsic parameter information, the second initial intrinsic parameter information, the initial rotation information, the initial translation information and an initial linear mapping matrix; according to the second chessboard image set, determining the optimal solution under the condition that the function value corresponding to the energy generation function is minimized, so as to generate the first intrinsic parameter information, the second intrinsic parameter information, the initial rotation information, the translation information and the linear mapping matrix as the calibration parameter information of the structured light system based on the laser galvanometer.

[0009] In a second aspect, some embodiments of the present disclosure provide a modeling and calibration device for a structured light system based on a laser galvanometer, comprising: a first acquisition unit, configured to acquire a first checkerboard image set for different checkerboard postures under projection by a projection device based on a laser galvanometer, wherein the corresponding hardware device of the structured light system based on the laser galvanometer comprises: the above-mentioned projection device based on the laser galvanometer and a shooting device, wherein the above-mentioned projection device based on the laser galvanometer is a device that converts the projector plane to the virtual picture plane corresponding to the pinhole imaging model through a linear mapping matrix to realize projection; a determination unit, configured to perform image preprocessing on the above-mentioned food shooting image to generate a first preprocessed image; a second acquisition unit, configured to determine the above-mentioned shooting device according to the above-mentioned first checkerboard image set. The corresponding first initial internal parameter information, the second initial internal parameter information corresponding to the above-mentioned projection device, the initial rotation information and the initial translation information between the above-mentioned shooting device and the above-mentioned projection device; the third acquisition unit is configured to acquire the energy generation function, wherein the initial parameter information corresponding to the above-mentioned energy generation function includes: the above-mentioned first initial internal parameter information, the above-mentioned second initial internal parameter information, the above-mentioned initial rotation information, the above-mentioned initial translation information and the initial linear mapping matrix; the generation unit is configured to determine the optimal solution under the condition of minimizing the corresponding function value of the above-mentioned energy generation function according to the above-mentioned second chessboard image set, so as to generate the first internal parameter information, the second internal parameter information, the initial rotation information, the translation information and the linear mapping matrix as the calibration parameter information of the structured light system based on the laser galvanometer.

[0010] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0011] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner in the first aspect is implemented.

[0012] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the laser galvanometer of some embodiments of the present disclosure, high-precision calibration can be achieved for the structured light system, so as to accurately determine the three-dimensional object model information of the target object later. Specifically, the reason why the relevant structured light system calibration is not accurate enough is that: first, the calibration process of the entire structured light system needs to introduce an additional three-dimensional reconstruction system to obtain the point cloud coordinates, which increases the complexity of the calibration system and the calibration process. Second, during the calibration process, the requirements for the position and posture of the calibration plate are high. And in order to obtain differential calibration data, the position and posture of the calibration plate need to be set additionally. Based on this, the modeling and calibration method of the structured light system based on the laser galvanometer of some embodiments of the present disclosure, first, obtain a first chessboard image set for different chessboard postures under the projection of the projection device based on the laser galvanometer, wherein the corresponding hardware device of the structured light system based on the laser galvanometer includes: the above-mentioned projection device based on the laser galvanometer and the shooting device, wherein the above-mentioned projection device based on the laser galvanometer is a device that converts the projector plane to the virtual screen plane corresponding to the pinhole imaging model through a linear mapping matrix to realize projection. Here, the linear mapping matrix can be used to convert the projector plane to the virtual screen plane under the pinhole imaging model to solve the problem that the pinhole imaging cannot be achieved under the laser galvanometer, and the laser galvanometer can be used to achieve efficient calibration under the structured light system, so as to accurately realize the reconstruction of the three-dimensional object model later. Then, according to the above-mentioned first chessboard image set, the first initial internal parameter information corresponding to the above-mentioned shooting device, the second initial internal parameter information corresponding to the above-mentioned projection device, the initial rotation information and the initial translation information between the above-mentioned shooting device and the above-mentioned projection device can be accurately determined to perform preliminary calibration. Next, the second chessboard image set for different coded images under the projection of the projection device is obtained, wherein the above-mentioned second chessboard image set is generated based on the target chessboard posture, and here, the second chessboard image set is obtained to facilitate the determination of the phase information of the midpoint of the image. Then, the energy generation function is obtained, wherein the initial parameter information corresponding to the above-mentioned energy generation function includes: the above-mentioned first initial internal parameter information, the above-mentioned second initial internal parameter information, the above-mentioned initial rotation information, the above-mentioned initial translation information and the initial linear mapping matrix. Finally, according to the second chessboard image set, the optimal solution under the condition that the corresponding function value of the energy generation function is minimized is determined to accurately and efficiently generate the first intrinsic parameter information, the second intrinsic parameter information, the initial rotation information, the translation information and the linear mapping matrix as the calibration parameter information of the structured light system based on the laser galvanometer. In summary, the generation of the calibration parameter information of the structured light system is achieved by using a projection device based on a laser galvanometer and an energy generation function through a two-stage calibration method. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0014] Figure 1 is a flow chart of some embodiments of a method for modeling and calibrating a structured light system based on a laser galvanometer according to the present disclosure;

[0015] Figure 2 is a schematic structural diagram of some embodiments of a modeling and calibration device for a structured light system based on a laser galvanometer according to the present disclosure;

[0016] Figure 3 is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure;

[0017] Figure 4 is a schematic diagram showing information of each vertex according to a modeling and calibration method of a structured light system based on a laser galvanometer disclosed in the present invention;

[0018] Figure 5 It is a projection schematic diagram showing a projection device based on a laser galvanometer according to a modeling and calibration method of a structured light system based on a laser galvanometer disclosed in the present invention;

[0019] Figure 6 It is a schematic diagram showing the corresponding hardware device of the structured light system based on the laser galvanometer according to the modeling and calibration method of the structured light system based on the laser galvanometer disclosed in the present invention. DETAILED DESCRIPTION

[0020] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0021] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0022] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0023] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0024] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0025] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0026] refer to Figure 1 , shows a process 100 of some embodiments of the modeling and calibration method of the structured light system based on the laser galvanometer according to the present disclosure. The modeling and calibration method of the structured light system based on the laser galvanometer includes the following steps:

[0027] Step 101 : obtaining a first chessboard image set for different chessboard postures projected by a laser galvanometer-based projection device.

[0028] In some embodiments, the execution subject (for example, an electronic device) of the modeling and calibration method of the structured light system based on the laser galvanometer can obtain the first chessboard image set for different chessboard postures under the projection of the projection device based on the laser galvanometer through a wired connection or a wireless connection. Among them, the projection device based on the laser galvanometer can be a device that projects with a laser beam emitted by the laser galvanometer. Each chessboard posture can be a different chessboard posture. Among them, each first chessboard in the first chessboard image set corresponds to a chessboard posture. Among them, the hardware device corresponding to the structured light system based on the laser galvanometer includes: the above-mentioned projection device based on the laser galvanometer and a shooting device. The shooting device can be a camera. Among them, the above-mentioned projection device based on the laser galvanometer is a device that converts the projector plane to the virtual screen plane corresponding to the pinhole imaging model through a linear mapping matrix to realize projection. In practice, considering the working principle of the laser galvanometer, the existing pinhole imaging model is not suitable as a projection model of the laser galvanometer, so the projection plane projected by the laser galvanometer is converted to the virtual screen plane through the virtual screen plane to be projected through the pinhole imaging model. The linear mapping matrix can represent the image mapping relationship between the virtual image plane and the projection image. The projection plane can be a plane corresponding to the two-dimensional coordinate system under the projection device. Each point in the virtual image plane represents the phase information of the corresponding image pixel.

[0029] See also Figure 4 , a schematic diagram of a first checkerboard image set for different checkerboard postures is shown.

[0030] See also Figure 5, shows a projection schematic diagram of a projection device based on a laser galvanometer.

[0031] Among them, R and T in the figure represent the initial rotation information and the initial translation information respectively. P1, P2, P3 and P4 are the vertices in the checkerboard image respectively. U' in the figure is the linear mapping matrix.

[0032] See also Figure 6 , showing a schematic diagram of the hardware device corresponding to the laser galvanometer-based structured light system.

[0033] In some optional implementations of some embodiments, the virtual screen plane is a plane obtained by multiplying the projection plane corresponding to the projection device based on the laser galvanometer by the linear mapping matrix.

[0034] As an example, the expression formula corresponding to the projection plane is multiplied by the linear mapping matrix to obtain the expression formula corresponding to the virtual picture plane.

[0035] Step 102: Determine, based on the first chessboard image set, first initial internal parameter information corresponding to the camera, second initial internal parameter information corresponding to the projection device, initial rotation information and initial translation information between the camera and the projection device.

[0036] In some embodiments, the execution subject may determine the first initial internal parameter information corresponding to the shooting device, the second initial internal parameter information corresponding to the projection device, the initial rotation information and the initial translation information between the shooting device and the projection device based on the first checkerboard image set. Among them, the second initial internal parameter information may characterize the mapping relationship between the three-dimensional coordinates in the three-dimensional coordinate system under the projection device and the two-dimensional coordinates in the two-dimensional coordinate system. The three-dimensional coordinate system is a coordinate system with the projection device as the coordinate center and the shooting direction as the z-axis. The two-dimensional coordinate system and the initial linear mapping matrix may characterize the changes in the phase changes caused by pinhole imaging. The first initial internal parameter information may characterize the mapping relationship between the three-dimensional coordinates in the three-dimensional coordinate system under the shooting device and the two-dimensional coordinates in the two-dimensional coordinate system. The three-dimensional coordinate system under the shooting device is a coordinate system with the shooting device as the coordinate center and the shooting direction as the z-axis. The two-dimensional coordinate system under the shooting device may characterize the changes in the phase changes caused by pinhole imaging. For details, please refer to Figure 5, showing a schematic diagram of each dimensional coordinate system under the shooting device and the projection device. The initial rotation information can represent the rotation mapping relationship between the three-dimensional coordinate system corresponding to the shooting device and the three-dimensional coordinate system corresponding to the above-mentioned projection device. The initial translation information can represent the translation mapping relationship between the three-dimensional coordinate system corresponding to the shooting device and the three-dimensional coordinate system corresponding to the above-mentioned projection device. The first initial internal parameter information, the second initial internal parameter information, the initial rotation information and the initial translation information can be the parameters to be further refined.

[0037] In some optional implementations of some embodiments, the execution subject may determine, based on the first chessboard image set, first initial internal parameter information corresponding to the shooting device, second initial internal parameter information corresponding to the projection device, initial rotation information and initial translation information between the shooting device and the projection device, including the following steps:

[0038] In the first step, for each first chessboard image in the first chessboard image set, the following first generation step is performed:

[0039] Sub-step 1, determining the vertex information set and the first corner point information set corresponding to the first chessboard image. The vertex information may be the vertex pixel coordinates of the chessboard vertices in the first chessboard image. The vertex information set may include: upper left vertex information, lower left vertex information, upper right vertex information, and lower right vertex information. Figure 4 , shows a schematic diagram of each vertex information. The white dots in the figure are vertex information. The first corner point information may be the pixel coordinates of the corner point in the first chessboard image.

[0040] Sub-step 2: determining the second candidate intrinsic parameter information, candidate rotation information and candidate translation information based on the vertex information set.

[0041] As an example, first, according to the vertex information set, the second candidate internal parameter information is determined by the mapping method corresponding to the three-dimensional coordinate system and the two-dimensional coordinate system of the projection device. According to the vertex information set, the third candidate internal parameter information is determined by the mapping method corresponding to the three-dimensional coordinate system and the two-dimensional coordinate system of the shooting device. Then, the rotation relationship and translation relationship between the matrix corresponding to the second candidate internal parameter information and the matrix corresponding to the third candidate internal parameter information are determined to obtain candidate rotation information and candidate translation information.

[0042] Sub-step 3: determining the first candidate intrinsic parameter information according to the first corner point information set.

[0043] As an example, according to the above-mentioned first corner point information set, the first candidate internal parameter information is determined by a mapping method corresponding to the three-dimensional coordinate system and the two-dimensional coordinate system of the shooting device.

[0044] The second step is to generate the second initial internal parameter information according to the obtained second candidate internal parameter information set.

[0045] As an example, the execution subject may select the second candidate internal reference information with the highest number of corresponding repetitions from the second candidate internal reference information set as the second initial internal reference information.

[0046] As another example, the execution entity may perform matrix weighted summation on the second candidate internal parameter matrix set corresponding to the second candidate internal parameter information set to generate the second initial internal parameter information.

[0047] The third step is to generate the above-mentioned initial rotation information and initial internal parameter information according to the obtained candidate rotation information set and candidate translation information set.

[0048] As an example, the execution subject may select the candidate rotation information with the highest number of corresponding repetitions from the candidate rotation information set as the initial rotation information.

[0049] As another example, the execution entity may perform matrix weighted summation on the candidate rotation matrix set corresponding to the candidate rotation information set to generate the initial rotation information.

[0050] As an example, the execution subject may select candidate translation information having the highest number of corresponding repetitions from the candidate translation information set as the initial translation information.

[0051] As another example, the execution entity may perform matrix weighted summation on the candidate translation matrix set corresponding to the candidate translation information set to generate the initial translation information.

[0052] The fourth step is to generate the first initial internal parameter information according to the obtained first candidate internal parameter information set.

[0053] As an example, the execution subject may select the first candidate internal reference information with the highest number of corresponding repetitions from the first candidate internal reference information set as the first initial internal reference information.

[0054] As another example, the execution entity may perform matrix weighted summation on the first candidate internal parameter matrix set corresponding to the first candidate internal parameter information set to generate the first initial internal parameter information.

[0055] Step 103: Acquire a second chessboard image set for different coded images projected by a projection device.

[0056] In some embodiments, the execution subject may obtain a set of second checkerboard images for different coded images projected by a projection device, wherein each coded image may be pre-set, and each second checkerboard image may correspond to a different coded image.

[0057] Step 104, obtaining an energy generation function.

[0058] In some embodiments, the execution subject may obtain an energy generation function. The initial parameter information corresponding to the energy generation function includes: the first initial intrinsic parameter information, the second initial intrinsic parameter information, the initial rotation information, the initial translation information and the initial linear mapping matrix. Among them, the energy generation function may be a function for generating energy information. The energy information may characterize the accuracy of each calibrated parameter information in the structured light system. The lower the energy information, the more accurate the corresponding calibrated parameter information. In practice, the energy generation function may be a function for generating energy information constructed by a relevant large language model based on the current scene.

[0059] Step 105, based on the above-mentioned second chessboard image set, determine the optimal solution under the condition that the corresponding function value of the above-mentioned energy generation function is minimized, so as to generate the first internal parameter information, the second internal parameter information, the initial rotation information, the translation information and the linear mapping matrix as the calibration parameter information of the structured light system based on the laser galvanometer.

[0060] In some embodiments, the above-mentioned execution entity can determine the optimal solution under the condition of minimizing the corresponding function value of the above-mentioned energy generation function based on the above-mentioned second checkerboard image set in various ways to generate first intrinsic parameter information, second intrinsic parameter information, initial rotation information, translation information and linear mapping matrix as calibration parameter information of the structured light system based on the laser galvanometer.

[0061] In some optional implementations of some embodiments, the execution subject may determine the optimal solution under the condition that the corresponding function value of the energy generation function is minimized based on the second chessboard image set to generate first intrinsic parameter information, second intrinsic parameter information, initial rotation information, translation information and a linear mapping matrix as calibration parameter information of a structured light system based on a laser galvanometer, including the following steps:

[0062] The first step is to group the second chessboard images in the second chessboard image set to generate a second chessboard image group set, wherein the second chessboard image group corresponds to at least one image.

[0063] As an example, the execution subject may perform grouping processing on each second chessboard image in the second chessboard image set in a random division manner to generate a second chessboard image group set.

[0064] In the second step, for each second chessboard image group in the second chessboard image group set, the following second generating step is performed:

[0065] Sub-step 1: Select a second chessboard image group from the second chessboard image group set as a target chessboard image group.

[0066] As an example, the execution entity may randomly select a second chessboard image group from the second chessboard image group set as the target chessboard image group.

[0067] Sub-step 2: for the second checkerboard image in the target checkerboard image group, perform the following third generation step:

[0068] The first sub-step is to determine a second corner point information set corresponding to the second chessboard image.

[0069] As an example, the execution entity may determine each corner point information in the second chessboard image as the second corner point information set.

[0070] In the second sub-step, two adjacent second corner point information are selected from the second corner point information set to generate a second corner point information group, thereby obtaining at least one second corner point information group, wherein the two corner point information in the second corner point information group have the same horizontal coordinate or the same vertical coordinate.

[0071] The third sub-step is to determine the phase information corresponding to each second corner point information in the second corner point information set in the virtual picture plane, so as to obtain the two-dimensional coordinate information in the virtual picture plane, and obtain the two-dimensional coordinate information set. The horizontal coordinate and the vertical coordinate in the two-dimensional coordinate information can represent the phase shift degree under the corresponding pinhole imaging model.

[0072] As an example, the above-mentioned execution entity can determine the phase information corresponding to each second corner point information in the above-mentioned second corner point information set under the virtual picture plane in a manner of encoding and decoding the second checkerboard image, so as to obtain the two-dimensional coordinate information under the above-mentioned virtual picture plane and obtain the two-dimensional coordinate information set.

[0073] The fourth sub-step is to input the at least one second corner point information group and the two-dimensional coordinate information set into the energy generation function to generate at least one energy information.

[0074] Sub-step 3, determining whether the obtained energy information group meets the preset energy requirement condition. The energy requirement condition may be a preset requirement condition set for the energy information. For example, the energy requirement condition may be that the standard deviation of each energy information in the energy information group is less than the first target value, or that the variation range is less than the second target value.

[0075] As an example, the execution subject may determine the degree of change of each piece of energy information corresponding to the energy information group to determine whether the degree of change complies with a preset numerical value degree of change.

[0076] Sub-step 4, in response to determining compliance, determines the first initial internal parameter information, the second initial internal parameter information, the initial rotation information, the initial translation information and the initial linear mapping matrix included in the energy generation function as the first internal parameter information, the second internal parameter information, the rotation information, the translation information and the linear mapping matrix.

[0077] In some optional implementations of some embodiments, after determining whether the energy information group obtained meets the preset energy requirement condition, the method further includes:

[0078] The first step, in response to determining non-compliance, is to update the parameters of the first initial internal parameter information, the second initial internal parameter information, the initial rotation information, the initial translation information and the initial linear mapping matrix included in the energy generation function according to the obtained energy information group, so as to generate first updated parameter information, second updated parameter information, updated rotation information, updated translation information and updated linear mapping matrix, and obtain an updated energy generation function.

[0079] As an example, first, the execution subject may determine the energy requirement loss corresponding to the energy information group. Then, according to the energy requirement loss, the gradient descent method is used to update the parameters of the first initial intrinsic parameter information, the second initial intrinsic parameter information, the initial rotation information, the initial translation information and the initial linear mapping matrix included in the energy generation function to generate first updated parameter information, second updated parameter information, updated rotation information, updated translation information and updated linear mapping matrix, and obtain an updated energy generation function.

[0080] The second step is to remove the target chessboard image group from the second chessboard image group set to generate a second chessboard image group set after removal.

[0081] The third step is to select a candidate checkerboard image group from the second checkerboard image group set after the above removal.

[0082] The fourth step is to use the candidate checkerboard image group as the target checkerboard image group and execute the third generation step.

[0083] In some optional implementations of some embodiments, the energy optimization function includes: a spatial point distance restriction function, a relative flatness restriction function, and a rotation information constraint condition. The spatial point distance restriction function may be a function for generating spatial point distances. The relative flatness restriction function may be a function for generating flatness restriction information. The rotation information constraint condition may be a condition for constraining rotation information.

[0084] The energy optimization function can be the following formula:

[0085] E(u c , v c ,u p ; K c , K p , R, T, H) = λ d E d (u c , v c ,u p )+λ p E p (u c , v c ,u p )

[0086] sR T =1

[0087] Among them, K c is the first initial internal reference information. p is the second initial internal parameter information. R: is the initial rotation information. T is the initial translation information. H is the initial linear mapping matrix. u c is the horizontal coordinate corresponding to the corner point information of the camera. c It is the ordinate corresponding to the corner point information of the camera. p It can be the phase information corresponding to the corner point information of the projection device. d (u c , v c ,u p ) can be a spatial point distance restriction function. p (u c , v c ,u p ) can be a relative flatness restriction function. E(u c , v c ,u p ; K c , K p , R, T, H) can be energy information. d It can be the first weight corresponding to the spatial point distance restriction function. p It can be the second weight corresponding to the relative plane restriction function. 1 =1 can be a rotation information constraint condition.

[0088]

[0089] Among them, d i It can be the actual real distance under the i-th second angle information.

[0090]

[0091] Where N represents the number of second corner point information. c , v c ,u p ) can be in u c , v c ,u p The predicted depth information based on z i It can be the actual real depth corresponding to the second corner point information.

[0092] Optionally, the execution subject may input the at least one second corner point information group and the two-dimensional coordinate information set into the energy generation function to generate at least one energy information, including the following steps:

[0093] The first step is to perform the following fourth generation step for each second corner point information group in the at least one second corner point information group:

[0094] Sub-step 1, determining the real point cloud group and the two-dimensional coordinate information group corresponding to the above-mentioned second angle information group. Among them, the second angle information in the second angle information group and the real point cloud in the real point cloud group have a one-to-one correspondence. The real point cloud can be the real point cloud data under the second angle information. The two-dimensional coordinate information in the two-dimensional coordinate information group and the second angle information in the second angle information group have a one-to-one correspondence. The two-dimensional coordinate information can represent the phase transformation under the coordinates corresponding to the second angle information.

[0095] Sub-step 2: determining the actual distance corresponding to the above-mentioned real point cloud group.

[0096] As an example, the execution subject may determine a cosine distance between two real point clouds in the real point cloud group to generate an actual real distance.

[0097] Sub-step 3, using the first initial internal reference information, the initial rotation information, and the initial translation information, determine the shooting 3D coordinate information group corresponding to the 2D coordinate information group, wherein the 2D coordinate information in the 2D coordinate information group and the shooting 3D coordinate information in the shooting 3D coordinate information group have a one-to-one correspondence.

[0098] As an example, first, the first initial internal reference information is used to determine the first three-dimensional coordinate information under the projection device corresponding to the two-dimensional coordinate information. Then, according to the initial translation information and the initial rotation information, the first three-dimensional coordinate information is converted to generate the shooting three-dimensional coordinate information.

[0099] Sub-step 4, determining the coordinate distance corresponding to the above-mentioned three-dimensional coordinate information group.

[0100] Sub-step 5, inputting the above coordinate distance and the above actual real distance into the spatial point distance restriction function to generate the spatial point distance.

[0101] Sub-step 6: determining a predicted depth information set corresponding to the second corner point information set, wherein the second angle information in the second angle information set and the predicted depth information in the predicted depth information set have a one-to-one correspondence.

[0102] Sub-step 7, determining an actual real depth set corresponding to the real point cloud set. The real point cloud set corresponds to the second corner point information set. There is a one-to-one correspondence between the real point cloud in the real point cloud set and the second corner point information in the second corner point information set.

[0103] Sub-step 8: inputting the predicted depth information set and the actual true depth set into a relative flatness restriction function to generate flatness restriction information.

[0104] Sub-step 9: performing weighted sum processing on the above-mentioned spatial point distance and the above-mentioned plane restriction information to generate energy information.

[0105] In some optional implementations of some embodiments, the execution entity may determine the predicted depth information set corresponding to the second corner point information set, including the following steps:

[0106] Determine each second corner point information in the second corner point information set, and perform the following fifth generation step:

[0107] Sub-step 1: determine the two-dimensional coordinate information corresponding to the second corner point information to obtain the target two-dimensional coordinate information.

[0108] Sub-step 2, based on the above-mentioned target two-dimensional coordinate information, the above-mentioned first initial internal parameter information, the above-mentioned second initial internal parameter information and the above-mentioned initial linear mapping matrix, using the depth information generation model to generate predicted depth information. The depth information generation model can be a prediction model for generating depth information. The depth information generation model is a model for obtaining the corresponding depth of a spatial point based on the initial linear mapping matrix and triangulation in the structured light system.

[0109] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the laser galvanometer of some embodiments of the present disclosure, high-precision calibration can be achieved for the structured light system, so as to accurately determine the three-dimensional object model information of the target object later. Specifically, the reason why the relevant structured light system calibration is not accurate enough is that: first, the calibration process of the entire structured light system needs to introduce an additional three-dimensional reconstruction system to obtain the point cloud coordinates, which increases the complexity of the calibration system and the calibration process. Second, during the calibration process, the requirements for the position and posture of the calibration plate are high. And in order to obtain differential calibration data, the position and posture of the calibration plate need to be set additionally. Based on this, the modeling and calibration method of the structured light system based on the laser galvanometer of some embodiments of the present disclosure, first, obtain a first chessboard image set for different chessboard postures under the projection of the projection device based on the laser galvanometer, wherein the corresponding hardware device of the structured light system based on the laser galvanometer includes: the above-mentioned projection device based on the laser galvanometer and the shooting device, wherein the above-mentioned projection device based on the laser galvanometer is a device that converts the projector plane to the virtual screen plane corresponding to the pinhole imaging model through a linear mapping matrix to realize projection. Here, the linear mapping matrix can be used to convert the projector plane to the virtual screen plane under the pinhole imaging model to solve the problem that the pinhole imaging cannot be achieved under the laser galvanometer, and the laser galvanometer can be used to achieve efficient calibration under the structured light system, so as to accurately realize the reconstruction of the three-dimensional object model later. Then, according to the above-mentioned first chessboard image set, the first initial internal parameter information corresponding to the above-mentioned shooting device, the second initial internal parameter information corresponding to the above-mentioned projection device, the initial rotation information and the initial translation information between the above-mentioned shooting device and the above-mentioned projection device can be accurately determined to perform preliminary calibration. Next, the second chessboard image set for different coded images under the projection of the projection device is obtained, wherein the above-mentioned second chessboard image set is generated based on the target chessboard posture, and here, the second chessboard image set is obtained to facilitate the determination of the phase information of the midpoint of the image. Then, the energy generation function is obtained, wherein the initial parameter information corresponding to the above-mentioned energy generation function includes: the above-mentioned first initial internal parameter information, the above-mentioned second initial internal parameter information, the above-mentioned initial rotation information, the above-mentioned initial translation information and the initial linear mapping matrix. Finally, according to the second chessboard image set, the optimal solution under the condition that the corresponding function value of the energy generation function is minimized is determined to accurately and efficiently generate the first intrinsic parameter information, the second intrinsic parameter information, the initial rotation information, the translation information and the linear mapping matrix as the calibration parameter information of the structured light system based on the laser galvanometer. In summary, the generation of the calibration parameter information of the structured light system is achieved by using a projection device based on a laser galvanometer and an energy generation function through a two-stage calibration method.

[0110] Further references Figure 2As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a modeling and calibration device for a structured light system based on a laser galvanometer. These device embodiments are similar to Figure 1 Corresponding to the method embodiments shown, the modeling and calibration device of the structured light system based on the laser galvanometer can be specifically applied to various electronic devices.

[0111] like Figure 2 As shown, a modeling and calibration device 200 of a structured light system based on a laser galvanometer includes: a first acquisition unit 201, a determination unit 202, a second acquisition unit 203, a third acquisition unit 204 and a generation unit 205. The first acquisition unit 201 is configured to acquire a first chessboard image set for different chessboard postures under the projection of a projection device based on a laser galvanometer, wherein the hardware device corresponding to the structured light system based on the laser galvanometer includes: the above-mentioned projection device based on the laser galvanometer and the shooting device, wherein the above-mentioned projection device based on the laser galvanometer is a device that converts the projector plane to the virtual screen plane corresponding to the pinhole imaging model through a linear mapping matrix to achieve projection; the determination unit 202 is configured to perform image preprocessing on the above-mentioned food shooting image to generate a first preprocessed image; the second acquisition unit 203 is configured to determine the first initial internal reference information corresponding to the above-mentioned shooting device and the above-mentioned projection device according to the above-mentioned first chessboard image set. The first embodiment of the present invention relates to a structured light system having a plurality of internal parameters, a plurality of internal parameters, and a plurality of internal parameters. The first embodiment of the present invention relates to a structured light system having a plurality of internal parameters, a plurality of internal parameters, and a plurality of internal parameters. The first embodiment of the present invention relates to a structured light system having a plurality of internal parameters, a plurality of internal parameters, and a plurality of internal parameters. The first embodiment of the present invention relates to a structured light system having a plurality of internal parameters, a plurality of internal parameters, and a plurality of internal parameters. The first embodiment of the present invention relates to a structured light system having a plurality of internal parameters, a plurality of internal parameters, and a plurality of internal parameters. The first embodiment of the present invention relates to a structured light system having a plurality of internal parameters, a plurality of internal parameters, and a plurality of internal parameters.

[0112] It can be understood that the units recorded in the modeling and calibration device 200 of the structured light system based on the laser galvanometer are similar to the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the modeling and calibration device 200 for the structured light system based on the laser galvanometer and the units contained therein, and will not be described in detail here.

[0113] Reference below Figure 3 , which shows a structural schematic diagram of an electronic device (eg, an electronic device) 300 suitable for implementing some embodiments of the present disclosure. Figure 3The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0114] like Figure 3 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0115] Typically, the following devices may be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 309. The communication devices 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0116] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0117] It should be noted that the computer-readable medium in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0118] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0119] The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or it may exist independently without being assembled into the electronic device. The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: obtains a first checkerboard image set for different checkerboard postures under the projection of a laser galvanometer-based projection device, wherein the hardware device corresponding to the laser galvanometer-based structured light system includes: the above-mentioned laser galvanometer-based projection device and shooting device, wherein the above-mentioned laser galvanometer-based projection device is a device that converts the projector plane to the virtual picture plane corresponding to the pinhole imaging model through a linear mapping matrix to realize projection; according to the above-mentioned first checkerboard image set, determines the first initial internal parameter information corresponding to the above-mentioned shooting device, the second initial internal parameter information corresponding to the above-mentioned projection device, and the connection between the above-mentioned shooting device and the above-mentioned projection device. The initial rotation information and initial translation information of the energy generation function are as follows: obtaining a second checkerboard image set for different encoded images under the projection of the projection device, wherein the second checkerboard image set is generated based on the target checkerboard posture; obtaining an energy generation function, wherein the initial parameter information corresponding to the energy generation function includes: the first initial intrinsic parameter information, the second initial intrinsic parameter information, the initial rotation information, the initial translation information and the initial linear mapping matrix; according to the second checkerboard image set, determining the optimal solution under the condition that the function value corresponding to the energy generation function is minimized, so as to generate the first intrinsic parameter information, the second intrinsic parameter information, the initial rotation information, the translation information and the linear mapping matrix as the calibration parameter information of the structured light system based on the laser galvanometer.

[0120] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0121] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0122] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The units described may also be provided in a processor, for example, may be described as: a processor comprising a first acquisition unit, a determination unit, a second acquisition unit, a third acquisition unit, and a generation unit. The names of these units do not, in some cases, constitute limitations on the units themselves, for example, the first acquisition unit may also be described as "a unit for acquiring a first set of checkerboard images for different checkerboard postures projected by a projection device based on a laser galvanometer".

[0123] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0124] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A modeling and calibration method for a structured light system based on a laser galvanometer, comprising: Acquire a first chessboard image set for different chessboard postures under projection by a laser galvanometer-based projection device, wherein the hardware device corresponding to the laser galvanometer-based structured light system includes: the laser galvanometer-based projection device and a shooting device, wherein the laser galvanometer-based projection device is a device that converts the projector plane to the virtual screen plane corresponding to the pinhole imaging model through a linear mapping matrix to realize projection; Determine, according to the first chessboard image set, first initial internal parameter information corresponding to the shooting device, second initial internal parameter information corresponding to the projection device, initial rotation information and initial translation information between the shooting device and the projection device; Acquire a second checkerboard image set for different coded images projected by a projection device, wherein the second checkerboard image set is generated based on a target checkerboard posture; Acquire an energy generation function, wherein the initial parameter information corresponding to the energy generation function includes: the first initial internal parameter information, the second initial internal parameter information, the initial rotation information, the initial translation information and an initial linear mapping matrix; According to the second checkerboard image set, the optimal solution under the condition of minimizing the corresponding function value of the energy generation function is determined to generate first intrinsic parameter information, second intrinsic parameter information, initial rotation information, translation information and a linear mapping matrix as calibration parameter information of the structured light system based on the laser galvanometer.

2. The method according to claim 1, wherein: The determining, according to the first chessboard image set, first initial internal reference information corresponding to the shooting device, second initial internal reference information corresponding to the projection device, initial rotation information and initial translation information between the shooting device and the projection device, comprises: For each first checkerboard image in the first checkerboard image set, the following first generating step is performed: Determine a vertex information set and a first corner point information set corresponding to the first chessboard image; Determining the second candidate intrinsic parameter information, candidate rotation information, and candidate translation information according to the vertex information set; Determining first candidate intrinsic parameter information according to the first corner point information set; Generate the second initial internal parameter information according to the obtained second candidate internal parameter information set; generate the initial rotation information and initial internal parameter information according to the obtained candidate rotation information set and candidate translation information set; The first initial internal parameter information is generated according to the obtained first candidate internal parameter information set.

3. The method according to claim 1, wherein: The method of determining the optimal solution under the condition that the function value corresponding to the energy generation function is minimized according to the second chessboard image set to generate first internal parameter information, second internal parameter information, initial rotation information, translation information and a linear mapping matrix as calibration parameter information of the structured light system based on the laser galvanometer includes: performing grouping processing on each second checkerboard image in the second checkerboard image set to generate a second checkerboard image group set, wherein the number of images corresponding to the second checkerboard image group is at least one; For each second checkerboard image group in the second checkerboard image group set, the following second generating step is performed: Selecting a second checkerboard image group from the second checkerboard image group set as a target checkerboard image group; For the second checkerboard image in the target checkerboard image group, the following third generation step is performed: Determine a second corner point information set corresponding to the second chessboard image; Selecting two adjacent second corner point information from the second corner point information set to generate a second corner point information group, and obtaining at least one second corner point information group; Determine the phase information corresponding to each second corner point information in the second corner point information set on the virtual picture plane to obtain two-dimensional coordinate information on the virtual picture plane, thereby obtaining a two-dimensional coordinate information set; Inputting the at least one second corner point information group and the two-dimensional coordinate information set into the energy generation function to generate at least one energy information; Determine whether the energy information obtained meets the preset energy requirement conditions; In response to determining compliance, the first initial intrinsic parameter information, the second initial intrinsic parameter information, the initial rotation information, the initial translation information and the initial linear mapping matrix included in the energy generation function are determined as first intrinsic parameter information, second intrinsic parameter information, rotation information, translation information and a linear mapping matrix.

4. The method according to claim 3, wherein: After determining whether the energy information group obtained meets the preset energy requirement condition, the method further includes: In response to determining that it does not comply, according to the obtained energy information group, the first initial internal parameter information, the second initial internal parameter information, the initial rotation information, the initial translation information and the initial linear mapping matrix included in the energy generation function are parameter updated to generate first updated parameter information, second updated parameter information, updated rotation information, updated translation information and updated linear mapping matrix, and obtain an updated energy generation function; removing the target checkerboard image group from the second checkerboard image group set to generate a second checkerboard image group set after removal; Selecting a candidate checkerboard image group from the removed second checkerboard image group set; The candidate checkerboard image group is used as the target checkerboard image group to perform the third generation step.

5. The method according to claim 4, wherein: The energy optimization function includes: a space point distance restriction function, a relative flatness restriction function and a rotation information constraint condition; and The step of inputting the at least one second corner point information group and the two-dimensional coordinate information set into the energy generation function to generate at least one energy information comprises: For each second corner point information group in the at least one second corner point information group, the following fourth generating step is performed: Determine a real point cloud group and a two-dimensional coordinate information group corresponding to the second angle information group; Determine the actual real distance corresponding to the real point cloud group; Determine a shooting three-dimensional coordinate information group corresponding to the two-dimensional coordinate information group by using the first initial internal reference information, the second initial internal reference information, the initial rotation information, and the initial translation information; Determine the coordinate distance corresponding to the photographed three-dimensional coordinate information group; Inputting the coordinate distance and the actual real distance into a spatial point distance restriction function to generate a spatial point distance; Determine a predicted depth information set corresponding to the second corner point information set; Determine an actual real depth set corresponding to a real point cloud set, wherein the real point cloud set corresponds to the second corner point information set; Inputting the predicted depth information set and the actual true depth set into a relative flatness restriction function to generate flatness restriction information; A weighted sum process is performed on the spatial point distance and the plane restriction information to generate energy information.

6. The method according to claim 1, wherein: The virtual picture plane is a plane obtained by multiplying the projection plane corresponding to the projection device based on the laser galvanometer by the linear mapping matrix.

7. The method according to claim 5, wherein: The determining the predicted depth information set corresponding to the second corner point information set includes: Determine each second corner point information in the second corner point information set, and perform the following fifth generation step: Determine the two-dimensional coordinate information corresponding to the second corner point information to obtain the target two-dimensional coordinate information; A depth information generation model is used to generate predicted depth information according to the target two-dimensional coordinate information, the first initial internal parameter information, the second initial internal parameter information and the initial linear mapping matrix.

8. A modeling and calibration device for a structured light system based on a laser galvanometer, comprising: A first acquisition unit is configured to acquire a first chessboard image set for different chessboard postures under projection by a laser galvanometer-based projection device, wherein the hardware device corresponding to the laser galvanometer-based structured light system includes: the laser galvanometer-based projection device and a shooting device, wherein the laser galvanometer-based projection device is a device that converts a projector plane to a virtual screen plane corresponding to a pinhole imaging model through a linear mapping matrix to achieve projection; a determining unit configured to perform image preprocessing on the food photographed image to generate a first preprocessed image; a second acquisition unit, configured to determine, according to the first chessboard image set, first initial internal parameter information corresponding to the shooting device, second initial internal parameter information corresponding to the projection device, initial rotation information and initial translation information between the shooting device and the projection device; A third acquisition unit is configured to acquire an energy generation function, wherein the initial parameter information corresponding to the energy generation function includes: the first initial intrinsic parameter information, the second initial intrinsic parameter information, the initial rotation information, the initial translation information and an initial linear mapping matrix; A generating unit is configured to determine, based on the second checkerboard image set, an optimal solution under the condition that the corresponding function value of the energy generating function is minimized, so as to generate first intrinsic parameter information, second intrinsic parameter information, initial rotation information, translation information and a linear mapping matrix as calibration parameter information of a structured light system based on a laser galvanometer.

9. An electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images